From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov  1 14:38:38 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 1 Nov 2000 16:38:38 +0200 (IST)
Subject: [Python-Dev] Bug Changes
Message-ID: <Pine.GSO.4.10.10011011635030.21805-200000@sundial>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

---559023410-2110444415-973089518=:21805
Content-Type: TEXT/PLAIN; charset=US-ASCII

I've noticed the SF-FAQ still has the old "Use Jitterbug" thing about
bugs, even though we've moved to SourceForge bug manager. Attached is
a patch to correct everything. 

I haven't checked it in, because I'm not sure I my explanations are
clear at all. I'd be happy if someone went over it and saw if it's all
right.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez

---559023410-2110444415-973089518=:21805
Content-Type: TEXT/PLAIN; charset=US-ASCII; name=file
Content-ID: <Pine.GSO.4.10.10011011638380.21805@sundial>
Content-Description: 
Content-Disposition: attachment; filename=file
Content-Transfer-Encoding: BASE64

SW5kZXg6IHNmLWZhcS5odG1sDQo9PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpS
Q1MgZmlsZTogL2N2c3Jvb3QvcHl0aG9uL3B5dGhvbi9ub25kaXN0L3NmLWh0
bWwvc2YtZmFxLmh0bWwsdg0KcmV0cmlldmluZyByZXZpc2lvbiAxLjE1DQpk
aWZmIC1jIC1yMS4xNSBzZi1mYXEuaHRtbA0KKioqIHNmLWZhcS5odG1sCTIw
MDAvMDgvMjUgMDc6NTU6NDgJMS4xNQ0KLS0tIHNmLWZhcS5odG1sCTIwMDAv
MTEvMDEgMTQ6MzM6NTANCioqKioqKioqKioqKioqKg0KKioqIDU3LDYzICoq
KioNCiAgPGgyPjxhIGhyZWY9IiNidWdzIj42LiBCdWcgUmVwb3J0aW5nPC9h
PjwvaDI+DQogIDxvbD4NCiAgICA8bGk+PGEgaHJlZj0iI2IxIj5XaGVyZSBj
YW4gSSBzdWJtaXQvdmlldyBidWdzIGZvciBQeXRob24/PC9hPjwvbGk+DQoh
ICAgPGxpPjxhIGhyZWY9IiNiMiI+SG93IGRvIEkgdXNlIGppdHRlcmJ1Zz88
L2E+PC9saT4NCiAgPC9vbD4NCiAgDQogIDxoMj48YSBocmVmPSIjYXBwZW5k
aXgiPkEuIEFwcGVuZGl4PC9hPjwvaDI+DQotLS0gNTcsNjMgLS0tLQ0KICA8
aDI+PGEgaHJlZj0iI2J1Z3MiPjYuIEJ1ZyBSZXBvcnRpbmc8L2E+PC9oMj4N
CiAgPG9sPg0KICAgIDxsaT48YSBocmVmPSIjYjEiPldoZXJlIGNhbiBJIHN1
Ym1pdC92aWV3IGJ1Z3MgZm9yIFB5dGhvbj88L2E+PC9saT4NCiEgICA8bGk+
PGEgaHJlZj0iI2IyIj5Ib3cgZG8gSSB1c2UgdGhlIHNvdXJjZWZvcmdlIGJ1
ZyBtYW5hZ2VyPzwvYT48L2xpPg0KICA8L29sPg0KICANCiAgPGgyPjxhIGhy
ZWY9IiNhcHBlbmRpeCI+QS4gQXBwZW5kaXg8L2E+PC9oMj4NCioqKioqKioq
KioqKioqKg0KKioqIDQ0Niw0NzIgKioqKg0KICA8aDQ+UTogV2hlcmUgY2Fu
IEkgc3VibWl0L3ZpZXcgYnVncyBmb3IgUHl0aG9uPzwvaDQ+DQogIA0KICA8
aDQ+QTo8L2g0Pg0KISBBcyBvZiBub3cgWzI1LUp1bC0yMDBdIHRoZSBQeXRo
b24gcHJvamVjdCBkb2VzIG5vdCB1c2UgU291cmNlRm9yZ2UncyBidWcNCiEg
dHJhY2tpbmcgZmFjaWxpdHkuIFRoaXMgbWF5IGNoYW5nZSB3aGVuIHRoZXJl
IGlzIGEgd2F5IHRvIGltcG9ydCB0aGUgZXhpc3RpbmcNCiEgaml0dGVyYnVn
IGRhdGFiYXNlLiBUaGUgaml0dGVyYnVnIGRhdGFiYXNlIGNhbiBiZSBhY2Nl
c3NlZCB0aHJvdWdoIHRoZQ0KISBmb2xsb3dpbmcgaW50ZXJmYWNlOg0KICAN
Ci0gPGJsb2NrcXVvdGU+DQotICAgPHR0PjxhDQotICAgaHJlZj0iaHR0cDov
L3d3dy5weXRob24ub3JnL3B5dGhvbi1idWdzIj5odHRwOi8vd3d3LnB5dGhv
bi5vcmcvcHl0aG9uLWJ1Z3M8L2E+PC90dD48L2Jsb2NrcXVvdGU+DQotIA0K
ICA8aDM+PGEgbmFtZT0iYjEiIGlkPSJiMSI+PC9hPjYuMS46PC9oMz4NCiAg
DQohIDxoND5ROiBIb3cgZG8gSSB1c2Ugaml0dGVyYnVnPzwvaDQ+DQogIA0K
ICA8aDQ+QTo8L2g0Pg0KISBUbyBnZXQgdGhlIGxpc3Qgb2Ygb3BlbiBidWdz
IGNsaWNrIG9uIDx0dD5vcGVuPC90dD4gKGhpZGRlbiBiZXR3ZWVuIHRoZSBz
ZWNvbmQNCiEgbGFzdCBhbmQgdGhlIGxhc3QgaG9yaXpvbnRhbCBydWxlciku
DQogIA0KISA8cD5UbyBnZXQgYSBsaXN0IG9mIHRoZSBidWdzIHdoaWNoIGFy
ZSByZWxhdGVkIHRvIHNvbWUgYXJlYSwgZW50ZXIgYW4NCiEgYXBwcm9wcmlh
dGUgcmVndWxhciBleHByZXNzaW9uIGFuZCBwcmVzcyAiU2VsZWN0IE1lc3Nh
Z2VzIi4gVGhlbiBzZWxlY3QNCiEgPHR0Pm9wZW48L3R0PiAob3Igd2hhdGV2
ZXIgY2F0ZWdvcnkgeW91IHdvdWxkIGxpa2UgdG8gdmlldykgYXMgZGVzY3Jp
YmVkDQohIGFib3ZlLjwvcD4NCiAgDQogIDxoMT48YSBuYW1lPSJhcHBlbmRp
eCIgaWQ9ImFwcGVuZGl4Ij48L2E+QS4gQXBwZW5kaXg8L2gxPg0KICANCi0t
LSA0NDYsNDY2IC0tLS0NCiAgPGg0PlE6IFdoZXJlIGNhbiBJIHN1Ym1pdC92
aWV3IGJ1Z3MgZm9yIFB5dGhvbj88L2g0Pg0KICANCiAgPGg0PkE6PC9oND4N
CiEgVGhlIFB5dGhvbiBwcm9qZWN0IHVzZXMgU291cmNlRm9yZ2UncyBidWcN
CiEgdHJhY2tpbmcgZmFjaWxpdHkuIEdvIHRvIA0KISA8YSBocmVmPSJodHRw
Oi8vc291cmNlZm9yZ2UubmV0L2J1Z3MvP2dyb3VwX2lkPTU0NzAiPmh0dHA6
Ly9zb3VyY2Vmb3JnZS5uZXQvYnVncy8/Z3JvdXBfaWQ9NTQ3MDwvYT4gZm9y
IGFsbCBidWcgbWFuYWdlbWVudCBuZWVkcy4NCiAgDQogIDxoMz48YSBuYW1l
PSJiMSIgaWQ9ImIxIj48L2E+Ni4xLjo8L2gzPg0KICANCiEgPGg0PlE6IEhv
dyBkbyBJIHVzZSB0aGUgc291cmNlZm9yZ2UgYnVnIG1hbmFnZXI/PC9oND4N
CiAgDQogIDxoND5BOjwvaDQ+DQohIEJ5IGRlZmF1bHQsIHlvdSB3aWxsIHNl
ZSB0aGUgbGlzdCBvZiBhbGwgT3BlbiBidWdzLiBZb3UgY2FuIGNoYW5nZQ0K
ISB3aGljaCBidWdzIHlvdSdyZSB2aWV3aW5nIGJ5IHNlbGVjdGluZyB0aGUg
YXNzaWduZWRfdG8vc3RhdHVzL2FyZWEvdHlwZQ0KISBzZWxlY3QgYm94cy4N
CiAgDQohIDxwPlRvIHN1Ym1pdCBhIGJ1ZywgdXNlIHRoZSAiU3VibWl0IGEg
QnVnIiBsaW5rLCBuZWFyIHRoZSB0b3Agb2YgdGhlIHBhZ2UuDQohIDwvcD4N
CiAgDQogIDxoMT48YSBuYW1lPSJhcHBlbmRpeCIgaWQ9ImFwcGVuZGl4Ij48
L2E+QS4gQXBwZW5kaXg8L2gxPg0KICANCioqKioqKioqKioqKioqKg0KKioq
IDYwMCw2MTYgKioqKg0KICAgICAgbWFpbGluZyBsaXN0IGFkZHJlc3M7IHRo
aXMgYWRkcmVzcyBzaG91bGQgbm8gbG9uZ2VyIGJlIHVzZWQgZm9yIHBhdGNo
DQogICAgICBzdWJtaXNzaW9uLiBUaGUgcGF0Y2ggbWFuYWdlciBpcyBmb3Ig
PGI+cGF0Y2hlczwvYj4gb25seTsgaWYgeW91IGhhdmUgYQ0KICAgICAgcHJv
YmxlbSBvciBzdWdnZXN0aW9uIGJ1dCBkb24ndCBrbm93IGhvdyB0byB3cml0
ZSB0aGUgY29kZSBmb3IgaXQsIHVzZSB0aGUNCiEgICAgIDxhIGhyZWY9Imh0
dHA6Ly93d3cucHl0aG9uLm9yZy9zZWFyY2gvc2VhcmNoX2J1Z3MuaHRtbCI+
UHl0aG9uIEJ1Z3MNCiEgICAgIExpc3Q8L2E+IGluc3RlYWQuIFRoZSBidWdz
IGxpc3QgaXMgc2VhcmNoYWJsZTsgaWYgeW91IGhhdmUgYSBwcm9ibGVtIGFu
ZA0KICAgICAgeW91J3JlIG5vdCBzdXJlIGlmIGl0IGhhcyBiZWVuIHJlcG9y
dGVkIG9yIGZpeGVkIGFscmVhZHksIHRoaXMgaXMgdGhlDQogICAgICBmaXJz
dCBwbGFjZSB0byBsb29rLiAoVGhlcmUgdXNlZCB0byBiZSBhIHNlcGFyYXRl
IFRPRE8gbGlzdDsgd2Ugbm93IHByZWZlcg0KICAgICAgdGhhdCB5b3UgdXNl
IHRoZSBidWdzIGxpc3QgZm9yIHN1Z2dlc3Rpb25zIGFuZCByZXF1ZXN0cyB0
b28uKTwvbGk+DQogICAgPGxpIHN0eWxlPSJsaXN0LXN0eWxlOiBub25lIj48
Yj5TdWJtaXQgZG9jdW1lbnRhdGlvbiBwYXRjaGVzIHRoZSBzYW1lDQogICAg
ICB3YXkuPC9iPiBXaGVuIGFkZGluZyB0aGUgcGF0Y2gsIGJlIHN1cmUgdG8g
c2V0IHRoZSAiPGI+Q2F0ZWdvcnk8L2I+IiBmaWVsZA0KICAgICAgdG8gIjxi
PmRvY3VtZW50YXRpb248L2I+Ii4gRm9yIGRvY3VtZW50YXRpb24gZXJyb3Jz
IHdpdGhvdXQgcGF0Y2hlcywNCiEgICAgIHBsZWFzZSB1c2UgdGhlIDxhDQoh
ICAgICBocmVmPSJodHRwOi8vd3d3LnB5dGhvbi5vcmcvc2VhcmNoL3NlYXJj
aF9idWdzLmh0bWwiPlB5dGhvbiBCdWdzIExpc3Q8L2E+DQohICAgICBpbnN0
ZWFkLjwvbGk+DQogICAgPGxpPldlIGxpa2UgY29udGV4dCBkaWZmcy4gV2Ug
Z3J1ZGdpbmdseSBhY2NlcHQgdW5pZmllZCBkaWZmcy4gPGI+U3RyYWlnaHQN
CiAgICAgICgiZWQtc3R5bGUiKSBkaWZmcyBhcmUgcmlnaHQgb3V0ITwvYj4g
SWYgeW91IGRvbid0IGtub3cgaG93IHRvIGdlbmVyYXRlDQogICAgICBjb250
ZXh0IGRpZmZzLCB5b3UncmUgcHJvYmFibHkgbm90IHF1YWxpZmllZCB0byBw
cm9kdWNlIGhpZ2gtcXVhbGl0eQ0KLS0tIDU5NCw2MDggLS0tLQ0KICAgICAg
bWFpbGluZyBsaXN0IGFkZHJlc3M7IHRoaXMgYWRkcmVzcyBzaG91bGQgbm8g
bG9uZ2VyIGJlIHVzZWQgZm9yIHBhdGNoDQogICAgICBzdWJtaXNzaW9uLiBU
aGUgcGF0Y2ggbWFuYWdlciBpcyBmb3IgPGI+cGF0Y2hlczwvYj4gb25seTsg
aWYgeW91IGhhdmUgYQ0KICAgICAgcHJvYmxlbSBvciBzdWdnZXN0aW9uIGJ1
dCBkb24ndCBrbm93IGhvdyB0byB3cml0ZSB0aGUgY29kZSBmb3IgaXQsIHVz
ZSB0aGUNCiEgICAgIDxhIGhyZWY9IiNiMSI+YnVnIHJlcG9ydGluZyBtZWNo
YW5pc208L2E+IGluc3RlYWQuIA0KISAgICAgVGhlIGJ1Z3MgbGlzdCBpcyBz
ZWFyY2hhYmxlOyBpZiB5b3UgaGF2ZSBhIHByb2JsZW0gYW5kDQogICAgICB5
b3UncmUgbm90IHN1cmUgaWYgaXQgaGFzIGJlZW4gcmVwb3J0ZWQgb3IgZml4
ZWQgYWxyZWFkeSwgdGhpcyBpcyB0aGUNCiAgICAgIGZpcnN0IHBsYWNlIHRv
IGxvb2suIChUaGVyZSB1c2VkIHRvIGJlIGEgc2VwYXJhdGUgVE9ETyBsaXN0
OyB3ZSBub3cgcHJlZmVyDQogICAgICB0aGF0IHlvdSB1c2UgdGhlIGJ1Z3Mg
bGlzdCBmb3Igc3VnZ2VzdGlvbnMgYW5kIHJlcXVlc3RzIHRvby4pPC9saT4N
CiAgICA8bGkgc3R5bGU9Imxpc3Qtc3R5bGU6IG5vbmUiPjxiPlN1Ym1pdCBk
b2N1bWVudGF0aW9uIHBhdGNoZXMgdGhlIHNhbWUNCiAgICAgIHdheS48L2I+
IFdoZW4gYWRkaW5nIHRoZSBwYXRjaCwgYmUgc3VyZSB0byBzZXQgdGhlICI8
Yj5DYXRlZ29yeTwvYj4iIGZpZWxkDQogICAgICB0byAiPGI+ZG9jdW1lbnRh
dGlvbjwvYj4iLiBGb3IgZG9jdW1lbnRhdGlvbiBlcnJvcnMgd2l0aG91dCBw
YXRjaGVzLA0KISAgICAgcGxlYXNlIHVzZSB0aGUgPGEgaHJlZj0iYjEiPmJ1
ZyByZXBvcnRpbmcgbWVjaGFuaXNtPC9hPi48L2xpPg0KICAgIDxsaT5XZSBs
aWtlIGNvbnRleHQgZGlmZnMuIFdlIGdydWRnaW5nbHkgYWNjZXB0IHVuaWZp
ZWQgZGlmZnMuIDxiPlN0cmFpZ2h0DQogICAgICAoImVkLXN0eWxlIikgZGlm
ZnMgYXJlIHJpZ2h0IG91dCE8L2I+IElmIHlvdSBkb24ndCBrbm93IGhvdyB0
byBnZW5lcmF0ZQ0KICAgICAgY29udGV4dCBkaWZmcywgeW91J3JlIHByb2Jh
Ymx5IG5vdCBxdWFsaWZpZWQgdG8gcHJvZHVjZSBoaWdoLXF1YWxpdHkNCg==
---559023410-2110444415-973089518=:21805--


From mwh21@cam.ac.uk  Wed Nov  1 18:13:07 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 01 Nov 2000 18:13:07 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Jeremy Hylton's message of "Mon, 30 Oct 2000 09:59:00 -0500 (EST)"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>
Message-ID: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>

Jeremy Hylton <jeremy@alum.mit.edu> writes:

> >>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:
> 
>   MAL> Jeremy Hylton wrote:
>   >>
>   >> Update of /cvsroot/python/python/dist/src/Python In directory
>   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
>   >>
>   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
>   >> implementation is getting really hairy; should review it to see
>   >> if it can be simplified.
> 
>   MAL> How about a complete redesign of the whole call mechanism ?!
>
[chomp]
> 
> I'd be interested in looking at it.

Random idea that occurred while answering a post on comp.lang.python:

How about dumping the CALL_FUNCTION* opcodes, and replacing them with
two non-argumented opcodes, called for the sake of argument NCALL_FUNC
and NCALL_FUNC_KW.

NCALL_FUNC would pop a function object and a tuple off the stack and
apply the function to the tuple.

NCALL_FUNC_KW would do the same, then pop a dictionary and then do
the moral equivalent of f(*args,**kw).

As a preliminary it would be sensible to rework BUILD_MAP so that it
built dictionaries off the stack (a bit like BUILD_LIST, and like
CALL_FUNCTION now does with keyword arguments...) (and extend the
compiler to use this for literal dictionaries).

This would add an opcode or so per function call, but would probably
make life much simpler.

No time for implementation tonight, but could probably knock something
up tomorrow (depending how hard it turns out to be).

Thoughts?  Is that like what you did, Marc?

M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp



From jeremy@alum.mit.edu  Wed Nov  1 19:06:46 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:06:46 -0500 (EST)
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
References: <200010301715.JAA32564@slayer.i.sourceforge.net>
 <39FDB5EA.B60EA39A@lemburg.com>
 <14845.36020.17063.951147@bitdiddle.concentric.net>
 <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <14848.27078.923932.758419@bitdiddle.concentric.net>

My first impression is that this sounds like a nice simplifcation.

One question is how expensive this is for the common case.  Right now
arguments are pushed on the interpreter stack before CALL_FUNCTION is
executed, which is just a pointer assignment.  The pointers on the
stack are then assigned into the fast locals of the function after the
call.  Your scheme sounds like it would increase all function calls by
the cost of a tuple allocation.

It certainly wouldn't hurt to implement this, as it would provide some
practical implementation experience that would inform a PEP on the
subject.

On a related note, I have proposed a pep to add nested lexical scopes
for Python 2.1.  Barry's away for the moment, so it hasn't been
assigned a number yet.  It's just a proposal, not sure what Guido will
say in the end, but it also involves revising the function call
architecture.  I'll send a copy of the current draft (just notes)
under a separate subject.

Jeremy


From jeremy@alum.mit.edu  Wed Nov  1 19:07:10 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:07:10 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
Message-ID: <14848.27102.223001.369662@bitdiddle.concentric.net>

Title: Statically Nested Scopes
Author: Jeremy Hylton <jeremy@digicool.com>
Status: Draft
Type: Standards Track
Created: 01-Nov-2000

Abstract

    This PEP proposes the additional of statically nested scoping
    (lexical scoping) for Python 2.1.  The current language definition
    defines exactly three namespaces that are used to resolve names --
    the local, global, and built-in namespaces.  The addition of
    nested scopes would allow resolution of unbound local names in
    enclosing functions' namespaces.

    One consequence of this change that will be most visible to Python
    programs is that lambda statements could reference variables in
    the namespaces where the lambda is defined.  Currently, a lambda
    statement uses default arguments to explicitly creating bindings
    in the lambda's namespace.

Notes

    This section describes several issues that will be fleshed out and
    addressed in the final draft of the PEP.  Until that draft is
    ready, please direct comments to the author.

    This change has been proposed many times in the past.  It has
    always been stymied by the possibility of creating cycles that
    could not be collected by Python's reference counting garbage
    collector.  The additional of the cycle collector in Python 2.0
    eliminates this concern.

    Guido once explained that his original reservation about nested
    scopes was a reaction to their overuse in Pascal.  In large Pascal
    programs he was familiar with, block structure was overused as an
    organizing principle for the program, leading to hard-to-read
    code. 

    Greg Ewing developed a proposal "Python Nested Lexical Scoping
    Enhancement" in Aug. 1999.  It is available from
    http://www.cosc.canterbury.ac.nz/~greg/python/lexscope.html

    Michael Hudson's bytecodehacks projects at
    http://sourceforge.net/projects/bytecodehacks/
    provides facilities to support nested scopes using the closure
    module. 

    Examples:

    def make_adder(n):
        def adder(x):
            return x + n
        return adder
    add2 = make_adder(2)
    add2(5) == 7

    
    from Tkinter import *
    root = Tk()
    Button(root, text="Click here",
           command = lambda : root.test.configure(text="..."))

    
    One controversial issue is whether it should be possible to modify
    the value of variables defined in an enclosing scope.  

    One part of the issue is how to specify that an assignment in the
    local scope should reference to the definition of the variable in
    an enclosing scope.  Assignment to a variable in the current scope
    creates a local variable in the scope.  If the assignment is
    supposed to refer to a global variable, the global statement must
    be used to prevent a local name from being created.  Presumably,
    another keyword would be required to specify "nearest enclosing
    scope."

    Guido is opposed to allow modifications (need to clarify exactly
    why).  If you are modifying variables bound in enclosing scopes,
    you should be using a class, he says.

    The problem occurs only when a program attempts to rebind the name
    in the enclosing scope.  A mutable object, e.g. a list or
    dictionary, can be modified by a reference in a nested scope; this
    is an obvious consequence of Python's reference semantics.  The
    ability to change mutable objects leads to an inelegant
    workaround: If a program needs to rebind an immutable object,
    e.g. a number or tuple, store the object in a list and have all
    references to the object use this list:

    def bank_account(initial_balance):
        balance = [initial_balance]
        def deposit(amount):
            balance[0] = balance[0] + amount
        def withdraw(amount):
            balance[0] = balance[0] - amount
        return deposit, withdraw

    I would prefer for the language to support this style of
    programming directly rather than encouraging programs to use this
    somewhat obfuscated style.  Of course, an instance would probably
    be clearer in this case.
 
    One implementation issue is how to represent the environment that
    stores variables that are referenced by nested scopes.  One
    possibility is to add a pointer to each frame's statically
    enclosing frame and walk the chain of links each time a non-local
    variable is accessed.  This implementation has some problems,
    because access to nonlocal variables is slow and causes garbage to
    accumulate unncessarily.  Another possibility is to construct an
    environment for each function that provides access to only the
    non-local variables.  This environment would be explicitly passed
    to nested functions.


From mal@lemburg.com  Wed Nov  1 20:31:08 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:31:08 +0100
Subject: [Python-Dev] Python Call Mechanism
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <3A007D8C.A9943D90@lemburg.com>

Michael Hudson wrote:
> 
> Jeremy Hylton <jeremy@alum.mit.edu> writes:
> 
> > >>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:
> >
> >   MAL> Jeremy Hylton wrote:
> >   >>
> >   >> Update of /cvsroot/python/python/dist/src/Python In directory
> >   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
> >   >>
> >   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
> >   >> implementation is getting really hairy; should review it to see
> >   >> if it can be simplified.
> >
> >   MAL> How about a complete redesign of the whole call mechanism ?!
> >
> [chomp]
> >
> > I'd be interested in looking at it.
> 
> Random idea that occurred while answering a post on comp.lang.python:
> 
> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).
> 
> As a preliminary it would be sensible to rework BUILD_MAP so that it
> built dictionaries off the stack (a bit like BUILD_LIST, and like
> CALL_FUNCTION now does with keyword arguments...) (and extend the
> compiler to use this for literal dictionaries).
> 
> This would add an opcode or so per function call, but would probably
> make life much simpler.
> 
> No time for implementation tonight, but could probably knock something
> up tomorrow (depending how hard it turns out to be).
> 
> Thoughts?  Is that like what you did, Marc?

No, I just cleaned up the intertwine calling scheme currently
implemented in ceval.c. This allows a few improvments, one of
them being the possibility to inline C function calls in the
main loop (anyone ever trace the path Python takes when calling
a builtin function or method... you'd be surprised).

About your idea with the new opcodes: you could be touching
a performance relevant section there -- a ceval round-trip
may cost more than the added if()s in the CALL_FUNCION opcode.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Wed Nov  1 20:37:12 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:37:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <3A007EF8.2D9BDCF5@lemburg.com>

[pre-PEP]

This will break code... I'm not sure whether it's worth going
down this path just for the sake of being able to define
functions within functions.

Wouldn't it be a better idea to somehow add native acqusition
to Python's objects ?

We already have a slot which implements the "contains"
relationship. All we'd need is a way for a contained
object to register itself with the container in a 
way that doesn't produce cycles.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From jeremy@alum.mit.edu  Wed Nov  1 20:48:53 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 15:48:53 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A007EF8.2D9BDCF5@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <3A007EF8.2D9BDCF5@lemburg.com>
Message-ID: <14848.33205.361821.679508@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.

How will this break code?  Any code written to use the scoping rules
will not work today.

Python already allows programs to define functions within functions.
That's not at issue.  The issue is how hard it is to use nested
functions, including lambdas.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?

No.

Seriously, I don't see how acquistion addresses the same issues at
all.  Feel free to explain what you mean.

  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.

The contains relationship has to do with container objects and their
contents.  A function's environment is not a container in the same
sense, so I don't see how this is related.

As I noted in the PEP, I don't see a compelling reason to avoid
cycles.

Jeremy


From mal@lemburg.com  Wed Nov  1 20:51:16 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:51:16 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <3A007EF8.2D9BDCF5@lemburg.com> <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <3A008244.8F558C8B@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:
> 
>   MAL> [pre-PEP] This will break code... I'm not sure whether it's
>   MAL> worth going down this path just for the sake of being able to
>   MAL> define functions within functions.
> 
> How will this break code?  Any code written to use the scoping rules
> will not work today.
> 
> Python already allows programs to define functions within functions.
> That's not at issue.  The issue is how hard it is to use nested
> functions, including lambdas.

The problem is that with nested scoping, a function defined
within another function will suddenly reference the variables
of the enclosing function as globals and not the module
globals... this could break code.

Another problem is that you can't reach out for the defining
module globals anymore (at least not in an easy way like today).

>   MAL> Wouldn't it be a better idea to somehow add native acqusition
>   MAL> to Python's objects ?
> 
> No.
> 
> Seriously, I don't see how acquistion addresses the same issues at
> all.  Feel free to explain what you mean.

It's not related to *statically* nested scopes, but is to
dynamically nested ones. Acquisition is basically about the
same thing: you acquire attributes from containers. The only
difference here is that the containment relationships are
defined at run-time.
 
>   MAL> We already have a slot which implements the "contains"
>   MAL> relationship. All we'd need is a way for a contained object to
>   MAL> register itself with the container in a way that doesn't
>   MAL> produce cycles.
> 
> The contains relationship has to do with container objects and their
> contents.  A function's environment is not a container in the same
> sense, so I don't see how this is related.
> 
> As I noted in the PEP, I don't see a compelling reason to avoid
> cycles.

Ok, we have cycle GC, but why create cycles when you don't
have to (Python can also be run without GC and then you'd run
into problems...) ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From skip@mojam.com (Skip Montanaro)  Wed Nov  1 21:57:17 2000
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 1 Nov 2000 15:57:17 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <3A007EF8.2D9BDCF5@lemburg.com>
 <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <14848.37309.3616.710295@beluga.mojam.com>

    MAL> [pre-PEP] This will break code...

    Jeremy> How will this break code?

Suppose you have

    x = 1
    def f1():
        x = 2
        def inner():
            print x
        inner()

Today, calling f1() prints "1".  After your proposed changes I suspect it
would print "2".

Skip


From jeremy@alum.mit.edu  Wed Nov  1 21:18:49 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:18:49 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <3A007EF8.2D9BDCF5@lemburg.com>
 <14848.33205.361821.679508@bitdiddle.concentric.net>
 <3A008244.8F558C8B@lemburg.com>
Message-ID: <14848.35001.109767.606384@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.
  >>
  >> How will this break code?  Any code written to use the scoping
  >> rules will not work today.
  >>
  >> Python already allows programs to define functions within
  >> functions.  That's not at issue.  The issue is how hard it is to
  >> use nested functions, including lambdas.

  MAL> The problem is that with nested scoping, a function defined
  MAL> within another function will suddenly reference the variables
  MAL> of the enclosing function as globals and not the module
  MAL> globals... this could break code.

That's right it could, in the unlikely case that someone has existing
code today using nested functions where an intermediate function
defines a local variable that shadows a variable defined in an
enclosing scope.  

It should be straightfoward to build a tool that would detect this
case.

It would be pretty poor programming style, so I think it would be fine
to break backwards compatibility here.

  MAL> Another problem is that you can't reach out for the defining
  MAL> module globals anymore (at least not in an easy way like
  MAL> today).

I think we would want to keep globals implemented just the way they
are.  The compiler would need to determine exactly which variables are
access from enclosing scopes and which are globals.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?
  >>
  >> Seriously, I don't see how acquistion addresses the same issues
  >> at all.  Feel free to explain what you mean.

  MAL> It's not related to *statically* nested scopes, but is to
  MAL> dynamically nested ones. Acquisition is basically about the
  MAL> same thing: you acquire attributes from containers. The only
  MAL> difference here is that the containment relationships are
  MAL> defined at run-time.

Static scoping and dynamic scoping are entirely different beasts,
useful for different things.  I want to fix, among other things,
lambdas.  That's a static issue.
 
  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.
  >>
  >> The contains relationship has to do with container objects and
  >> their contents.  A function's environment is not a container in
  >> the same sense, so I don't see how this is related.
  >>
  >> As I noted in the PEP, I don't see a compelling reason to avoid
  >> cycles.

  MAL> Ok, we have cycle GC, but why create cycles when you don't have
  MAL> to (Python can also be run without GC and then you'd run into
  MAL> problems...) ?

If we can avoid cycles, sure.  I would prefer a simple design that
allowed cycles to a complicated design that avoided them.  Exactly
where to draw the line between simple and complex is a matter of
taste, of course.

Jeremy


From jeremy@alum.mit.edu  Wed Nov  1 21:20:08 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:20:08 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.37309.3616.710295@beluga.mojam.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <3A007EF8.2D9BDCF5@lemburg.com>
 <14848.33205.361821.679508@bitdiddle.concentric.net>
 <14848.37309.3616.710295@beluga.mojam.com>
Message-ID: <14848.35080.73182.888834@bitdiddle.concentric.net>

Thanks.  I expect there is very little code that depends on this sort
of behavior, since it is confusing to read.  Many readers,
particularly novices, could reasonably expect Python to print 2 now.

As I explained to MAL, I think we would need to provide a code
analysis tool that identified these problems.  It's probably helpful
to generate warning about this right now, since it's rather
obfuscated.

Jeremy


From jeremy@alum.mit.edu  Wed Nov  1 22:35:14 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:35:14 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
Message-ID: <14848.39586.832800.139182@bitdiddle.concentric.net>

I just did a clean configure and make from the latest CVS tree.  It
seems to get stuck in a loop calling makesetup over and over again.
(Output below.)

Any idea what's going wrong?

Jeremy

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
making Makefile in subdirectory .
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3'
making Makefile in subdirectory Parser
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
making Makefile in subdirectory Grammar
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
make[1]: Nothing to be done for `Makefile'.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
making Makefile in subdirectory Objects
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
making Makefile in subdirectory Python
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Python'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Python'
making Makefile in subdirectory Modules
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------

[etc.]


From thomas@xs4all.net  Wed Nov  1 22:41:34 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Nov 2000 23:41:34 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <200011012237.OAA06642@slayer.i.sourceforge.net>; from twouters@users.sourceforge.net on Wed, Nov 01, 2000 at 02:37:39PM -0800
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
Message-ID: <20001101234134.O12812@xs4all.nl>

On Wed, Nov 01, 2000 at 02:37:39PM -0800, Thomas Wouters wrote:

> Modified Files:
> 	pep-0204.txt 
> Log Message:

> Update this PEP to current, harsh, reality. It's been rejected :) If at all
> possible, the reasoning should be extended to include the real reasons it
> was rejected -- this is just guesswork from my side. (This means you, Guido,
> or anyone who can channel Guido enough to call himself Guido.)

In addition to that, PEP 0 also needs to be updated. Shall I do that myself,
now that Barry is apparently away ? While I was at it, I also noticed PEP
0200 still says 'Incomplete', though that might be by design.

Yay-another-first-for-me---first-rejected-PEP-ly y'rs, ;)
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From jeremy@alum.mit.edu  Wed Nov  1 22:54:07 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:54:07 -0500 (EST)
Subject: [Python-Dev] Move idle PEPs to status deferred
Message-ID: <14848.40719.914227.606929@bitdiddle.concentric.net>

Barry,

We should start working on any PEPs that are going to be considered
for Python 2.1.  There are a bunch of old PEPs that have been
assigned and then ignored.  You have marked many of them as deferred.
We should assign new authors for the PEPs we care about and move all
the other deferred PEPs somewhere else in the index.

The following PEPs have been inactive for at least two months:
 I     2  pep-0002.txt  Procedure for Adding New Modules       Raymond
 S   202  pep-0202.txt  List Comprehensions                    Peters
 SD  205  pep-0205.txt  Weak References                        Drake
 I   206  pep-0206.txt  2.0 Batteries Included                 Zadka
 SD  207  pep-0207.txt  Rich Comparisons                       Ascher
 SD  208  pep-0208.txt  Reworking the Coercion Model           Ascher
 SD  209  pep-0209.txt  Adding Multidimensional Arrays         Ascher
 SD  210  pep-0210.txt  Decoupling the Interpreter Loop        Ascher
 SD  211  pep-0211.txt  Adding New Linear Algebra Operators    Wilson
 SD  212  pep-0212.txt  Loop Counter Iteration                 Schneider-Kamp
 SD  213  pep-0213.txt  Attribute Access Handlers              Prescod
 SD  215  pep-0215.txt  String Interpolation                   Yee
 I   216  pep-0216.txt  Docstring Format                       Zadka
 SD  217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 SD  218  pep-0218.txt  Adding a Built-In Set Object Type      Wilson
 SD  219  pep-0219.txt  Stackless Python                       McMillan
 I   220  pep-0220.txt  Coroutines, Generators, Continuations  McMillan
 S   224  pep-0224.txt  Attribute Docstrings                   Lemburg

I think we should find new authors for PEPs 207 and 208 and work on
them for 2.1.  I imagine David would be happy to pass the torch on
these issues.

I assume Gordon will be working on the stackless PEPs, but we should
confirm this.

For all of the other PEPs, authors who want to have them considered
for 2.1 should provide some updates to their PEPs.

We should also simplfy the PEP index so that deferred PEPs are
collected at the bottom or something like that.

Jeremy


From greg@cosc.canterbury.ac.nz  Thu Nov  2 00:34:04 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:34:04 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <200011020034.NAA29444@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy@alum.mit.edu>:

> Seriously, I don't see how acquistion addresses the same issues at
> all.

My proposal for nested scopes was actually an acquisition-like
mechanism. The idea was to avoid unbreakable cycles by deferring
the creation of a closure from when the function is defined
to when it is used.

Guido rejected my implementation for various good reasons.
It could be modified to overcome most of those objections,
however.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Thu Nov  2 00:37:00 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:37:00 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
Message-ID: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal@lemburg.com>:

> The problem is that with nested scoping, a function defined
> within another function will suddenly reference the variables
> of the enclosing function

This could be avoided by requiring that variables which are
to be visible in an inner scope be marked somehow in the
scope where they are defined.

I don't think it's a serious enough problem to be worth
fixing that way, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From gmcm@hypernet.com  Thu Nov  2 01:37:09 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 1 Nov 2000 20:37:09 -0500
Subject: [Python-Dev] Move idle PEPs to status deferred
In-Reply-To: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <3A007EF5.21540.6BA02C9@localhost>

Jeremy wrote:

> I assume Gordon will be working on the stackless PEPs, but we should
> confirm this.

Yes, I will.

- Gordon


From tim_one@email.msn.com  Thu Nov  2 07:02:04 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 02:02:04 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

Note that this problem will be much worse in Python:  in Pascal, you could
always "look up" for the closest-containing func/proc that explicitly
declares a referenced vrbl.  In Python, you have to indirectly *deduce*
which vrbls are local to a def, by searching the entire body for an
appearance as a binding target.  So you have to "look up" and "look down"
from the reference point, and it's easy to miss a binding target.

    i = 6
    def f(x):
        def g():
            print i
        # ...
        # skip to the next page
        # ...
        for i in x:  # ah, i *is* local to f, so this is what g sees
            pass
        g()

>    def bank_account(initial_balance):
>        balance = [initial_balance]
>        def deposit(amount):
>            balance[0] = balance[0] + amount
>        def withdraw(amount):
>            balance[0] = balance[0] - amount
>        return deposit, withdraw

Unfortunately for proponents, this is exactly the kind of SICP example that
is much better done via a class.  Not only is the closure version strained
by comparison, but as is usual it manages to create a bank account with a
write-only balance <0.9 wink>.

    def deposit(amount):
        global bank_account.balance
        balance += amount

is one old suggested way to explicitly declare non-local names and the
enclosing block to which they are local (and in analogy with current
"global", mandatory if you want to rebind the non-local name, optional if
you only want to reference it).  There are subtleties here, but explicit is
better than implicit, and the subtleties are only subtler if you refuse
(like Scheme) to make the intent explicit.

for-real-fun-think-about-"exec"-abuses-ly y'rs  - tim




From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 08:32:11 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 10:32:11 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

[Tim Peters]
> Note that this problem will be much worse in Python:  in Pascal, you could
> always "look up" for the closest-containing func/proc that explicitly
> declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> which vrbls are local to a def, by searching the entire body for an
> appearance as a binding target.  So you have to "look up" and "look down"
> from the reference point, and it's easy to miss a binding target.

This is a tool problem, and should be solved with good tools.
Of course, installing the corret tools in people's minds will require
some technological discoveries.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From gstein@lyra.org  Thu Nov  2 08:46:04 2000
From: gstein@lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 00:46:04 -0800 (PST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
Message-ID: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>

On Thu, 2 Nov 2000, Moshe Zadka wrote:
> [Tim Peters]
> > Note that this problem will be much worse in Python:  in Pascal, you could
> > always "look up" for the closest-containing func/proc that explicitly
> > declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> > which vrbls are local to a def, by searching the entire body for an
> > appearance as a binding target.  So you have to "look up" and "look down"
> > from the reference point, and it's easy to miss a binding target.
> 
> This is a tool problem, and should be solved with good tools.
> Of course, installing the corret tools in people's minds will require
> some technological discoveries.

Bleck. Those tools are a crutch to deal with a poor language design /
feature. And are those tools portable? Are they part of everybody's
standard tool set? Will vi, emacs, and MS DevStudio all have those
capabilities?

Not a chance. Personally, I'll take Guido's point of view and say they are
inherently hard to deal with; therefore, punt them.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/




From mal@lemburg.com  Thu Nov  2 12:12:42 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 13:12:42 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
Message-ID: <3A015A3A.4EC7DBC6@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com>:
> 
> > The problem is that with nested scoping, a function defined
> > within another function will suddenly reference the variables
> > of the enclosing function
> 
> This could be avoided by requiring that variables which are
> to be visible in an inner scope be marked somehow in the
> scope where they are defined.
> 
> I don't think it's a serious enough problem to be worth
> fixing that way, though.

It may not look serious, but changing the Python lookup scheme
is, since many inspection tools rely and reimplement exactly
that scheme. With nested scopes, there would be next to no
way to emulate the lookups using these tools.

To be honest, I don't think static nested scopes buy us all that
much. You can do the same now, by using keyword arguments which
isn't all that nice, but works great and makes the scope clearly
visible.

Dynamic nested scopes is another topic... those are *very*
useful; especially when it comes to avoiding global variables
and implementing programs which work using control objects
instead of global function calls.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From barry@wooz.org  Thu Nov  2 14:30:26 2000
From: barry@wooz.org (barry@wooz.org)
Date: Thu, 2 Nov 2000 09:30:26 -0500 (EST)
Subject: [Python-Dev] Re: Move idle PEPs to status deferred
References: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <14849.31362.805142.553781@anthem.concentric.net>

I was away from email for most of the day yesterday.  I'll do a swing
through all the outstanding PEP updates later today -- after I finish
catching up on email. :(

-Barry



From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 14:49:37 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 16:49:37 +0200 (IST)
Subject: [Python-Dev] PEP-0217
Message-ID: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>

I need some help:

1) BDFL pronouncement
2) someone to see about the Jython issue.

Thank you for your co-operation.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From jeremy@alum.mit.edu  Thu Nov  2 15:18:47 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 10:18:47 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
References: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
 <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
Message-ID: <14849.34263.310260.404940@bitdiddle.concentric.net>

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

  GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
  >> This is a tool problem, and should be solved with good tools.  Of
  >> course, installing the corret tools in people's minds will
  >> require some technological discoveries.

  GS> Bleck. Those tools are a crutch to deal with a poor language
  GS> design / feature. And are those tools portable? Are they part of
  GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
  GS> all have those capabilities?

Are you saying that compilers are a crutch and we should get rid of
them?  I don't think you intend that, but this is a completely
straightforward tool to build.  It is needed only for backwards
compatibility -- to identify scripts that depend on the changed
behavior.  There is no need for vi, emacs, or devstudio to understand
what's going on.

Jeremy


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 15:16:41 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:16:41 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.34263.310260.404940@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> >>>>> "GS" == Greg Stein <gstein@lyra.org> writes:
> 
>   GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
>   >> This is a tool problem, and should be solved with good tools.  Of
>   >> course, installing the corret tools in people's minds will
>   >> require some technological discoveries.
> 
>   GS> Bleck. Those tools are a crutch to deal with a poor language
>   GS> design / feature. And are those tools portable? Are they part of
>   GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
>   GS> all have those capabilities?
> 
> Are you saying that compilers are a crutch and we should get rid of
> them?  I don't think you intend that, but this is a completely
> straightforward tool to build.  It is needed only for backwards
> compatibility -- to identify scripts that depend on the changed
> behavior.  There is no need for vi, emacs, or devstudio to understand
> what's going on.

you guys are talking about different things.
Jeremy is talking about a tool to warn against incompatible changes
Greg is talking about a tool to identify, for each variable, what scope
it belongs to.

as-usual-the-answer-is-"you're-both-right"-ly y'rs, Z.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Thu Nov  2 03:26:14 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:26:14 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
 <3A015A3A.4EC7DBC6@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
 <3A015A3A.4EC7DBC6@lemburg.com>
Message-ID: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>

[MAL]
> Dynamic nested scopes is another topic... those are *very*
> useful; especially when it comes to avoiding global variables
> and implementing programs which work using control objects
> instead of global function calls.

Marc-Andre, what are Dynamic nested scopes?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 15:26:20 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:26:20 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> [MAL]
> > Dynamic nested scopes is another topic... those are *very*
> > useful; especially when it comes to avoiding global variables
> > and implementing programs which work using control objects
> > instead of global function calls.
> 
> Marc-Andre, what are Dynamic nested scopes?

If MAL means dynamic scoping (which I understood he does), then this 
simply means:

when looking for a variable "foo", you first search for it in the local
namespace. If not there, the *caller's* namespace, and so on. In the
end, the caller is the __main__ module, and if not found there, it is
a NameError.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Thu Nov  2 03:29:03 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:29:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
 <3A015A3A.4EC7DBC6@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
 <3A015A3A.4EC7DBC6@lemburg.com>
Message-ID: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>

> It may not look serious, but changing the Python lookup scheme
> is, since many inspection tools rely and reimplement exactly
> that scheme. With nested scopes, there would be next to no
> way to emulate the lookups using these tools.

So fix the tools.

> To be honest, I don't think static nested scopes buy us all that
> much. You can do the same now, by using keyword arguments which
> isn't all that nice, but works great and makes the scope clearly
> visible.

Yes.  It's a hack that gets employed over and over.  And it has
certain problems.  We added 'import as' to get rid of a common
practice that was perceived unclean.  Maybe we should support nested
scopes to get rid of another unclean common practice?

I'm not saying that we definitely should add this to 2.1 (there's
enough on our plate already) but we should at least consider it, and
now that we have cycle GC, the major argument against it (that it
causes cycles) is gone...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 15:29:55 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:29:55 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

This is the perfect moment to ask: what do we have on our plates for 2.1?
Shouldn't we have a list of goals for it or something? As a first-order
approximation, what PEPs are expected to be included? And, most the
conspiracy-theory question, what are Digital Creations' goals for Python?

We now return you to our regularily scheduled bug fixing.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From akuchlin@mems-exchange.org  Thu Nov  2 15:34:42 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 10:34:42 -0500
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 02, 2000 at 05:29:55PM +0200
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>
Message-ID: <20001102103442.B5027@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
>Shouldn't we have a list of goals for [Python 2.1] or something? As a 
>first-order approximation, what PEPs are expected to be included? And, 

Stuff I personally want to get done:
  * Finish PEP 222, "Web Programming Improvements" and implement whatever
    emerges from it.

  * Write a PEP on using Distutils to build the modules that come with
    Python, and implement it if accepted.

  * Work on something CPAN-like.  This may or may not have repercussions for
    the core; I don't know.

--amk


From thomas@xs4all.net  Thu Nov  2 15:42:39 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 16:42:39 +0100
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102103442.B5027@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Thu, Nov 02, 2000 at 10:34:42AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <20001102164239.R12812@xs4all.nl>

On Thu, Nov 02, 2000 at 10:34:42AM -0500, Andrew Kuchling wrote:

>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Probably not, though perhaps a new module would be nice. As for the
CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
Apachecon, and I think it would be suited extremely well as the transmission
protocol for SPAM (or however you want to call the Python CPAN ;). You can
do the uploading, downloading and searching for modules using WebDAV without
too much pain, and there's excellent WebDAV support for Apache ;)

Is anyone working on something like this, or even thinking about it ? I'm
not deep enough into distutils to join that SIG, but I definately would join
a CPyAN SIG ;)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From bckfnn@worldonline.dk  Thu Nov  2 15:37:43 2000
From: bckfnn@worldonline.dk (Finn Bock)
Date: Thu, 02 Nov 2000 15:37:43 GMT
Subject: [Python-Dev] PEP-0217
In-Reply-To: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
Message-ID: <3a0186d5.25979536@smtp.worldonline.dk>

[Moshe Zadka]

>I need some help:
>
>1) BDFL pronouncement
>2) someone to see about the Jython issue.

I don't see any problems with this. This is already handled by a method
in the jython runtime (Py.printResult).

However, I think your example implementation should be:

    def displayhook(o):
        if o is None:
            return
        __builtin__._ = None
        print `o`
        __builtin__._ = o

I don't why, but that is the implementation currently used by Jython
(and I think CPython too).

regards,
finn


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 16:05:20 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:05:20 +0200 (IST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <Pine.GSO.4.10.10011021804210.1290-100000@sundial>

On Thu, 2 Nov 2000, Finn Bock wrote:

> However, I think your example implementation should be:
> 
>     def displayhook(o):
>         if o is None:
>             return
>         __builtin__._ = None
>         print `o`
>         __builtin__._ = o
> 

You're right.
I'll just add the necessary Jython changes to PEP-0217.
Thanks a lot.

I don't like this either, but the good news is that in Py2.1, you'll
be able to change this in site.py <wink>

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Thu Nov  2 16:05:14 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:05:14 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <3A0190BA.940FFADA@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 1 Nov 2000, Guido van Rossum wrote:
> 
> > [MAL]
> > > Dynamic nested scopes is another topic... those are *very*
> > > useful; especially when it comes to avoiding global variables
> > > and implementing programs which work using control objects
> > > instead of global function calls.
> >
> > Marc-Andre, what are Dynamic nested scopes?
> 
> If MAL means dynamic scoping (which I understood he does), then this
> simply means:
> 
> when looking for a variable "foo", you first search for it in the local
> namespace. If not there, the *caller's* namespace, and so on. In the
> end, the caller is the __main__ module, and if not found there, it is
> a NameError.

That would be one application, yes.

With dynamic scoping I meant that the context of a lookup is
defined at run-time and by explicitely or implicitely
hooking together objects which then define the nesting.

Environment acquisition is an example of such dynamic scoping:
attribute lookups are passed on to the outer scope in case they
don't resolve on the inner scope, e.g. say you have
object a with a.x = 1; all other objects don't define .x.
Then a.b.c.d.x will result in lookups
1. a.b.c.d.x
2. a.b.c.x
3. a.b.x
4. a.x -> 1

This example uses attribute lookup -- the same can be done for
other nested objects by explicitely specifying the nesting
relationship.

Jim's ExtensionClasses allow the above by using a lot of
wrappers around objects -- would be nice if we could come
up with a more general scheme which then also works for
explicit nesting relationships (e.g. dictionaries which
get hooked together -- Jim's MultiMapping does this).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Thu Nov  2 16:06:38 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:06:38 +0100
Subject: [Python-Dev] Python 2.1 tasks
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <3A01910E.A19FFBB2@lemburg.com>

Andrew Kuchling wrote:
> 
> On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
> >Shouldn't we have a list of goals for [Python 2.1] or something? As a
> >first-order approximation, what PEPs are expected to be included? And,
> 
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.
> 
>   * Write a PEP on using Distutils to build the modules that come with
>     Python, and implement it if accepted.
> 
>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Most important for 2.1 are probably:

1. new C level coercion scheme
2. rich comparisons
3. making the std lib Unicode compatible

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From jeremy@alum.mit.edu  Thu Nov  2 16:11:30 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:11:30 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <14849.37426.860007.989619@bitdiddle.concentric.net>

>>>>> "TP" == Tim Peters <tim_one@email.msn.com> writes:

  TP> [Jeremy Hylton]
  >> ...  Guido once explained that his original reservation about
  >> nested scopes was a reaction to their overuse in Pascal.  In
  >> large Pascal programs he was familiar with, block structure was
  >> overused as an organizing principle for the program, leading to
  >> hard-to-read code.

  TP> Note that this problem will be much worse in Python: in Pascal,
  TP> you could always "look up" for the closest-containing func/proc
  TP> that explicitly declares a referenced vrbl.  In Python, you have
  TP> to indirectly *deduce* which vrbls are local to a def, by
  TP> searching the entire body for an appearance as a binding target.
  TP> So you have to "look up" and "look down" from the reference
  TP> point, and it's easy to miss a binding target.

I agree that visual inspection is a tad harder, but I contend that
existing programs that use the same name for a global variable and a
local variable -- and intend for the global to be visible within a
function nested in the local variable's region -- are confusing.  
It's too hard for a first-time reader of the code to figure out what
is going on.

Incidentally, I have yet to see an example of this problem occurring
in anyone's code.  All the examples seem a bit contrived.  I wonder if
anyone has an example in existing code.

  [My SICP example omitted]

  TP> Unfortunately for proponents, this is exactly the kind of SICP
  TP> example that is much better done via a class.

Indeed, the PEP says exactly that: This kind of program is better
done via a class.  My intent was not to show a compelling use of
mutable state.  Instead it was to show that with read-only access,
people could still modify values on enclosing scopes.  The issue is
whether the language allows the programmer to express this intent
clearly or if she has to jump through some hoops to accomplish it.

  TP>                                                Not only is the
  TP> closure version strained by comparison, but as is usual it
  TP> manages to create a bank account with a write-only balance <0.9
  TP> wink>.

  TP>     def deposit(amount):
  TP>         global bank_account.balance balance += amount

  TP> is one old suggested way to explicitly declare non-local names
  TP> and the enclosing block to which they are local (and in analogy
  TP> with current "global", mandatory if you want to rebind the
  TP> non-local name, optional if you only want to reference it).
  TP> There are subtleties here, but explicit is better than implicit,
  TP> and the subtleties are only subtler if you refuse (like Scheme)
  TP> to make the intent explicit.

I'm still not sure I like it, because it mixes local variables of a
function with attribute access on objects.  I'll add it to the
discussion in the PEP (if Barry approves the PEP <wink>), though.

Do you have any opinion on the subtleties?  The two that immediately
come to mind are: 1) whether the function's local are available as
attributes anywhere or only in nested scopes and 2) whether you can
create new local variable using this notation.

Jeremy


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  2 16:09:08 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:09:08 +0200 (IST)
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102164239.R12812@xs4all.nl>
Message-ID: <Pine.GSO.4.10.10011021807060.1290-100000@sundial>

On Thu, 2 Nov 2000, Thomas Wouters wrote:

> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

Cries for this sig have been already made in c.l.py.
I'm moving this discussion to meta-sig. Please discuss it there.
I'm willing to champion it, but I'll defer if Andrew or Greg want
to do it.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From jeremy@alum.mit.edu  Thu Nov  2 16:13:52 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:13:52 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A015A3A.4EC7DBC6@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
 <3A015A3A.4EC7DBC6@lemburg.com>
Message-ID: <14849.37568.510427.834971@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:

  MAL> It may not look serious, but changing the Python lookup scheme
  MAL> is, since many inspection tools rely and reimplement exactly
  MAL> that scheme. With nested scopes, there would be next to no way
  MAL> to emulate the lookups using these tools.

Can you say more about this issue?  It sounds like it is worth
discussing in the PEP, but I can't get a handle on exactly what the
problem is.  Any tool needs to implement or model Python's name
resolution algorithm, call it algorithm A.  If we change name
resolution to use algorithm B, then the tools need to implement or
model a new algorithm.  I don't see where the impossibility of
emulation comes in.

Jeremy



From guido@python.org  Thu Nov  2 04:24:29 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:24:29 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "01 Nov 2000 18:13:07 GMT."
 <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>
 <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <200011020424.XAA07810@cj20424-a.reston1.va.home.com>

> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).

No, this is a bad idea.  Long, long ago, all calls requird building a
tuple for the arguments first.  This tuple creation turned out to be a
major bottleneck.  That's why the current call opcode exists.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry@wooz.org  Thu Nov  2 16:22:32 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:22:32 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38088.903104.936944@anthem.concentric.net>

If we get lexical scoping, there should be a fast (built-in) way to
get at all the accessible names from Python.  I.e. currently I can do

    d = globals().copy()
    d.update(locals())

and know that `d' contains a dictionary of available names, with the
right overloading semantics.  (PEP 42 now includes a feature request
to make vars() do this by default.)

-Barry


From barry@wooz.org  Thu Nov  2 16:23:56 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:23:56 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38172.908316.107381@anthem.concentric.net>

This has been added as PEP 227.


From guido@python.org  Thu Nov  2 04:28:00 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:28:00 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "Wed, 01 Nov 2000 14:06:46 EST."
 <14848.27078.923932.758419@bitdiddle.concentric.net>
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
 <14848.27078.923932.758419@bitdiddle.concentric.net>
Message-ID: <200011020428.XAA07841@cj20424-a.reston1.va.home.com>

> It certainly wouldn't hurt to implement this, as it would provide some
> practical implementation experience that would inform a PEP on the
> subject.

If it solves the mess with supporting extended call syntax, adding
these opcodes might be a good idea.

But as I said, for the normal (not extended) case, the existing
CALL_FUNCTION opcode is the right thing to use unless you want things
to slow down significantly.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@wooz.org  Thu Nov  2 16:31:35 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:31:35 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
 <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <14849.38631.997377.600214@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:

    MZ> If MAL means dynamic scoping (which I understood he does),
    MZ> then this simply means:

    MZ> when looking for a variable "foo", you first search for it in
    MZ> the local namespace. If not there, the *caller's* namespace,
    MZ> and so on. In the end, the caller is the __main__ module, and
    MZ> if not found there, it is a NameError.

This is how Emacs Lisp behaves, and it's used all the time in ELisp
programs.  On the one hand it's quite convenient for customizing the
behavior of functions.  On the other hand, it can make documenting the
interface of functions quite difficult because all those dynamically
scoped variables are now part of the function's API.

It's interesting to note that many ELispers really hate dynamic
scoping and pine for a move toward lexical scoping.  I'm not one of
them.

I'm not as concerned about "fixing" nested functions because I hardly
ever use them, and rarely see them much in Python code.  Fixing
lambdas would be nice, but since Guido considers lambdas themselves a
mistake, and given that lamda use /can/ be a performance hit in some
situations, does it make sense to change something as fundamental as
Python's scoping rules to fix this eddy of the language?

-Barry


From jeremy@alum.mit.edu  Thu Nov  2 16:36:45 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:36:45 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <3A0190BA.940FFADA@lemburg.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
 <3A0190BA.940FFADA@lemburg.com>
Message-ID: <14849.38941.59576.682495@bitdiddle.concentric.net>

Moshe's explanation of "dynamic scope" is the definition I've seen in
every programming language text I've ever read.  The essence of the
defintion, I believe, is that a free variable is resolved in the
environment created by the current procedure call stack.

I think it muddles the discussion to use "dynamic scope" to describe
acquistion, though it is a dynamic feature.

Python using dynamic scope for exceptions.  If any exception is
raised, the exception handler that is triggered is determined by the
environment in which the procedure was called.

There are few languages that use dynamic scoping for normal name
resolution.  Many early Lisp implementations did, but I think all the
modern ones use lexical scoping instead.  It is hard to write modular
code using dynamic scope, because the behavior of a function with free
variables can not be determined by the module that defines it.  Not
saying it isn't useful, just that it makes it much harder to reason
about how a particular modular or function works in isolation from the
rest of the system.

Jeremy


From jeremy@alum.mit.edu  Thu Nov  2 16:38:04 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:38:04 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
 <Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.39020.532855.494649@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:

  MZ> you guys are talking about different things.  Jeremy is talking
  MZ> about a tool to warn against incompatible changes Greg is
  MZ> talking about a tool to identify, for each variable, what scope
  MZ> it belongs to.

Not sure we're talking about different things.  The compiler will need
to determine the scope of each variable.  It's a tool.  If it
implements the specifiction for name binding, other tools can too.

Jeremy


From barry@wooz.org  Thu Nov  2 16:36:11 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:36:11 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
 <Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.38907.808752.186414@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez@math.huji.ac.il> writes:

    MZ> you guys are talking about different things.  Jeremy is
    MZ> talking about a tool to warn against incompatible changes Greg
    MZ> is talking about a tool to identify, for each variable, what
    MZ> scope it belongs to.

And Greg's point is well taken, because it /will/ be harder to tell at
a glance where a name is coming from, so programming tools will have
to find ways to help with this.

-Barry


From barry@wooz.org  Thu Nov  2 16:44:06 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:44:06 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
 <20001101234134.O12812@xs4all.nl>
Message-ID: <14849.39382.960359.909365@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas@xs4all.net> writes:

    >> Update this PEP to current, harsh, reality. It's been rejected
    >> :) If at all possible, the reasoning should be extended to
    >> include the real reasons it was rejected -- this is just
    >> guesswork from my side. (This means you, Guido, or anyone who
    >> can channel Guido enough to call himself Guido.)

    TW> In addition to that, PEP 0 also needs to be updated. Shall I
    TW> do that myself, now that Barry is apparently away ?

I've just done it.
    
    TW> While I was at it, I also noticed PEP 0200 still says
    TW> 'Incomplete', though that might be by design.

I've updated both these too, thanks.

-Barry


From mal@lemburg.com  Thu Nov  2 16:45:12 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:45:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
 <3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>
Message-ID: <3A019A18.20D12FE1@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal@lemburg.com> writes:
> 
>   MAL> It may not look serious, but changing the Python lookup scheme
>   MAL> is, since many inspection tools rely and reimplement exactly
>   MAL> that scheme. With nested scopes, there would be next to no way
>   MAL> to emulate the lookups using these tools.
> 
> Can you say more about this issue?  It sounds like it is worth
> discussing in the PEP, but I can't get a handle on exactly what the
> problem is.  Any tool needs to implement or model Python's name
> resolution algorithm, call it algorithm A.  If we change name
> resolution to use algorithm B, then the tools need to implement or
> model a new algorithm.  I don't see where the impossibility of
> emulation comes in.

Well first you'd have to change all tools to use the new
scheme (this includes debuggers, inspection tools, reflection
kits, etc.). This certainly is not a smart thing to do since
Python IDEs are just starting to appear -- you wouldn't want
to break all those.

What get's harder with the nested scheme is that
you can no longer be certain that globals() reaches out to
the module namespace. But this is needed by some lazy evaluation
tools. Writing to globals() would not be defined anymore --
where should you bind the new variable ?

Another problem is that there probably won't be a way to access
all the different nesting levels on a per-level basis (could be
that I'm missing something here, but debugging tools would need
some sort of scope() builtin to access the different scopes).
I'm not sure whether this is possible to do without some sort
of link between the scopes. We currently don't need such links.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From barry@wooz.org  Thu Nov  2 16:52:50 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:52:50 -0500 (EST)
Subject: [Python-Dev] PEP-0217
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
 <3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <14849.39906.374949.489679@anthem.concentric.net>

>>>>> "FB" == Finn Bock <bckfnn@worldonline.dk> writes:

    FB> I don't see any problems with this. This is already handled by
    FB> a method in the jython runtime (Py.printResult).

Yep, should be easy to add to Jython.
-Barry


From mwh21@cam.ac.uk  Thu Nov  2 17:00:37 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:00:37 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Wed, 01 Nov 2000 23:24:29 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com>
Message-ID: <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido@python.org> writes:

> > How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> > two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> > and NCALL_FUNC_KW.
> > 
> > NCALL_FUNC would pop a function object and a tuple off the stack and
> > apply the function to the tuple.
> > 
> > NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> > the moral equivalent of f(*args,**kw).
> 
> No, this is a bad idea.  Long, long ago, all calls requird building a
> tuple for the arguments first.  This tuple creation turned out to be a
> major bottleneck.  That's why the current call opcode exists.

Yes, I realize this now.  I made my suggestion having not actually
looked at the code or thought about it very much.

I still think there is (or at least might be) value in rewriting the
code for the more complex cases, and moving the dictionary creation
out of the implementation of CALL_FUNCTION.

Then the following could be made essentially equivalent:

f(a,b,c=d,e=f)

dict = {c:d,e:f}
f(a,b,**dict)

(modulo evaluation order).

(I've made some changes to get BUILD_MAP using it's argument and
construct literal dictionaries using it, which I'll whack up onto sf
just as soon as I get round to it... ah, it's at

https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102227&group_id=5470

).

Cheers,
M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp



From gstein@lyra.org  Thu Nov  2 17:07:29 2000
From: gstein@lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 09:07:29 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <14849.39382.960359.909365@anthem.concentric.net>; from barry@wooz.org on Thu, Nov 02, 2000 at 11:44:06AM -0500
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>
Message-ID: <20001102090729.A1874@lyra.org>

On Thu, Nov 02, 2000 at 11:44:06AM -0500, Barry A. Warsaw wrote:
> 
> >>>>> "TW" == Thomas Wouters <thomas@xs4all.net> writes:
> 
>     >> Update this PEP to current, harsh, reality. It's been rejected
>     >> :) If at all possible, the reasoning should be extended to
>     >> include the real reasons it was rejected -- this is just
>     >> guesswork from my side. (This means you, Guido, or anyone who
>     >> can channel Guido enough to call himself Guido.)
> 
>     TW> In addition to that, PEP 0 also needs to be updated. Shall I
>     TW> do that myself, now that Barry is apparently away ?
> 
> I've just done it.

Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
Barry's administrative overload.

I mean, geez... this is what source control is about. Let a lot of people in
there, but be able to back up in case somebody totally goofs it.

This goes for adding new PEPs, too. I'm not as convinced here, since some
level of "good enough for a PEP" filtering is probably desirable, but then
again, it would seem that the people with commit access probably have that
filter in their head anyways.

Just a thought... how can we grease things up a bit more...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From pf@artcom-gmbh.de  Thu Nov  2 16:59:21 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Thu, 2 Nov 2000 17:59:21 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011021618.IAA15298@slayer.i.sourceforge.net> from Barry Warsaw at "Nov 2, 2000  8:18:27 am"
Message-ID: <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>

Barry Warsaw wrote:
> Update of /cvsroot/python/python/nondist/peps
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv15290
> 
> Added Files:
> 	pep-0227.txt 
> Log Message:
> PEP 227, Statically Nested Scopes, Jeremy Hylton
> 
> 
> ***** Error reading new file: (2, 'No such file or directory')

It was obviously not intended to be mailed out that way again.  
Problem with pathname and/or current directory?  Barry got this 
right once, now it is broken again.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From mal@lemburg.com  Thu Nov  2 17:07:15 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 18:07:15 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
 <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <3A019F43.E13604BA@lemburg.com>

Guido van Rossum wrote:
> 
> > It may not look serious, but changing the Python lookup scheme
> > is, since many inspection tools rely and reimplement exactly
> > that scheme. With nested scopes, there would be next to no
> > way to emulate the lookups using these tools.
> 
> So fix the tools.

Eek. Are you proposing to break all the Python IDE that are
just appearing out there ?

> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?

I think the common practice mainly comes from the fact,
that by making globals locals which can benefit from LOAD_FAST
you get a noticable performance boost.

So the "right" solution to these weird looking hacks would
be to come up with a smart way by which the Python compiler
itself can do the localizing.

Nested scopes won't help eliminating the current keyword 
practice.
 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

Hmm, so far the only argument for changing Python lookups
was to allow writing lambdas without keyword hacks. Does this
really warrant breaking code ?

What other advantages would statically nested scopes have ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Thu Nov  2 05:16:43 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:16:43 -0500
Subject: [Python-Dev] PEP-0217
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:49:37 +0200."
 <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
Message-ID: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>

> 1) BDFL pronouncement

I believe Ping has also proposed such a display hook.  I'm not against
the idea, but I'm also not much in favor -- so I'm kind of +/- 0...

I've always thought that the interpreter mainloop should be rewritten
in Python.  That would be another PEP, and this would be a good place
to add a display hook as a feature.

Note that the example curreltly in the PEP has a fatal flaw: it's got
a recursive reference to print.  This is one of the things to consider
when proposing such a feature.

Sorry I can't be of more help...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Thu Nov  2 17:20:12 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:20:12 -0500 (EST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
 <200011020516.AAA08400@cj20424-a.reston1.va.home.com>
Message-ID: <14849.41548.308591.119778@bitdiddle.concentric.net>

I think the current draft of PEP 217 is far too thin and vague to be
acceptable.  You ought to re-read the PEP guidlines and make sure
you've covered all the points.  You must address at least:

- motivation
- specification of feature independent of implementation

Jeremy


From claird@starbase.neosoft.com  Thu Nov  2 17:18:20 2000
From: claird@starbase.neosoft.com (Cameron Laird)
Date: Thu, 2 Nov 2000 11:18:20 -0600 (CST)
Subject: [Python-Dev] Tk news you'll want to read
Message-ID: <200011021718.LAA70431@starbase.neosoft.com>

<URL:http://deja.com/=dnc/getdoc.xp?AN=688816476>

Note the clear intention to co-operate with Perl and Python,
the ambition to be considerably more portable and superior in
performance to GTK+, and so on.

I encourage you to e-mail this on to others it might interest.
A lot of the Perl people don't receive anything from me because
my ISP is ORBS-blacklisted (don't ask).


From guido@python.org  Thu Nov  2 05:29:27 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:29:27 -0500
Subject: [Python-Dev] CPyAN
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:42:39 +0100."
 <20001102164239.R12812@xs4all.nl>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
 <20001102164239.R12812@xs4all.nl>
Message-ID: <200011020529.AAA08549@cj20424-a.reston1.va.home.com>

[Andrew]
> > * Work on something CPAN-like.  This may or may not have repercussions for
> >   the core; I don't know.

[Thomas]
> Probably not, though perhaps a new module would be nice. As for the
> CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
> Apachecon, and I think it would be suited extremely well as the transmission
> protocol for SPAM (or however you want to call the Python CPAN ;). You can
> do the uploading, downloading and searching for modules using WebDAV without
> too much pain, and there's excellent WebDAV support for Apache ;)
> 
> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

This is a nice thing to have, but I don't see why it should be tied to
the 2.1 effort.  Let's not couple projects that can be carried out
independently!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh21@cam.ac.uk  Thu Nov  2 17:29:52 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:29:52 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Michael Hudson's message of "02 Nov 2000 17:00:37 +0000"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21@cam.ac.uk> writes:

> Guido van Rossum <guido@python.org> writes:
> 
> Then the following could be made essentially equivalent:
> 
> f(a,b,c=d,e=f)
> 
> dict = {c:d,e:f}
> f(a,b,**dict)

... except when you have stuff like

f(a,b=c,**kw)

Gruntle.  Maybe Python function calls are just complicated!  (Has
anyone looked at bytecodehacks.xapply? - and that doesn't even handle
*-ed and **-ed arguments...).

Hmm - the interaction of "a=b" style args & *-ed args is a bit
counter-intuitive, particularly as the "a=b" args syntatically have to
come before the *-ed args:

>>> def f(a,b,c,d):
...  return a,b,c,d
... 
>>> f(1,c=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'c' redefined in call to f()
>>> f(1,b=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'b' redefined in call to f()
>>> f(1,d=4,*(2,3))
(1, 2, 3, 4)

I humbly submit that This Is Wrong.  I haven't seen anybody complain
about it, which suggests to me that noone is using this combination,
and I propose either:

1) banning it
2) demanding that the *-ed arg precede the "a=b" args

Of course, if noone is using this "feature", then maybe this dusty
little corner of the language should be left undisturbed.

And I haven't even thought about default arguments yet...

about-to-make-an-opcode-called-BODGE_KEYWORD_ARGUMENTS-ly y'rs
M.

-- 
  nonono,  while we're making wild  conjectures about the behavior 
  of completely  irrelevant tasks,  we must not  also make serious 
  mistakes, or the data might suddenly become statistically valid.
                                        -- Erik Naggum, comp.lang.lisp



From barry@wooz.org  Thu Nov  2 17:32:13 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:32:13 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
 <20001101234134.O12812@xs4all.nl>
 <14849.39382.960359.909365@anthem.concentric.net>
 <20001102090729.A1874@lyra.org>
Message-ID: <14849.42269.297360.615404@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

    GS> Shouldn't we allow other people to tweak PEP 0? It would
    GS> certainly lighten Barry's administrative overload.

I certainly don't mind at the very least, people modifying PEP 0 when
the status of their own peps change.

    GS> I mean, geez... this is what source control is about. Let a
    GS> lot of people in there, but be able to back up in case
    GS> somebody totally goofs it.

    GS> This goes for adding new PEPs, too. I'm not as convinced here,
    GS> since some level of "good enough for a PEP" filtering is
    GS> probably desirable, but then again, it would seem that the
    GS> people with commit access probably have that filter in their
    GS> head anyways.

    GS> Just a thought... how can we grease things up a bit more...

I do like to make a sanity pass through the text before approving it,
just to make sure we've got consistent format throughout the peps.
Also, I know we're all on "internet time" here <wink>, but one day
isn't too much time to let pass before taking action on things. :)
I'd also prefer it if there's /some/ limited editorial review before
these things get added.

That having been said, I'm very happy if someone wants to co-edit the
peps.  The pep 0 re-organization and slacker scolding would definitely
benefit from more than one watchdog.

Volunteers? :)

-Barry


From barry@wooz.org  Thu Nov  2 17:33:11 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:33:11 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
References: <200011021618.IAA15298@slayer.i.sourceforge.net>
 <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>
Message-ID: <14849.42327.23020.553510@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf@artcom-gmbh.de> writes:

    PF> It was obviously not intended to be mailed out that way again.
    PF> Problem with pathname and/or current directory?  Barry got
    PF> this right once, now it is broken again.

Except that AFAIK, I didn't do anything to fix it, or to break it
again.

-Barry


From guido@python.org  Thu Nov  2 05:45:09 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:45:09 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:05:14 +0100."
 <3A0190BA.940FFADA@lemburg.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
 <3A0190BA.940FFADA@lemburg.com>
Message-ID: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>

> > > [MAL]
> > > > Dynamic nested scopes is another topic... those are *very*
> > > > useful; especially when it comes to avoiding global variables
> > > > and implementing programs which work using control objects
> > > > instead of global function calls.
> > >
> > > Marc-Andre, what are Dynamic nested scopes?

[Moshe]
> > If MAL means dynamic scoping (which I understood he does), then this
> > simply means:
> > 
> > when looking for a variable "foo", you first search for it in the local
> > namespace. If not there, the *caller's* namespace, and so on. In the
> > end, the caller is the __main__ module, and if not found there, it is
> > a NameError.

Ah, yuck.  For variable namespace, this is a really bad idea.  For
certain other things (e.g. try/except blocks, Jeremy's example) it is
of course OK.

[MAL]
> That would be one application, yes.
> 
> With dynamic scoping I meant that the context of a lookup is
> defined at run-time and by explicitely or implicitely
> hooking together objects which then define the nesting.

But certainly you're not thinking of doing this to something as basic
to Python's semantics as local/global variable lookup!

> Environment acquisition is an example of such dynamic scoping:
> attribute lookups are passed on to the outer scope in case they
> don't resolve on the inner scope, e.g. say you have
> object a with a.x = 1; all other objects don't define .x.
> Then a.b.c.d.x will result in lookups
> 1. a.b.c.d.x
> 2. a.b.c.x
> 3. a.b.x
> 4. a.x -> 1

This seems only remotely related to dynamic scopes.  There are
namespaces here, but not "scopes" as I think of them: a scope defines
the validity for an unadorned variable.  Static scopes mean that this
is determined by the program source structure.  Dynamic scopes means
that that this is determined by run-time mechanisms.  Python uses
static scoping for local variables, but a dynamic scoping mechanism
for non-local variable references: first it does a module-global
lookup, then it looks for a built-in name.

When we're not talking about simple name lookup, we should speak of
namespaces (which can also have lifetimes).

> This example uses attribute lookup -- the same can be done for
> other nested objects by explicitely specifying the nesting
> relationship.
> 
> Jim's ExtensionClasses allow the above by using a lot of
> wrappers around objects -- would be nice if we could come
> up with a more general scheme which then also works for
> explicit nesting relationships (e.g. dictionaries which
> get hooked together -- Jim's MultiMapping does this).

I think we're getting way off track here.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Nov  2 17:43:03 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 12:43:03 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDIHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.

I believe all early Lisps were dynamically scoped.  Scheme changed that, and
Common Lisp followed.  Many interpreted languages *start* life with dynamic
scoping because it's easy to hack together, but that never lasts.  REBOL
went thru this a couple years ago, switching entirely from dynamic to
lexical before its first public release, and breaking most existing programs
in the process.  Perl also started with dynamic scoping, but, in Perl-like
fashion, Perl5 *added* lexical scoping on top of dynamic ("local" vars use
dynamic scoping; "my" vars lexical; and all Perl5 gotcha guides stridently
recommend never using "local" anymore).  Here's some Perl5:

$i = 1;

sub f {
    local($i) = 2;
    &g();
}

sub g {
    return $i;
}

print "i at start is $i\n";
print "i in g called directly is ", &g(), "\n";
print "i in g called indirectly via f is ", &f(), "\n";
print "i at end is $i\n";

Here's what it prints:

i at start is 1
i in g called directly is 1
i in g called indirectly via f is 2
i at end is 1

> It is hard to write modular code using dynamic scope, because the
> behavior of a function with free variables can not be determined by
> the module that defines it.

As shown above; dynamic scoping is a nightmare even in the absence of nested
functions.

> Not saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

People who spend too much time writing meta-systems <wink> overestimate its
usefulness.  Most programmers who need this kind of effect would be much
better off explicitly passing a dict of name->value mappings explicitly
manipulated, or simply passing the values of interest (if I want a function
that sucks the value of "i" out of its caller, what clearer way than for the
caller to pass i in the arglist?!  dynamic scoping is much subtler, of
course -- it sucks the value of "i" out of *whatever* function up the call
chain happened to define an i "most recently").

Lexical scoping is much tamer, and, indeed, Python is one of the few modern
languages that doesn't support it.  Last time Guido and I hand-wrung over
this, that was the chief reason to implement it:  newcomers are endlessly
surprised that when they write functions that visually nest, their scopes
nevertheless don't nest.  There's certainly nothing novel or unexpected
about lexical scoping anymore.

The problem unique to Python is its rule for determining which vrbls are
local; in Lisps, REBOL and Perl, you have to explicitly name all vrbls local
to a given scope, which renders "how do you rebind a non-local name?" a
non-issue.

has-the-feel-of-inevitability-ly y'rs  - tim




From guido@python.org  Thu Nov  2 05:56:03 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:56:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:11:30 EST."
 <14849.37426.860007.989619@bitdiddle.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
 <14849.37426.860007.989619@bitdiddle.concentric.net>
Message-ID: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>

[Jeremy and Tim argue about what to do about write access for
variables at intermediate levels of nesting, neither local nor
module-global.]

I'll risk doing a pronouncement, even though I know that Jeremy (and
maybe also Tim?) disagree.

You don't need "write access" (in the sense of being able to assign)
for variables at the intermediate scopes, so there is no need for a
syntax to express this.  Assignments are to local variables (normally)
or to module-globals (when 'global' is used).  Use references search
for a local, then for a local of the containing function definition,
then for a local in its container, and so forth, until it hits the
module globals, and then finally it looks for a builtin.

We can argue over which part of this is done statically and which part
is done dynamically: currently, locals are done dynamically and
everything else is done statically.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Thu Nov  2 17:57:09 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:57:09 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
 <3A0190BA.940FFADA@lemburg.com>
 <200011020545.AAA08743@cj20424-a.reston1.va.home.com>
Message-ID: <14849.43765.419161.105395@bitdiddle.concentric.net>

I don't think I buy your explanation that Python uses dynamic scope
for resolving globals.  As I understand the mechanism, the module
namespace and builtins namespace are those for the module in which the
function was defined.  If so, this is still static scope.

Here's a quick example that illustrates the difference:

module foo:
-----------------------
a = 12

def add(b):
    return a + b
-----------------------

module bar:
-----------------------
from foo import add

a = -1
print add(1)
-----------------------

If Python used static scope, "python bar.py" should print 13 (it
does).  If it used dynamic scope, I would expect the answer to be 0.

Jeremy


From fdrake@acm.org  Thu Nov  2 17:55:11 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 12:55:11 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14848.39586.832800.139182@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
Message-ID: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I just did a clean configure and make from the latest CVS tree.  It
 > seems to get stuck in a loop calling makesetup over and over again.

  Please do an update and run autoconf, then try again.
  I didn't check in the autoconf output; Guido thinks that we want to
stick with autoconf 2.13, and I've got 2.14.1.  The reason is that
there are enough differences in the m4 expansions that seeing the real
effects of the configure.in changes is hard, just because there are so
many changed lines which are not related to the actual change.
  (If you do have autoconf 2.13, please check in the new configure.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From guido@python.org  Thu Nov  2 06:02:10 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:02:10 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:22:32 EST."
 <14849.38088.903104.936944@anthem.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <14849.38088.903104.936944@anthem.concentric.net>
Message-ID: <200011020602.BAA08974@cj20424-a.reston1.va.home.com>

> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
> 
>     d = globals().copy()
>     d.update(locals())
> 
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.  (PEP 42 now includes a feature request
> to make vars() do this by default.)

Note that I just deleted that feature request from PEP 42 -- vars() or
locals() returns the dictionary containing the variables, and you
can't just change the semantics to return a newly copied dictionary
(which could be quite expensive too!).

I don't think you need to have a mechanism to find all accessible
names; I don't see a common use for that.  It's sufficient to have a
mechanism to look up any specific name according to whatever mechanism
we decide upon.  This is needed for internal use of course; it can
also be useful for e.g. variable substitution mechanisms like the one
you recently proposed or Ping's Itmpl.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy@alum.mit.edu  Thu Nov  2 18:04:10 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:04:10 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
 <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44186.494602.297323@bitdiddle.concentric.net>

I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
placed three files in Modules: Makefile.pre, Setup, and Setup.config.

I ran "make" and it immediately reported an error:
"../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
Setup.local: No such file or directory"

Jeremy


From guido@python.org  Thu Nov  2 06:07:04 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:07:04 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:31:35 EST."
 <14849.38631.997377.600214@anthem.concentric.net>
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
 <14849.38631.997377.600214@anthem.concentric.net>
Message-ID: <200011020607.BAA09013@cj20424-a.reston1.va.home.com>

[Barry, on dynamic scopes]
> This is how Emacs Lisp behaves, and it's used all the time in ELisp
> programs.  On the one hand it's quite convenient for customizing the
> behavior of functions.  On the other hand, it can make documenting the
> interface of functions quite difficult because all those dynamically
> scoped variables are now part of the function's API.
> 
> It's interesting to note that many ELispers really hate dynamic
> scoping and pine for a move toward lexical scoping.  I'm not one of
> them.

I don't care what you pine for in ELisp, but for Python this would be
a bad idea.

> I'm not as concerned about "fixing" nested functions because I hardly
> ever use them, and rarely see them much in Python code.  Fixing
> lambdas would be nice, but since Guido considers lambdas themselves a
> mistake, and given that lamda use /can/ be a performance hit in some
> situations, does it make sense to change something as fundamental as
> Python's scoping rules to fix this eddy of the language?

I referred to this in our group meeting as "fixing lambda" because
that's where others seem to need it most often.  But it is a real
problem that exists for all nested functions.  So let me rephrase
that: "fixing nested function definitions" is useful, if it can be
done without leaking memory, without breaking too much existing code,
and without slowing down code that doesn't use the feature.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Thu Nov  2 18:06:12 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:06:12 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
 <14849.37426.860007.989619@bitdiddle.concentric.net>
 <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <14849.44308.333212.37111@bitdiddle.concentric.net>

Don't know if you saw the discussion in the PEP or not.  I made two
arguments for being able to assign to variables bound in enclosing
scopes.

1. Every other language that supports nested lexical scoping allows
   this.  To the extent that programmers have seen these other
   languages, they will expect it to work.

2. It is possible to work around this limitation by using containers.
   If you want to have an integer that can be updated by nested
   functions, you wrap the interger in a list and make all assignments
   and references refer to list[0].  It would be unfortunate if
   programmers used this style, because it is obscure.  I'd rather see
   the language provide a way to support this style of programming
   directly. 

Jeremy


From esr@thyrsus.com  Thu Nov  2 22:10:16 2000
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:10:16 -0800
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>; from guido@python.org on Wed, Nov 01, 2000 at 10:29:03PM -0500
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <20001102141016.B1838@thyrsus.com>

Guido van Rossum <guido@python.org>:
> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?
> 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

For whatever it's worth, I agree with both these arguments.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"The power to tax involves the power to destroy;...the power to
destroy may defeat and render useless the power to create...."
	-- Chief Justice John Marshall, 1819.


From fdrake@acm.org  Thu Nov  2 18:04:34 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:04:34 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44186.494602.297323@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
 <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
 <14849.44186.494602.297323@bitdiddle.concentric.net>
Message-ID: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
 > placed three files in Modules: Makefile.pre, Setup, and Setup.config.
 > 
 > I ran "make" and it immediately reported an error:
 > "../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
 > Setup.local: No such file or directory"

  Very interesting!  The first thing I get is this:

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/fdrake/projects/python/temp/Modules'
echo "# Edit this file for local setup changes" >Setup.local
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
make[1]: Leaving directory `/home/fdrake/projects/python/temp/Modules'

  Can you capture stout & stderr from a clean configure & make and
mail it to me?
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From guido@python.org  Thu Nov  2 06:10:57 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:10:57 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:36:45 EST."
 <14849.38941.59576.682495@bitdiddle.concentric.net>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com>
 <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>

> Moshe's explanation of "dynamic scope" is the definition I've seen in
> every programming language text I've ever read.  The essence of the
> defintion, I believe, is that a free variable is resolved in the
> environment created by the current procedure call stack.

Ah.  The term "free variable" makes sense here.

> I think it muddles the discussion to use "dynamic scope" to describe
> acquistion, though it is a dynamic feature.
> 
> Python using dynamic scope for exceptions.  If any exception is
> raised, the exception handler that is triggered is determined by the
> environment in which the procedure was called.

Then I think this is also muddles the discussion, since the look for
exception handlers has nothing to do with free variable lookup.

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

I think Python 3000 ought to use totally static scoping.  That will
make it possible to do optimize code using built-in names!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From esr@thyrsus.com  Thu Nov  2 22:12:57 2000
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:12:57 -0800
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <3A01910E.A19FFBB2@lemburg.com>; from mal@lemburg.com on Thu, Nov 02, 2000 at 05:06:38PM +0100
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <3A01910E.A19FFBB2@lemburg.com>
Message-ID: <20001102141257.C1838@thyrsus.com>

M.-A. Lemburg <mal@lemburg.com>:
> Most important for 2.1 are probably:
> 
> 1. new C level coercion scheme
> 2. rich comparisons
> 3. making the std lib Unicode compatible

I'd certainly like to see rich comparisons go in.  I have a "Set" class 
all ready for addition to the standard library except that it's waiting
on this feature in order to do partial ordering properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Every election is a sort of advance auction sale of stolen goods. 
	-- H.L. Mencken 


From esr@thyrsus.com  Thu Nov  2 22:14:24 2000
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:14:24 -0800
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 11:36:45AM -0500
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <20001102141424.D1838@thyrsus.com>

Jeremy Hylton <jeremy@alum.mit.edu>:
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.

Correct.  Based on my LISP experience, I would be strongly opposed to
dymamic scoping.  That path has been tried and found wanting.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"Guard with jealous attention the public liberty.  Suspect every one
who approaches that jewel.  Unfortunately, nothing will preserve it
but downright force.  Whenever you give up that force, you are
inevitably ruined."
	-- Patrick Henry, speech of June 5 1788


From jeremy@alum.mit.edu  Thu Nov  2 18:14:08 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:14:08 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
 <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
 <14849.44186.494602.297323@bitdiddle.concentric.net>
 <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44784.479281.162333@bitdiddle.concentric.net>

Despite the error, I did get a successful build this time.  I guess
your configure change worked.

Jeremy


From jeremy@alum.mit.edu  Thu Nov  2 18:16:25 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:16:25 -0500 (EST)
Subject: [Python-Dev] Larry Wall talk on Perl 6
Message-ID: <14849.44921.765397.205628@bitdiddle.concentric.net>

http://dev.perl.org/~ask/als/

Interesting reading.

Jeremy


From fdrake@acm.org  Thu Nov  2 18:12:38 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:12:38 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44784.479281.162333@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
 <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
 <14849.44186.494602.297323@bitdiddle.concentric.net>
 <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
 <14849.44784.479281.162333@bitdiddle.concentric.net>
Message-ID: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Despite the error, I did get a successful build this time.  I guess
 > your configure change worked.

  I don't think the bug I fixed and the bug you reported were tightly
related.  I don't understand why you got the error after before or
after my change.
  I'f you're running autoconf 2.13, though, please do check in the new
configure script!
  (If anyone else is, you can check it in too!  The winner gets one
free documentation download!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From barry@wooz.org  Thu Nov  2 18:20:06 2000
From: barry@wooz.org (barry@wooz.org)
Date: Thu, 2 Nov 2000 13:20:06 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
 <14849.38088.903104.936944@anthem.concentric.net>
 <200011020602.BAA08974@cj20424-a.reston1.va.home.com>
Message-ID: <14849.45142.860806.358489@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    >> If we get lexical scoping, there should be a fast (built-in)
    >> way to get at all the accessible names from Python.
    >> I.e. currently I can do d = globals().copy() d.update(locals())
    >> and know that `d' contains a dictionary of available names,
    >> with the right overloading semantics.  (PEP 42 now includes a
    >> feature request to make vars() do this by default.)

    GvR> Note that I just deleted that feature request from PEP 42 --
    GvR> vars() or locals() returns the dictionary containing the
    GvR> variables, and you can't just change the semantics to return
    GvR> a newly copied dictionary (which could be quite expensive
    GvR> too!).

Saw that.  I was just thinking that locals() already does what
vars()-no-args does, so why have two ways to do the same thing?

    GvR> I don't think you need to have a mechanism to find all
    GvR> accessible names; I don't see a common use for that.  It's
    GvR> sufficient to have a mechanism to look up any specific name
    GvR> according to whatever mechanism we decide upon.  This is
    GvR> needed for internal use of course; it can also be useful for
    GvR> e.g. variable substitution mechanisms like the one you
    GvR> recently proposed or Ping's Itmpl.

Ah, something like this then:

-------------------- snip snip --------------------
import sys
from UserDict import UserDict

class NamesDict(UserDict):
    def __init__(self, frame):
        self.__frame = frame
        UserDict.__init__(self)

    def __getitem__(self, key):
        if self.data.has_key(key):
            return self.data[key]
        locals = self.__frame.f_locals
        if locals.has_key(key):
            return locals[key]
        globals = self.__frame.f_globals
        if globals.has_key(key):
            return globals[key]
        raise KeyError, key

def _(s):
    try: raise 'oops'
    except: frame = sys.exc_info()[2].tb_frame.f_back
    return s % NamesDict(frame)

theirs = 'theirs'
def give(mine, yours):
    print _('mine=%(mine)s, yours=%(yours)s, theirs=%(theirs)s')
-------------------- snip snip --------------------
Python 2.0 (#128, Oct 18 2000, 04:48:44) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import dict
>>> dict.give('mine', 'yours')
mine=mine, yours=yours, theirs=theirs
>>> 

-Barry


From guido@python.org  Thu Nov  2 06:28:26 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:28:26 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: Your message of "Thu, 02 Nov 2000 09:07:29 PST."
 <20001102090729.A1874@lyra.org>
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>
 <20001102090729.A1874@lyra.org>
Message-ID: <200011020628.BAA09282@cj20424-a.reston1.va.home.com>

> Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
> Barry's administrative overload.
> 
> I mean, geez... this is what source control is about. Let a lot of people in
> there, but be able to back up in case somebody totally goofs it.

Agreed.

> This goes for adding new PEPs, too. I'm not as convinced here, since some
> level of "good enough for a PEP" filtering is probably desirable, but then
> again, it would seem that the people with commit access probably have that
> filter in their head anyways.

Here, common sense and good judgement should be applied.  If there
seems to be consensus that a PEP is needed, there's no need to wait
for Barry.  The update to PEP-0000 commits the assignment of the new
PEP number.  But the new PEP should follow all the rules for a new
PEP!

Having Barry in the loop makes sense for those who aren't sure they
can comply with all the rules, and for those outside the python-dev
community.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Nov  2 18:29:41 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 13:29:41 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com>

[Guido]
> [Jeremy and Tim argue about what to do about write access for
> variables at intermediate levels of nesting, neither local nor
> module-global.]
>
> I'll risk doing a pronouncement, even though I know that Jeremy (and
> maybe also Tim?) disagree.
>
> You don't need "write access" (in the sense of being able to assign)
> for variables at the intermediate scopes, so there is no need for a
> syntax to express this.

I can live with that!  Reference-only access to intermediate scopes would
address 99% of current gripes.  Of course, future gripes will shift to that
there's no rebinding access.  If we have to support that someday too, my
preferred way of spelling it in Python requires explicit new syntax, so
adding that later would not break anything.

> Assignments are to local variables (normally) or to module-globals (when
> 'global' is used).  Use references search for a local, then for a local of
> the containing function definition, then for a local in its container,

The Pascal standard coined "closest-containing scope" to describe this
succinctly, and I recommend it for clarity and brevity.

> and so forth, until it hits the module globals, and then finally it looks
> for a builtin.
>
> We can argue over which part of this is done statically and which part
> is done dynamically: currently, locals are done dynamically and
> everything else is done statically.

I'm not sure what you're trying to say there, but to the extent that I think
I grasp it, I believe it's backwards:  locals are static today but
everything else is dynamic (and not in the sense of "dynamic scoping", but
in the operational sense of "requires runtime search to resolve non-local
names", while local names are fully resolved at compile-time today (but in
the absence of "exec" and "import *")).

what's-in-a-name?-a-rose-in-any-other-scope-may-not-smell-as-
     sweet-ly y'rs  - tim




From guido@python.org  Thu Nov  2 06:35:00 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:35:00 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 18:07:15 +0100."
 <3A019F43.E13604BA@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
 <3A019F43.E13604BA@lemburg.com>
Message-ID: <200011020635.BAA09321@cj20424-a.reston1.va.home.com>

[MAL]
> > > It may not look serious, but changing the Python lookup scheme
> > > is, since many inspection tools rely and reimplement exactly
> > > that scheme. With nested scopes, there would be next to no
> > > way to emulate the lookups using these tools.

[GvR]
> > So fix the tools.

[MAL]
> Eek. Are you proposing to break all the Python IDE that are
> just appearing out there ?

Yes.  If a tool does variable scope analysis, it should be prepared
for changes in the rules.  Otherwise we might as well have refused the
syntax changes in 2.0 because they required changes to tools!

> > > To be honest, I don't think static nested scopes buy us all that
> > > much. You can do the same now, by using keyword arguments which
> > > isn't all that nice, but works great and makes the scope clearly
> > > visible.
> > 
> > Yes.  It's a hack that gets employed over and over.  And it has
> > certain problems.  We added 'import as' to get rid of a common
> > practice that was perceived unclean.  Maybe we should support nested
> > scopes to get rid of another unclean common practice?
> 
> I think the common practice mainly comes from the fact,
> that by making globals locals which can benefit from LOAD_FAST
> you get a noticable performance boost.
> 
> So the "right" solution to these weird looking hacks would
> be to come up with a smart way by which the Python compiler
> itself can do the localizing.

Can you elaborate?  I dun't understand what you are proposing here.

> Nested scopes won't help eliminating the current keyword 
> practice.

Why not?  I'd say that

    def create_adder(n):
        def adder(x, n=n): return x+n
	return adder

is a hack and that nested scopes can fix this by allowing you to write


    def create_adder(n):
        def adder(x): return x+n
	return adder
    
like one would expect.  (Don't tell me that it isn't a FAQ why this
doesn't work!)

> > I'm not saying that we definitely should add this to 2.1 (there's
> > enough on our plate already) but we should at least consider it, and
> > now that we have cycle GC, the major argument against it (that it
> > causes cycles) is gone...
> 
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?
> 
> What other advantages would statically nested scopes have ?

Doing what's proper.  Nested scopes are not a bad idea.  They weren't
implemented because they were hard to get right (perhaps impossible
without creating cycles), and I was okay with that because I didn't
like them; but I've been convinced that examples like the second
create_adder() above should reall work.  Just like float+int works
(in Python 0.1, this was a type-error -- you had to case the int arg
to a float to get a float result).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov  2 06:40:19 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:40:19 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:45:12 +0100."
 <3A019A18.20D12FE1@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>
 <3A019A18.20D12FE1@lemburg.com>
Message-ID: <200011020640.BAA09370@cj20424-a.reston1.va.home.com>

> Well first you'd have to change all tools to use the new
> scheme (this includes debuggers, inspection tools, reflection
> kits, etc.). This certainly is not a smart thing to do since
> Python IDEs are just starting to appear -- you wouldn't want
> to break all those.

I've seen a Komodo demo.  Yes, it does this.  But it's soooooooo far
from being done that adding this wouldn't really slow them down much,
I think.  More likely, the toolmakers will have fun competing with
each other to be the first to support this! :-)

> What get's harder with the nested scheme is that
> you can no longer be certain that globals() reaches out to
> the module namespace. But this is needed by some lazy evaluation
> tools. Writing to globals() would not be defined anymore --
> where should you bind the new variable ?

I think globals() should return the module's __dict__, and the global
statement should cause the variable to reach directly into there.

We'll need to introduce a new builtin to do a name *lookup* in the
nested scopes.

> Another problem is that there probably won't be a way to access
> all the different nesting levels on a per-level basis (could be
> that I'm missing something here, but debugging tools would need
> some sort of scope() builtin to access the different scopes).
> I'm not sure whether this is possible to do without some sort
> of link between the scopes. We currently don't need such links.

Correct.  That link may have to be added to the frame object.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik@effbot.org  Thu Nov  2 18:56:40 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Thu, 2 Nov 2000 19:56:40 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>              <3A019F43.E13604BA@lemburg.com>  <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <000f01c044ff$47b72cb0$3c6340d5@hagrid>

Guido van Rossum wrote:> [MAL]
> > Eek. Are you proposing to break all the Python IDE that are
> > just appearing out there ?
> 
> Yes.  If a tool does variable scope analysis, it should be prepared
> for changes in the rules.

we can live with that...

</F>



From guido@python.org  Thu Nov  2 06:50:45 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:50:45 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "02 Nov 2000 17:29:52 GMT."
 <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk>
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>
 <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011020650.BAA09505@cj20424-a.reston1.va.home.com>

> Hmm - the interaction of "a=b" style args & *-ed args is a bit
> counter-intuitive, particularly as the "a=b" args syntatically have to
> come before the *-ed args:
> 
> >>> def f(a,b,c,d):
> ...  return a,b,c,d
> ... 
> >>> f(1,c=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'c' redefined in call to f()
> >>> f(1,b=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'b' redefined in call to f()
> >>> f(1,d=4,*(2,3))
> (1, 2, 3, 4)
> 
> I humbly submit that This Is Wrong.  I haven't seen anybody complain
> about it, which suggests to me that noone is using this combination,
> and I propose either:
> 
> 1) banning it
> 2) demanding that the *-ed arg precede the "a=b" args

Interesting.  The *-ed arg cannot precede the a=b args currently, but
I agree that it would have made more sense to do it that way.

I'm not sure that it's worth allowing the combination of kw args and
*tuple, but I'l also not sure that it *isn't* worth it.

> Of course, if noone is using this "feature", then maybe this dusty
> little corner of the language should be left undisturbed.

I think it may be best to leave it alone.

> And I haven't even thought about default arguments yet...

That's unrelated.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From cgw@fnal.gov  Thu Nov  2 18:56:55 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Thu, 2 Nov 2000 12:56:55 -0600 (CST)
Subject: [Python-Dev] PythonLabs -> Digital Creations
Message-ID: <14849.47351.835291.365708@buffalo.fnal.gov>

So, when are the relevant websites going to be updated to reflect the
new reality?  Pythonlabs.com and BeOpen.com still reflect the old
regime...  I'd think the Digital Creations folks would be eager to
announce the changing of the guard...


From gstein@lyra.org  Thu Nov  2 18:57:07 2000
From: gstein@lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 10:57:07 -0800
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Thu, Nov 02, 2000 at 01:12:38PM -0500
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>
Message-ID: <20001102105707.H2037@lyra.org>

On Thu, Nov 02, 2000 at 01:12:38PM -0500, Fred L. Drake, Jr. wrote:
> 
> Jeremy Hylton writes:
>  > Despite the error, I did get a successful build this time.  I guess
>  > your configure change worked.
> 
>   I don't think the bug I fixed and the bug you reported were tightly
> related.  I don't understand why you got the error after before or
> after my change.
>   I'f you're running autoconf 2.13, though, please do check in the new
> configure script!

Wouldn't it make sense to do the upgrade to take advantages of the autoconf
fixes? If there are any problems with the change, then now is the time to do
it and to sort them out!

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From tim_one@email.msn.com  Thu Nov  2 19:07:18 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:07:18 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.37426.860007.989619@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>

[Jeremy]
> I agree that visual inspection is a tad harder, but I contend that
> existing programs that use the same name for a global variable and a
> local variable -- and intend for the global to be visible within a
> function nested in the local variable's region -- are confusing.
> It's too hard for a first-time reader of the code to figure out what
> is going on.
>
> Incidentally, I have yet to see an example of this problem occurring
> in anyone's code.  All the examples seem a bit contrived.  I wonder if
> anyone has an example in existing code.

I wasn't the one making the "will break code" argument (I'm sure it will,
but very little, and not at all for most people since most people never nest
functions in Python today apart from default-abusing lambdas).  Visual
inspection stands on its own as a potential problem.

>   [My SICP example omitted]
>
>   TP> Unfortunately for proponents, this is exactly the kind of SICP
>   TP> example that is much better done via a class.
>
> Indeed, the PEP says exactly that: This kind of program is better
> done via a class.  My intent was not to show a compelling use of
> mutable state.  Instead it was to show that with read-only access,
> people could still modify values on enclosing scopes.  The issue is
> whether the language allows the programmer to express this intent
> clearly or if she has to jump through some hoops to accomplish it.

Guido said "jump through some hoops", so I'm dropping it, but first noting
that "the container" in *idiomatic* Python will most often be "self":

    def yadda(self, ...):
        def whatever(amount):
            self.balance += amount
        return whatever

will work to rebind self.balance.  I don't think Guido will ever be
interested in supporting idiomatic Scheme.

>   TP>     def deposit(amount):
>   TP>         global bank_account.balance
>   TP>         balance += amount

> I'm still not sure I like it, because it mixes local variables of a
> function with attribute access on objects.  I'll add it to the
> discussion in the PEP (if Barry approves the PEP <wink>), though.

Actually, no connection to attribute access was intended there:  it was just
a backward-compatible way to spell the pair (name of containing scope, name
of vrbl in that scope).

    global back_account:balance
or
    global balance from bank_account

would do as well (and maybe better as they don't imply attribute access; but
maybe worse as they don't bring to mind attribute access <wink>).

> Do you have any opinion on the subtleties?  The two that immediately
> come to mind are: 1) whether the function's local are available as
> attributes anywhere or only in nested scopes

I didn't intend attribute-like access at all (although we had earlier talked
about that wrt JavaScript, I didn't have that in mind here).

> and 2) whether you can create new local variable using this notation.

Yes, that's the primary one I was thinking about.




From tim_one@email.msn.com  Thu Nov  2 19:10:52 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:10:52 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.38088.903104.936944@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>

[Barry A. Warsaw]
> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
>
>     d = globals().copy()
>     d.update(locals())
>
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.

It was long ago agreed (don't you love how I pull this stuff out of thin
historical air <wink>?) that if nested lexical scoping was added, we would
need also to supply a new mapping object that mimicked the full Python
lookup rules (including builtins).  Not necessarily a dictionary, though.




From guido@python.org  Thu Nov  2 07:15:50 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:15:50 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:33:11 EST."
 <14849.42327.23020.553510@anthem.concentric.net>
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>
 <14849.42327.23020.553510@anthem.concentric.net>
Message-ID: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>

>     PF> It was obviously not intended to be mailed out that way again.
>     PF> Problem with pathname and/or current directory?  Barry got
>     PF> this right once, now it is broken again.
> 
> Except that AFAIK, I didn't do anything to fix it, or to break it
> again.

We have too much to do to try and fix this now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Nov  2 19:14:50 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:14:50 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A019F43.E13604BA@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEPHOAA.tim_one@email.msn.com>

[MAL]
> ...
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?

*Some* amount of code, sure.  Hard to quantify, but hard to believe there's
much code at risk.

> What other advantages would statically nested scopes have ?

Pythonic obviousness.  Virtually everyone coming to Python from other
languages *expects* visually nested functions to work this way.  That they
don't today is a very frequent source of surprises and complaints.




From barry@wooz.org  Thu Nov  2 19:31:37 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 14:31:37 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.38088.903104.936944@anthem.concentric.net>
 <LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>
Message-ID: <14849.49433.197319.973848@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim_one@email.msn.com> writes:

    TP> It was long ago agreed (don't you love how I pull this stuff
    TP> out of thin historical air <wink>?) that if nested lexical
    TP> scoping was added, we would need also to supply a new mapping
    TP> object that mimicked the full Python lookup rules (including
    TP> builtins).  Not necessarily a dictionary, though.

Oh yeah, I vaguely remember now <0.5 scratch-head>.

Works for me, although I'll point out that we needn't wait for lexical
scoping to provide such an object!

-Barry


From guido@python.org  Thu Nov  2 07:34:53 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:34:53 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:57:09 EST."
 <14849.43765.419161.105395@bitdiddle.concentric.net>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <200011020545.AAA08743@cj20424-a.reston1.va.home.com>
 <14849.43765.419161.105395@bitdiddle.concentric.net>
Message-ID: <200011020734.CAA09751@cj20424-a.reston1.va.home.com>

> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

That's not what I meant, but I expressed it clumsily.  Perhaps the
terminology is just inadequate.  I simply meant that builtins can be
overridden by module-globals.  But only in the module whose globals
are searched for globals -- that part is still static.

Let's drop this thread...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Nov  2 19:31:59 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 14:31:59 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.49013.363302.823824@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
 <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
 <14849.44186.494602.297323@bitdiddle.concentric.net>
 <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
 <14849.45483.304866.556547@bitdiddle.concentric.net>
 <14849.45714.207143.196945@cj42289-a.reston1.va.home.com>
 <14849.47007.862290.364694@bitdiddle.concentric.net>
 <14849.48464.183713.819894@cj42289-a.reston1.va.home.com>
 <14849.49013.363302.823824@bitdiddle.concentric.net>
Message-ID: <14849.49455.97877.379601@cj42289-a.reston1.va.home.com>

[We've uncovered the root of Jeremy's problem here -- this is a
 problem/limitation of make with VPATH.]

Jeremy Hylton writes:
 > There is a Setup.local in the source directory.

  That's why it isn't being built.  Since it's on the VPATH, make
thinks it already exists in a usable form.  Since the command that
uses it doesn't look for the "most local" version but assumes it
exists in the right place, the command fails.
  This is a problem that exists with VPATH; the built files cannot
already exist in the source directories (hence, the build directories
can't refer to source dirs which have been used as build dirs; "make
clobber" isn't good enough for the Setup* files).
  Remove (or move) the existing file, and it should work fine.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From tim_one@email.msn.com  Thu Nov  2 19:41:12 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:41:12 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: <14849.47351.835291.365708@buffalo.fnal.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFBHOAA.tim_one@email.msn.com>

[Charles G Waldman]
> So, when are the relevant websites going to be updated to reflect the
> new reality?

Sorry, we have no idea.

> Pythonlabs.com and BeOpen.com still reflect the old regime...

And they're running on BeOpen.com machines, which we can no longer fiddle
with.  The people remaining at BeOpen.com probably don't view updating those
sites as a priority anymore (you all saw how long it took them to get
pythonlabs.com and Starship merely running again, although at the time we
couldn't tell you *why* we weren't able to fix them ourselves).

> I'd think the Digital Creations folks would be eager to announce the
> changing of the guard...

They're getting the word out in their own way.  Hope you saw the cover story
on LWN today!

    http://www.lwn.net/

There's a lot of good info there, including useful interviews with Paul
Everitt and Guido.




From mwh21@cam.ac.uk  Thu Nov  2 19:45:47 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 19:45:47 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Thu, 02 Nov 2000 01:50:45 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk> <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> <200011020650.BAA09505@cj20424-a.reston1.va.home.com>
Message-ID: <m3vgu6duok.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido@python.org> writes:

> > Hmm - the interaction of "a=b" style args & *-ed args is a bit
> > counter-intuitive, particularly as the "a=b" args syntatically have to
> > come before the *-ed args:
> > 
> > >>> def f(a,b,c,d):
> > ...  return a,b,c,d
> > ... 
> > >>> f(1,c=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'c' redefined in call to f()
> > >>> f(1,b=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'b' redefined in call to f()
> > >>> f(1,d=4,*(2,3))
> > (1, 2, 3, 4)
> > 
> > I humbly submit that This Is Wrong.  I haven't seen anybody complain
> > about it, which suggests to me that noone is using this combination,
> > and I propose either:
> > 
> > 1) banning it
> > 2) demanding that the *-ed arg precede the "a=b" args
> 
> Interesting.  The *-ed arg cannot precede the a=b args currently, but
> I agree that it would have made more sense to do it that way.
> 
> I'm not sure that it's worth allowing the combination of kw args and
> *tuple, but I'l also not sure that it *isn't* worth it.
> 
> > Of course, if noone is using this "feature", then maybe this dusty
> > little corner of the language should be left undisturbed.
> 
> I think it may be best to leave it alone.

Oh, I mostly agree and am not going to put any effort into this area -
but if I or someone else manages to make striking simplifications to
the function call code that has the side effect of banning/changing
the syntax of this combination, then this probably wouldn't be a good
enough reason for its rejection.

> > And I haven't even thought about default arguments yet...
> 
> That's unrelated.

I know, it was just something that occurred to me as adding more
complexity.  Hypothetical excercise: Write a description of Python's
current function call behaviour in sufficient detail to allow
reimplementation of it without playing with an existing interpreter in
less than 1000 words.  It's just hairy, and it amazes me that it feels
so natural most of the time...

Cheers,
M.

-- 
  Not only does the English Language borrow words from other
  languages, it sometimes chases them down dark alleys, hits 
  them over the head, and goes through their pockets.   -- Eddy Peters



From guido@python.org  Thu Nov  2 07:51:15 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:51:15 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:20:06 EST."
 <14849.45142.860806.358489@anthem.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <14849.38088.903104.936944@anthem.concentric.net> <200011020602.BAA08974@cj20424-a.reston1.va.home.com>
 <14849.45142.860806.358489@anthem.concentric.net>
Message-ID: <200011020751.CAA12539@cj20424-a.reston1.va.home.com>

[Barry]
> Saw that.  I was just thinking that locals() already does what
> vars()-no-args does, so why have two ways to do the same thing?

Not clear -- maybe one of them needs to be made obsolete.

>     GvR> I don't think you need to have a mechanism to find all
>     GvR> accessible names; I don't see a common use for that.  It's
>     GvR> sufficient to have a mechanism to look up any specific name
>     GvR> according to whatever mechanism we decide upon.  This is
>     GvR> needed for internal use of course; it can also be useful for
>     GvR> e.g. variable substitution mechanisms like the one you
>     GvR> recently proposed or Ping's Itmpl.
> 
> Ah, something like this then:
[Example deleted]

I'm not sure that the mechanism provided should follow the mapping
API.  *All* it needs to do it provide lookup capability.  Having
.keys(), .items(), .has_key() etc. just slows it down.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Nov  2 19:54:05 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:54:05 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.43765.419161.105395@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFCHOAA.tim_one@email.msn.com>

[Jeremy]
> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

It's dynamic in the shallow (but real!) sense that it can't be determined to
which of {module scope, builtin scope} a global name resolves today until
runtime.  Indeed, in

def f():
    print len

the resolving scope for "len" may even change with each invocation of f(),
depending on who's playing games with the containing module's __dict__.
That's not "dynamic scoping" in the proper sense of the term, but it's sure
dynamic!

words-lead-to-more-words-so-become-one-with-the-essence-instead<wink>-ly
    y'rs  - tim




From guido@python.org  Thu Nov  2 07:59:47 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:29:41 EST."
 <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com>
Message-ID: <200011020759.CAA12600@cj20424-a.reston1.va.home.com>

> [Guido]
> > [Jeremy and Tim argue about what to do about write access for
> > variables at intermediate levels of nesting, neither local nor
> > module-global.]
> >
> > I'll risk doing a pronouncement, even though I know that Jeremy (and
> > maybe also Tim?) disagree.
> >
> > You don't need "write access" (in the sense of being able to assign)
> > for variables at the intermediate scopes, so there is no need for a
> > syntax to express this.

[Tim]
> I can live with that!  Reference-only access to intermediate scopes would
> address 99% of current gripes.  Of course, future gripes will shift to that
> there's no rebinding access.  If we have to support that someday too, my
> preferred way of spelling it in Python requires explicit new syntax, so
> adding that later would not break anything.

Exactly my point.

> > Assignments are to local variables (normally) or to module-globals (when
> > 'global' is used).  Use references search for a local, then for a local of
> > the containing function definition, then for a local in its container,
> 
> The Pascal standard coined "closest-containing scope" to describe this
> succinctly, and I recommend it for clarity and brevity.

Thanks, that's a good term.  And the principle is totally
uncontroversial.

> > and so forth, until it hits the module globals, and then finally it looks
> > for a builtin.
> >
> > We can argue over which part of this is done statically and which part
> > is done dynamically: currently, locals are done dynamically and
> > everything else is done statically.
> 
> I'm not sure what you're trying to say there, but to the extent that I think
> I grasp it, I believe it's backwards:  locals are static today but
> everything else is dynamic (and not in the sense of "dynamic scoping", but
> in the operational sense of "requires runtime search to resolve non-local
> names", while local names are fully resolved at compile-time today (but in
> the absence of "exec" and "import *")).

Oops, yes, I had it backwards.

As I said elsewhere, in Python 3000 I'd like to do it all more
statically.  So perhaps we should look up nested locals based on
static information too.  Thus:

  x = "global-x"

  def foo():
      if 0:
          x = "x-in-foo"
      def bar():
	  return x
      return bar

  print foo()()

should raise UnboundLocalError, not print "global-x".

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov  2 07:59:55 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:55 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:06:12 EST."
 <14849.44308.333212.37111@bitdiddle.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com> <14849.37426.860007.989619@bitdiddle.concentric.net> <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
 <14849.44308.333212.37111@bitdiddle.concentric.net>
Message-ID: <200011020759.CAA12609@cj20424-a.reston1.va.home.com>

> Don't know if you saw the discussion in the PEP or not.

Sorry, I had no time.  I have read it now, but it doesn't change my
point of view.

> I made two
> arguments for being able to assign to variables bound in enclosing
> scopes.
> 
> 1. Every other language that supports nested lexical scoping allows
>    this.  To the extent that programmers have seen these other
>    languages, they will expect it to work.

But we have a unique way of declaring variables, which makes the
issues different.

Your PEP wonders why I am against allowing assignment to intermediate
levels.  Here's my answer: all the syntaxes that have been proposed
to spell this have problems.  So let's not provide a way to spell it.
I predict that it won't be a problem.  If it becomes a problem, we can
add a way to spell it later.  I expect that the mechanism that will be
used to find variables at intermediate levels can also be used to set
them, so it won't affect that part of the implementation much.

> 2. It is possible to work around this limitation by using containers.
>    If you want to have an integer that can be updated by nested
>    functions, you wrap the interger in a list and make all assignments
>    and references refer to list[0].  It would be unfortunate if
>    programmers used this style, because it is obscure.  I'd rather see
>    the language provide a way to support this style of programming
>    directly. 

I don't expect that programmers will use this style.  When they have
this need, they will more likely use a class.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov  2 08:08:14 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:08:14 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:56:55 CST."
 <14849.47351.835291.365708@buffalo.fnal.gov>
References: <14849.47351.835291.365708@buffalo.fnal.gov>
Message-ID: <200011020808.DAA16162@cj20424-a.reston1.va.home.com>

> So, when are the relevant websites going to be updated to reflect the
> new reality?  Pythonlabs.com and BeOpen.com still reflect the old
> regime...  I'd think the Digital Creations folks would be eager to
> announce the changing of the guard...

While technically I have the capability to update the pythonlabs.com
website, ethically I feel I cannot do this.  The site belongs to
BeOpen.  I'll ask them to make some kind of announcement.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov  2 08:16:53 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:16:53 -0500
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:57:07 PST."
 <20001102105707.H2037@lyra.org>
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>
 <20001102105707.H2037@lyra.org>
Message-ID: <200011020816.DAA16227@cj20424-a.reston1.va.home.com>

> Wouldn't it make sense to do the upgrade to take advantages of the autoconf
> fixes? If there are any problems with the change, then now is the time to do
> it and to sort them out!

Yeah, but I have too much to do to get to this any time soon -- it
just isn't important enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@mojam.com (Skip Montanaro)  Thu Nov  2 21:19:11 2000
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Nov 2000 15:19:11 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
Message-ID: <14849.55887.283432.505910@beluga.mojam.com>

>>>>> "Tim" == Tim Peters <tim_one@email.msn.com> writes:

    Tim> [Jeremy]
    >> All the examples seem a bit contrived.  I wonder if anyone has an
    >> example in existing code.

    Tim> I wasn't the one making the "will break code" argument ...

Nor was I.  Jeremy (I think) asked MAL how it could break code.  I posted a
(simple, but obviously contrived) example.  I have no particular opinion on
this subject.  I was just trying to answer Jeremy's question.

Skip


From jeremy@alum.mit.edu  Thu Nov  2 20:25:36 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 15:25:36 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.55887.283432.505910@beluga.mojam.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
 <14849.55887.283432.505910@beluga.mojam.com>
Message-ID: <14849.52672.943439.978785@bitdiddle.concentric.net>

Looks like we need to rehash this thread at least enough to determine
who is responsible for causing us to rehash it.

MAL said it would break code.  I asked how.  Skip and Tim obliged with
examples.  I said their examples exhibited bad style; neither of them
claimed they were good style.

In the end, I observed that while it could break code in theory, I
doubted it really would break much code.  Furthermore, I believe that
the code it will break is already obscure so we needn't worry about
it.

Jeremy


From tim_one@email.msn.com  Thu Nov  2 21:13:47 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 16:13:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFKHOAA.tim_one@email.msn.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
>
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.

Ah!  I wasn't trying to give an example of code that would break, but, ya,
now that you mention it, it would.  I was just giving an example of why
visual inspection will be harder in Python than in Pascal.

I expect that the kind of code I showed *will* be common: putting all the
nested "helper functions" at the top of a function, just as was also done in
Pascal.  The killer difference is that in Pascal, the containing function's
locals referenced by the nested helpers can be found declared at the top of
the containing function; but in Python you'll have to search all over the
place, and if you don't find a non-local var at once, you'll be left uneasy,
wondering whether you missed a binding target, or whether the var is truly
global, or what.




From guido@python.org  Thu Nov  2 09:41:01 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 04:41:01 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 15:25:36 EST."
 <14849.52672.943439.978785@bitdiddle.concentric.net>
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com>
 <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <200011020941.EAA18905@cj20424-a.reston1.va.home.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's quite enough rehashing.  I don't think we'll have to worry
about breakiung much code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Thu Nov  2 21:47:35 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 22:47:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 02:15:50AM -0500
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de> <14849.42327.23020.553510@anthem.concentric.net> <200011020715.CAA09616@cj20424-a.reston1.va.home.com>
Message-ID: <20001102224734.U12812@xs4all.nl>

On Thu, Nov 02, 2000 at 02:15:50AM -0500, Guido van Rossum wrote:

> >     PF> It was obviously not intended to be mailed out that way again.
> >     PF> Problem with pathname and/or current directory?  Barry got
> >     PF> this right once, now it is broken again.

> We have too much to do to try and fix this now.

But I haven't ;) I've made syncmail give some more information if it can't
find the file, to compensate for the fact that cvs.python.sourceforge.net
runs a Python before 1.5.2, and thus doesn't show the file it couldn't find.
I won't bother with creating files just for the hell of it, so we'll just
wait until Barry triggers it again.

(Not that I'm bored (not much, anyway), but this was a no-brainer.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From MarkH@ActiveState.com  Thu Nov  2 21:55:07 2000
From: MarkH@ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:55:07 +1100
Subject: [Python-Dev] please subscribe me ;-)
Message-ID: <LCEPIIGDJPKCOIHOBJEPAENNCFAA.MarkH@ActiveState.com>

Hi all,
	My skippinet.com.au address is temporarily bouncing, and thus I am no
longer getting python-dev mail.  I re-subscribed on Monday, but am still
waiting moderator approval.

Can someone approve me please?

Thanks,

Mark.



From MarkH@ActiveState.com  Thu Nov  2 21:57:58 2000
From: MarkH@ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:57:58 +1100
Subject: [Python-Dev] Dont bother subscribing me!
Message-ID: <LCEPIIGDJPKCOIHOBJEPEENNCFAA.MarkH@ActiveState.com>

All done already - sorry about the noise.

Thanks,

Mark.


From guido@python.org  Thu Nov  2 10:06:16 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 05:06:16 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:34:42 EST."
 <20001102103442.B5027@kronos.cnri.reston.va.us>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>
 <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>

[Andrew]
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.

I just skimmed PEP 222.  I agree that the classes defined by cgi.py
are unnecessarily arcane.  I wonder if it isn't better to just start
over rather than trying to add yet another new class to the already
top-heavy CGI module???

Regarding file uploads: you *seem* to be proposing that uploaded files
should be loaded into memory only.  I've got complaints from people
who are uploading 10 Mb files and don't appreciate their process
growing by that much.  Perhaps there should be more options, so that
the caller can control the disposition of uploads more carefully?
What's wrong with subclassing?  Maybe two example subclasses should be
provided?

Regarding templating -- what's wrong with HTMLgen as a starting point?
Just that it's too big?  I've never used it myself, but I've always
been impressed with its appearance. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Thu Nov  2 22:12:26 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 17:12:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 05:06:16AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>
Message-ID: <20001102171226.A21364@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:06:16AM -0500, Guido van Rossum wrote:
>I just skimmed PEP 222.  I agree that the classes defined by cgi.py
>are unnecessarily arcane.  I wonder if it isn't better to just start
>over rather than trying to add yet another new class to the already
>top-heavy CGI module???

I've wondered about that, too; writing a neat request class that wraps
up field values, cookies, and environment variables, and provides
convenience functions for re-creating the current URL, Something like
the request classes in Zope and Webware.

>Regarding file uploads: you *seem* to be proposing that uploaded files
>should be loaded into memory only.  I've got complaints from people

Mrr...?  The only file upload reference simply says you shouldn't have
to subclass in order to use them; it's not implying files have to be
read into memory.  (We have to deal with whackingly large mask layout
files at work, after all.)

>Regarding templating -- what's wrong with HTMLgen as a starting point?
>Just that it's too big?  I've never used it myself, but I've always
>been impressed with its appearance. :-)

I, personally, am against including templating, but it was suggested.
I'm against it because there are too many solutions with different
tradeoffs.  Do you want a simple regex search-and-replace,
constructing HTML pages as Python objects, or a full-blown
minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
Python-compatible syntax?  Much better just to move templating into
the "Rejected" category and give the above rationale.

--amk



From jeremy@alum.mit.edu  Thu Nov  2 22:07:37 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 17:07:37 -0500 (EST)
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <20001102171226.A21364@kronos.cnri.reston.va.us>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
 <Pine.GSO.4.10.10011021727341.1290-100000@sundial>
 <20001102103442.B5027@kronos.cnri.reston.va.us>
 <200011021006.FAA19152@cj20424-a.reston1.va.home.com>
 <20001102171226.A21364@kronos.cnri.reston.va.us>
Message-ID: <14849.58793.132399.810370@bitdiddle.concentric.net>

Since today has been busy on the meta-sig, I wonder if we should
create a web-sig to thrash out these issues.

Jeremy


From thomas@xs4all.net  Thu Nov  2 22:51:52 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 23:51:52 +0100
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: <200011020500.VAA11872@sf-web2.i.sourceforge.net>; from noreply@sourceforge.net on Wed, Nov 01, 2000 at 09:00:43PM -0800
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>
Message-ID: <20001102235152.N12776@xs4all.nl>

On Wed, Nov 01, 2000 at 09:00:43PM -0800, noreply@sourceforge.net wrote:

> Comment:
> Accepted and assigned back to Thomas.

> Guido approved of this "in theory" before, so go for it!

Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
prefix, etc) and I was about to check it in, when I got cold feet. This
change is going to break programs embedding or extending python, that rely
on getopt() (the C function) being available, one way or another. After this
move, that is no longer necessarily true. I got cold feet because of
Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
optind and optarg, and doesn't bother with #include <getopt.h> or some such.

Before, that would probably always work, depending on which library/archive
was searched first, for the getopt() function. It would only break if
system-libraries were searched first, for some reason, and the system
libraries provide a broken getopt. In all other cases, like there not being
a getopt, getopt needing seperate includes or defines, or separate link or
compile arguments, the Python getopt would be used, and it would do what was
expected (probably, anyway ;)

After the change, people need to use the system getopt(), and do their own
hoop-jumping to find out if it's there, how to call it, etc. Now I myself
have never written anything that would depend on Python providing getopt(),
but that doesn't say anything at all. Is this acceptable breakage ? Or
should we go the whole nine yards, make it an official API, document it, and
provide backwards-compatible symbols on platforms where getopt used to be
used ?

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Thu Nov  2 13:41:53 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:41:53 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:12:26 EST."
 <20001102171226.A21364@kronos.cnri.reston.va.us>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>
 <20001102171226.A21364@kronos.cnri.reston.va.us>
Message-ID: <200011021341.IAA19469@cj20424-a.reston1.va.home.com>

> >Regarding file uploads: you *seem* to be proposing that uploaded files
> >should be loaded into memory only.  I've got complaints from people
> 
> Mrr...?  The only file upload reference simply says you shouldn't have
> to subclass in order to use them; it's not implying files have to be
> read into memory.  (We have to deal with whackingly large mask layout
> files at work, after all.)

Mrrauw...?  Do you really have to subclass?  I thought it just says
that you can subclass if you're not happy with the given make_file()
implementation?

> >Regarding templating -- what's wrong with HTMLgen as a starting point?
> >Just that it's too big?  I've never used it myself, but I've always
> >been impressed with its appearance. :-)
> 
> I, personally, am against including templating, but it was suggested.
> I'm against it because there are too many solutions with different
> tradeoffs.  Do you want a simple regex search-and-replace,
> constructing HTML pages as Python objects, or a full-blown
> minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
> Python-compatible syntax?  Much better just to move templating into
> the "Rejected" category and give the above rationale.

Sure -- I'm perfectly happy with ad-hoc templating solutions myself
(see my FAQ wizard).  I've also heard Tom Christiansen complain that
Perl is slower to start up than Python for CGI work -- because the
templating classes are so big, and are all loaded at startup!

I do see a use for a helper or helpers to creates tables though --
tables are notoriously tag-intensive and hard to get right.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov  2 13:51:14 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:51:14 -0500
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: Your message of "Thu, 02 Nov 2000 23:51:52 +0100."
 <20001102235152.N12776@xs4all.nl>
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>
 <20001102235152.N12776@xs4all.nl>
Message-ID: <200011021351.IAA19544@cj20424-a.reston1.va.home.com>

> Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
> prefix, etc) and I was about to check it in, when I got cold feet. This
> change is going to break programs embedding or extending python, that rely
> on getopt() (the C function) being available, one way or another. After this
> move, that is no longer necessarily true. I got cold feet because of
> Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
> optind and optarg, and doesn't bother with #include <getopt.h> or some such.
> 
> Before, that would probably always work, depending on which library/archive
> was searched first, for the getopt() function. It would only break if
> system-libraries were searched first, for some reason, and the system
> libraries provide a broken getopt. In all other cases, like there not being
> a getopt, getopt needing seperate includes or defines, or separate link or
> compile arguments, the Python getopt would be used, and it would do what was
> expected (probably, anyway ;)
> 
> After the change, people need to use the system getopt(), and do their own
> hoop-jumping to find out if it's there, how to call it, etc. Now I myself
> have never written anything that would depend on Python providing getopt(),
> but that doesn't say anything at all. Is this acceptable breakage ? Or
> should we go the whole nine yards, make it an official API, document it, and
> provide backwards-compatible symbols on platforms where getopt used to be
> used ?

Don't worry -- getopt has never been something offered by Python.
It's something assumed to be in the system library, and if it isn't we
provide our own -- but we don't provide it for the benefit of those
who embed us.  (In fact, if Python is embedded, it will not *use*
getopt.)  We can't even guarantee to provide it, because it's likely
already in the system library.  (That's why I like that the prefix
begins with an underscore -- those are not provided for use by others,
only for internal use.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Fri Nov  3 03:57:26 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 22:57:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <14849.58793.132399.810370@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 05:07:37PM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com> <20001102171226.A21364@kronos.cnri.reston.va.us> <14849.58793.132399.810370@bitdiddle.concentric.net>
Message-ID: <20001102225726.A21509@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:07:37PM -0500, Jeremy Hylton wrote:
>Since today has been busy on the meta-sig, I wonder if we should
>create a web-sig to thrash out these issues.

There's already a python-web-modules@egroups.com list, for authors 
of Python Web frameworks; fairly on-topic for that list, I should think.

--amk


From Moshe Zadka <moshez@math.huji.ac.il>  Fri Nov  3 10:18:26 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Fri, 3 Nov 2000 12:18:26 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>

On Thu, 2 Nov 2000, Guido van Rossum wrote:

> I think Python 3000 ought to use totally static scoping.  That will
> make it possible to do optimize code using built-in names!

Isn't that another way of saying you want the builtin names to be
part of the language definition? Part of today's method advantages
is that new builtins can be added without any problems.

no-clear-win-either-way-ly y'rs, Z.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Fri Nov  3 10:02:13 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:02:13 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
 <14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <3A028D25.B6A227F2@lemburg.com>

Jeremy Hylton wrote:
> 
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's just what I was trying to say all along: statically
nested scopes don't buy you anything except maybe for lambdas
and nested functions (which is bad style programming, IMHO too).

The only true argument for changing scoping I see is that
of gained purity in language design... without much practical
use.  

Other issues that need sorting out:

x = 2
class C:
  x = 1
  C = 'some string'
  def a(self):
     print x
  def b(self):
     global x
     x = 3

class D(C):
  C = 'some string'
  def a(self):
     C.a(self)
     print C

o = C()
o.a()
o.b()
o.a()

o = D()
o.a()

What would the output look like under your proposal ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Fri Nov  3 10:46:59 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:46:59 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
Message-ID: <3A0297A3.A778F4FC@lemburg.com>

Moshe Zadka wrote:
> 
> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!
> 
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

+1.

Wouldn't it be more Python-like to provide the compiler with
a set of known-to-be-static global name bindings ?

A simple way of avoiding optimizations like these:

def f(x, str=str):
   return str(x) + '!'

would then be to have the compiler lookup "str" in the globals()
passed to it and assign the found value to the constants of the
function, provided that "str" appears in the list of
known-to-be-static global name bindings (perhaps as optional
addition parameter to compile() with some reasonable default
in sys.staticsymbols).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Fri Nov  3 10:58:10 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:58:10 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
 <3A019F43.E13604BA@lemburg.com> <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <3A029A42.3C11C774@lemburg.com>

Guido van Rossum wrote:
> 
> > > > To be honest, I don't think static nested scopes buy us all that
> > > > much. You can do the same now, by using keyword arguments which
> > > > isn't all that nice, but works great and makes the scope clearly
> > > > visible.
> > >
> > > Yes.  It's a hack that gets employed over and over.  And it has
> > > certain problems.  We added 'import as' to get rid of a common
> > > practice that was perceived unclean.  Maybe we should support nested
> > > scopes to get rid of another unclean common practice?
> >
> > I think the common practice mainly comes from the fact,
> > that by making globals locals which can benefit from LOAD_FAST
> > you get a noticable performance boost.
> >
> > So the "right" solution to these weird looking hacks would
> > be to come up with a smart way by which the Python compiler
> > itself can do the localizing.
> 
> Can you elaborate?  I dun't understand what you are proposing here.

See my other post... I would like to have the compiler do the
localization for me in case it sees a global which has been
"defined" static.

> > Nested scopes won't help eliminating the current keyword
> > practice.
> 
> Why not?  I'd say that
> 
>     def create_adder(n):
>         def adder(x, n=n): return x+n
>         return adder
> 
> is a hack and that nested scopes can fix this by allowing you to write
> 
>     def create_adder(n):
>         def adder(x): return x+n
>         return adder
> 
> like one would expect.  (Don't tell me that it isn't a FAQ why this
> doesn't work!)

I know... still, I consider function definitions within a
function bad style. Maybe just me, though ;-)
 
> > > I'm not saying that we definitely should add this to 2.1 (there's
> > > enough on our plate already) but we should at least consider it, and
> > > now that we have cycle GC, the major argument against it (that it
> > > causes cycles) is gone...
> >
> > Hmm, so far the only argument for changing Python lookups
> > was to allow writing lambdas without keyword hacks. Does this
> > really warrant breaking code ?
> >
> > What other advantages would statically nested scopes have ?
> 
> Doing what's proper.  Nested scopes are not a bad idea.  They weren't
> implemented because they were hard to get right (perhaps impossible
> without creating cycles), and I was okay with that because I didn't
> like them; but I've been convinced that examples like the second
> create_adder() above should reall work.  Just like float+int works
> (in Python 0.1, this was a type-error -- you had to case the int arg
> to a float to get a float result).

Ok, but how does nested scoping mix with class definitions
and global lookups then ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Fri Nov  3 12:50:17 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 13:50:17 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
 <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
 <14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net> <3A028D25.B6A227F2@lemburg.com>
Message-ID: <3A02B489.89EF108C@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Jeremy Hylton wrote:
> >
> > Looks like we need to rehash this thread at least enough to determine
> > who is responsible for causing us to rehash it.
> >
> > MAL said it would break code.  I asked how.  Skip and Tim obliged with
> > examples.  I said their examples exhibited bad style; neither of them
> > claimed they were good style.
> >
> > In the end, I observed that while it could break code in theory, I
> > doubted it really would break much code.  Furthermore, I believe that
> > the code it will break is already obscure so we needn't worry about
> > it.
> 
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions (which is bad style programming, IMHO too).
> 
> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.
> 
> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

[Moshe pointed out to me in private mail that the above would
continue to work as it does now due to a difference being made
between class and function scoping categories]

More questions:

How are you going to explain the different
scoping categories to a newbie ? What if you define a class 
within a method ? How can you explicitely attach a dynamically
defined class to a certain scope ?

More problems (?!):

Nested scopes will introduce cycles in all frame objects.
This means that with GC turned off, frame objects will live
forever -- Python will eat up memory at a very fast pace.

BTW, Python's GC only works for a few builtin types (frames
are not among the supported types): what if a frame finds its
way into a user defined type ? Will GC still be able to cleanup
the cycles ?

Perhaps I'm just being silly, but I still don't see the
benefits of breaking todays easy-to-grasp three level
scoping rules... 

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Fri Nov  3 13:05:08 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:05:08 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 12:18:26 +0200."
 <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
Message-ID: <200011031305.IAA22103@cj20424-a.reston1.va.home.com>

> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!

[Moshe]
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

The built-in names have always been part of the language definition in
my mind.  The way they are implemented doesn't reflect this, but
that's just an implementation detail.  How would you like it if
something claimed to be Python but didn't support len()?  Or map()?

That doesn't mean you can't add new built-ins, and I don't think that
the new implementation will prevent that -- but it *will* assume that
you don't mess with the definitions of the existing built-ins.

Of course you still will be able to define functions whose name
overrides a built-in -- in that case the compiler can see that you're
doing that (because it knows the scope rules and can see what you are
doing).  But you won't be able to confuse someone else's module by
secretly sticking a replacement built-in into their module's __dict__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Nov  3 13:12:11 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:12:11 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 11:02:13 +0100."
 <3A028D25.B6A227F2@lemburg.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>
 <3A028D25.B6A227F2@lemburg.com>
Message-ID: <200011031312.IAA22141@cj20424-a.reston1.va.home.com>

[MAL]
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions

That's a tautology -- that's what nested scopes are FOR!

> (which is bad style programming, IMHO too).

Not always.  The keyword argument hack is so common that it must serve
a purpose, and that's what we're trying to fix -- for lambda's *and*
nested functions (which are semantically equivalent anyway).

> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.

But there is practical use: get rid of the unintuitive, unobvious and
fragile keyword argument hack.

> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

This is a good point!  If we considered the class as a nested scope
here, I think it might break too much code, plus it would allow a new
coding style where you could reference class variables without a self
or <classname> prefix.  I don't like that prospect, so I'm in favor
for ruling this out.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@wooz.org  Fri Nov  3 14:16:20 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 09:16:20 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
 <200011031305.IAA22103@cj20424-a.reston1.va.home.com>
Message-ID: <14850.51380.7327.977719@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> The built-in names have always been part of the language
    GvR> definition in my mind.  The way they are implemented doesn't
    GvR> reflect this, but that's just an implementation detail.  How
    GvR> would you like it if something claimed to be Python but
    GvR> didn't support len()?  Or map()?

    GvR> That doesn't mean you can't add new built-ins, and I don't
    GvR> think that the new implementation will prevent that -- but it
    GvR> *will* assume that you don't mess with the definitions of the
    GvR> existing built-ins.

    GvR> Of course you still will be able to define functions whose
    GvR> name overrides a built-in -- in that case the compiler can
    GvR> see that you're doing that (because it knows the scope rules
    GvR> and can see what you are doing).  But you won't be able to
    GvR> confuse someone else's module by secretly sticking a
    GvR> replacement built-in into their module's __dict__.

I'm a little confused.  I've occasionally done the following within an
application:

----------driver.py
# need to override built-in open() to do extra debugging
def debuggin_open(filename, mode, bufsize):
    # ...
if EXTRA_DEBUGGING:
    import __builtin__.__dict__['open'] = debugging_open
-------------------- snip snip --------------------

Would this be illegal?  Would other modules in my application (even if
imported from the standard library!) automatically get
debugging_open() for open() like they do now?

-Barry


From gward@mems-exchange.org  Fri Nov  3 14:33:03 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 09:33:03 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sat, Oct 28, 2000 at 12:51:20PM +0200
References: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>
Message-ID: <20001103093303.A2683@ludwig.cnri.reston.va.us>

On 28 October 2000, Martin v. Loewis said:
> > I don't have access to a Solaris machine, so I can't do anything to
> > help these users.
> 
> The patch in 117606 looks right to me: gcc on Solaris (and on any
> other platform) needs -shared to build shared library; configure
> currently passes -G. I haven't actually tried the patch, since it is a
> pain to extract it from the SF bug report page. What happens is that
> gcc passes -G to the linker, which correctly produces a shared
> library.  However, gcc also links crt1/crti into the library, which
> causes the reference to main.

Well, I do have access to a Solaris machine -- I try not to use it if I
don't have to, and about the only purpose it has these days is
occasionally building Python to make sure it still works.

Incidentally, I'm the one who changed "ld -G" to "$(CC) -G" -- see
revision 1.124 of configure.in:

  revision 1.124
  date: 2000/05/26 12:22:54;  author: gward;  state: Exp;  lines: +6 -2
  When building on Solaris and the compiler is GCC, use '$(CC) -G' to
  create shared extensions rather than 'ld -G'.  This ensures that shared
  extensions link against libgcc.a, in case there are any functions in the
  GCC runtime not already in the Python core.

I think the checkin message there is fairly clear; the context was in
using Robin Dunn's extension for BSDDB 2.x, which does some 64-bit
arithmetic deep down inside.  Turned out that GCC compiled a 64-bit
divide into a function call, and that function is in GCC's own runtime
library.  Using "ld -G" -- that's Sun's linker, which knows nothing
about GCC's runtime library -- the function in question wasn't
available, so loading the extension failed.  I assume that if *Python*
did a 64-bit divide somewhere in *its* guts, that function would have
been available (linked into the python binary), which is why this
problem doesn't come up very often -- Python probably does use most of
GCC's runtime.  ;-)

Anyways, I thought the patch in bug #117606 looked fine, so I tried it
out.  Not so good; here's what happens when I try to build
arraymodule.so (the first extension, alphabetically) with "gcc -shared":

  Text relocation remains                         referenced
      against symbol                  offset      in file
  _PyObject_NewVar                    0x654       arraymodule.o
  <unknown>                           0x26cc      arraymodule.o
  <unknown>                           0x26c8      arraymodule.o
  [...many many <unknown> symbols...]
  PyErr_Occurred                      0x1274      arraymodule.o
  PyErr_Occurred                      0x4a0       arraymodule.o
  PyErr_Occurred                      0x22d8      arraymodule.o
  PyErr_Occurred                      0x115c      arraymodule.o
  PyErr_Occurred                      0x368       arraymodule.o
  PyErr_Occurred                      0x1074      arraymodule.o
  PyErr_Occurred                      0x1f50      arraymodule.o
  PyInt_FromLong                      0x3f4       arraymodule.o
  [...]
  _Py_NoneStruct                      0x19d4      arraymodule.o
  Py_InitModule4                      0x26b8      arraymodule.o
  ld: fatal: relocations remain against allocatable but non-writable sections
  collect2: ld returned 1 exit status

All told, there were 500+ relocation errors in that one extension.
About half where "Py" or "_Py" symbols; a bunch were "<unknown>", and
the rest were 'malloc', 'memcpy', 'sprintf', and other standard library
functions.  So apparently "-shared" tells GCC to forget everything it
knows about linking C code and be as stupid as it can.  Hmmm.

I tried adding "../libpython2.0.a", and "-L.. -lpython2.0", and instead
got 20,000 relocation errors, since of course libpython2.0.a needs a lot
more symbols than arraymodule.o.  Hmmm.

I have no idea what's going on here.  I've updated the bug report, and I
am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless, of
course, there are other linker options that make it work right...

        Greg


From gmcm@hypernet.com  Fri Nov  3 14:48:29 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 09:48:29 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14850.51380.7327.977719@anthem.concentric.net>
Message-ID: <3A0289ED.6247.EB4DD0E@localhost>

> 
> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

>     GvR> Of course you still will be able to define functions whose
>     GvR> name overrides a built-in -- in that case the compiler can
>     GvR> see that you're doing that (because it knows the scope rules
>     GvR> and can see what you are doing).  But you won't be able to
>     GvR> confuse someone else's module by secretly sticking a
>     GvR> replacement built-in into their module's __dict__.

[A Craven Dog overrides builtin open...]

> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

I have always found it very elegant & powerful that keyword 
"import" calls builtin __import__, and that well-behaved C goes 
through the same hook. In a similar vein, I have for quite 
awhile wanted to experiment with mountable virtual file 
systems so that I can "mount" a (for example) MetaKit 
database at /cheesewhiz and other modules in my app, when 
navigating into /cheesewhiz will, unbeknownst to them, be 
reading from the MetaKit DB (these ideas leaked from tcl-land 
by Jean-Claude).

I most definitely appreciate that these facilities are dangerous 
and this type of stuff tends to be abused gratuitously (eg, 
most import hacks), but disallowing them might be considered 
a gratuitous limitation (heck - Barry knows how to bypass the 
governor on every cheezewhizzer ever manufactured).

- Gordon


From guido@python.org  Fri Nov  3 15:36:22 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 10:36:22 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 09:16:20 EST."
 <14850.51380.7327.977719@anthem.concentric.net>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com>
 <14850.51380.7327.977719@anthem.concentric.net>
Message-ID: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>

[Barry]
> I'm a little confused.  I've occasionally done the following within an
> application:
> 
> ----------driver.py
> # need to override built-in open() to do extra debugging
> def debuggin_open(filename, mode, bufsize):
>     # ...
> if EXTRA_DEBUGGING:
>     import __builtin__.__dict__['open'] = debugging_open
> -------------------- snip snip --------------------
> 
> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

That's up for discussion.  Note that the open() function is special in
this respect -- I don't see you doing the same to range() or hash().

If this is deemed a useful feature (for open()), we can make a rule
about which built-ins you cannot override like this and which ones you
can.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@wooz.org  Fri Nov  3 15:45:05 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 10:45:05 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
 <200011031305.IAA22103@cj20424-a.reston1.va.home.com>
 <14850.51380.7327.977719@anthem.concentric.net>
 <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14850.56705.120643.110016@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> That's up for discussion.  Note that the open() function is
    GvR> special in this respect -- I don't see you doing the same to
    GvR> range() or hash().

Me neither, but special casing overridability seems like a fragile
hack.

    GvR> If this is deemed a useful feature (for open()), we can make
    GvR> a rule about which built-ins you cannot override like this
    GvR> and which ones you can.

Hmm, maybe we need __open__() and an open-hook? ;)

-Barry


From guido@python.org  Fri Nov  3 16:07:13 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 11:07:13 -0500
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
Message-ID: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>

At our PythonLabs group meeting last Tuesday we decided that we needed
to set up a Patches page and a FAQ for Python 2.0.  (This because we
don't see a reason yet to issue a bugfix release, but we do need to
answer some common questions and point people to some patches.)

I figured that we could do ourselves a favor by making this a set of
dynamic pages maintained wiki-style, rather than static pages
(especially since I've managed to personally become the bottleneck for
editing the static pages, until the move of python.org to a DC hosted
machine is complete :-).

So, when I saw the announcement of MoinMoin, a Wiki clone written in
Python as a single CGI script, I decided to try it -- instead of
ZWiki, which might be the obvious choice given my new employer.  This
is not because I don't like Zope or ZWiki, but because we can learn
from using different implementations of the same idea.

So, I humbly present the Python 2.0 Info Area: 

    http://www.python.org/cgi-bin/moinmoin

I've added one critical patch, three non-critical (misc) patches,
answers to two frequent bug reports to the FAQ page, and a bunch of
links to the front page.

Note that to generate patches, I use SourceForge cvsweb's diff
feature.  The URLs are ugly, but only the page editors see these, and
it saves having to store copies of the patches.

I'd like to get some feedback from the python-dev group before I link
it into the 2.0 home page.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Fri Nov  3 16:12:04 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:12:04 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 10:36:22AM -0500
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <20001103171203.G28658@xs4all.nl>

On Fri, Nov 03, 2000 at 10:36:22AM -0500, Guido van Rossum wrote:
> [Barry]

> > Would this [replacing builtins] be illegal?

> That's up for discussion.  Note that the open() function is special in
> this respect -- I don't see you doing the same to range() or hash().

Eep. I don't care much about scoping, so I'm not going to say much about
that, but I certainly do care about Python's flexibility. One of the great
things is that there are so little special cases, that (nearly) everything
is delightfully consistent. Being able to override open() but not hash() or
range() sounds directly contrary to that, at least to me. Being able to
change __builtins__ *just like any other dict* strikes me as terribly
Pythonic, though I realize this is probably contrary to Guido's view of
Python :-) It also makes for instance 'exec' and 'eval' terribly obvious.

I understand where the wish to restrict replacement and shadowing of
builtins comes from, but I'm hoping here that this is going to be
'optional', like Perl's -w and 'use strict'. Defaulting to maximum
simplicity and maximum flexibility (in that order:) but with optional
warnings (when shadowing/modifying a builtin) and optimizations (using
constants for builtins, when not modifying or shadowing them, for instance.)

Just-my-fl.0,04-ly y'rs,
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Fri Nov  3 16:31:07 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:31:07 +0100
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 11:07:13AM -0500
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <20001103173107.H28658@xs4all.nl>

On Fri, Nov 03, 2000 at 11:07:13AM -0500, Guido van Rossum wrote:

> So, I humbly present the Python 2.0 Info Area: 

>     http://www.python.org/cgi-bin/moinmoin

> I've added one critical patch, three non-critical (misc) patches,

Looks good. I'd never seen a Wiki thing before, and the naming requirements
takes a bit getting used to, but I think it looks great, regardless :)

It also reminds me that we still need to fix the erroneous conclusion by
configure that BSDI has large file support, just because it has an off_t
type that is 64 bit. (It does have that, it just doesn't use that in any of
the seek/tell functions available.) Trent, you were making noises about
looking at it, when I left for ApacheCon. Did you ever get to look at it ?
If not, I'll see if I can figure it out ;P Once it's fixed, I think it
should be added to CriticalPatches, but it might not be as straightforward
as pointing to a cvsweb URL ;P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From loewis@informatik.hu-berlin.de  Fri Nov  3 18:51:49 2000
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Fri, 3 Nov 2000 19:51:49 +0100 (MET)
Subject: [Python-Dev] build problems under Solaris
Message-ID: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>

>  Text relocation remains                         referenced
>      against symbol                  offset      in file
[...]

> I have no idea what's going on here.  I've updated the bug report,
> and I am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless,
> of course, there are other linker options that make it work right...

That happens only when using the system linker (/usr/ccs/bin/ld). The
GNU linker won't complain, and the resulting executables will run
fine.

To make the system linker happy, you also need to compile the modules
with -fPIC - which, according to Sun, we should have done all the time
when building shared libraries. The linker complains about relocations
in the text segment, which shouldn't be there - shared libraries
should be position independent.

Alternatively, you can use 'gcc -shared -mimpure-text'; that does not
request that remaining relocations cause errors.

Regards,
Martin


From mal@lemburg.com  Fri Nov  3 19:08:17 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 20:08:17 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
Message-ID: <3A030D21.413B8C8A@lemburg.com>

I'm having trouble opening ZIP files created using InfoZIP's
zip utility (which uses zlib) with zipfile.py:

>>> x = z.read('README')
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "/home/lemburg/lib/zipfile.py", line 242, in read
    bytes = dc.decompress(bytes)
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Is this due to the installed zlib on my system being incompatible,
or is this a bug in zipfile.py ? I have libz version 1.1.3 and
zip version 2.2.

Also, I wonder why zipfile forces the mode flag to be 'r',
'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
to the mode flag instead ?!

The ZipFile is also missing some kind of method which
extracts files in the ZIP archive to a file-like object. This
would be very useful for extracting large files from a ZIP
archive without having to first read in the whole file into
memory.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From skip@mojam.com (Skip Montanaro)  Fri Nov  3 21:04:44 2000
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:04:44 -0600 (CST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
 <200011031305.IAA22103@cj20424-a.reston1.va.home.com>
 <14850.51380.7327.977719@anthem.concentric.net>
 <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14851.10348.715861.488845@beluga.mojam.com>

    Guido> If this is deemed a useful feature (for open()), we can make a
    Guido> rule about which built-ins you cannot override like this and
    Guido> which ones you can.

I thought we were all adults...

For Py3k I think it should be sufficient to define the semantics of the
builtin functions so that if people want to override them they can, but that
overriding them in incompatible ways is likely to create some problems.
(They might have to run with a "no optimize" flag to keep the compiler from
assuming semantics, for instance.)  I see no particular reason to remove the
current behavior unless there are clear instances where something important
is not going to work properly.

Modifying builtins seems to me to be akin to linking a C program with a
different version of malloc.  As long as the semantics of the new functions
remain the same as the definition, everyone's happy.  You can have malloc
leave a logfile behind or keep histograms of allocation sizes.  If someone
links in a malloc library that only returns a pointer to a region that's
only half the requested size though, you're likely to run into problems.

Skip



From skip@mojam.com (Skip Montanaro)  Fri Nov  3 21:17:12 2000
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:17:12 -0600 (CST)
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <14851.11096.337480.434853@beluga.mojam.com>

    Guido> I'd like to get some feedback from the python-dev group before I
    Guido> link it into the 2.0 home page.

Looks good to me.

Skip


From mal@lemburg.com  Fri Nov  3 20:14:17 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:14:17 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
 <200011031305.IAA22103@cj20424-a.reston1.va.home.com>
 <14850.51380.7327.977719@anthem.concentric.net>
 <200011031536.KAA22620@cj20424-a.reston1.va.home.com> <14851.10348.715861.488845@beluga.mojam.com>
Message-ID: <3A031C99.7F7DD8E2@lemburg.com>

[Making builtins static...]

What about the idea to have the compiler make the decision
whether a global symbol may be considered static based on a 
dictionary ?

In non-optimizing mode this dictionary would be empty, with -O
it would include all builtins which should never be overloaded
and with -OO even ones which can be overloaded such as open()
in addition to some standard modules which are known to only
contain static symbols.

Perhaps this needs some additional help of the "define" statement
we planned as dynamic compiler interface ?! ...

define static_symbols = *            # all globals in this module
define static_symbols = func1, func2 # only these two
etc.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Fri Nov  3 20:31:51 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 15:31:51 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:04:44 CST."
 <14851.10348.715861.488845@beluga.mojam.com>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
 <14851.10348.715861.488845@beluga.mojam.com>
Message-ID: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>

>     Guido> If this is deemed a useful feature (for open()), we can make a
>     Guido> rule about which built-ins you cannot override like this and
>     Guido> which ones you can.

[Skip]
> I thought we were all adults...

And consenting as well... :-)

> For Py3k I think it should be sufficient to define the semantics of the
> builtin functions so that if people want to override them they can, but that
> overriding them in incompatible ways is likely to create some problems.
> (They might have to run with a "no optimize" flag to keep the compiler from
> assuming semantics, for instance.)  I see no particular reason to remove the
> current behavior unless there are clear instances where something important
> is not going to work properly.
> 
> Modifying builtins seems to me to be akin to linking a C program with a
> different version of malloc.  As long as the semantics of the new functions
> remain the same as the definition, everyone's happy.  You can have malloc
> leave a logfile behind or keep histograms of allocation sizes.  If someone
> links in a malloc library that only returns a pointer to a region that's
> only half the requested size though, you're likely to run into problems.

Actually, the C standard specifically says you are *not* allowed to
override standard library functions like malloc().

I'm thinking of the example of the rules in Fortran for intrinsic
functions (Fortran's name for built-ins).  Based on what Tim has told
me, I believe that Fortran by default assumes that you're not doing
anything funky with intrinsics (like sin, cos, tan) it can use a
shortcut, e.g. inline them.  But there are also real functions by
these names in the Fortran standard library, and you can call those by
declaring e.g. "external function sin".  (There may also be an
explicit way to say that you're happy with the intrinsic one.)  I
believe that when you use the external variant, they may be overridden
by the user.

I'm thinking of something similar here for Python.  If the bytecode
compiler knows that the builtins are vanilla, it can generate better
(== more efficient) code for e.g.

   for i in range(10):
       ...

Ditto for expressions like len(x) -- the len() operation is typically
so fast that the cost is dominated by the two dict lookup operations
(first in globals(), then in __builtins__).

Why am I interested in this?  People interested in speed routinely use
hacks that copy a built-in function into a local variable so that they
don't have dictlookups in their inner loop; it's really silly to have
to do this, and if certain built-ins were recognized by the compiler
it wouldn't be necessary.  There are other cases where this is not so
easy without much more analysis; but the built-ins (to me) seem
low-hanging fruit.  (Search the archives for that term, I've used it
before in this context.)

I assume that it's *really* unlikely that there are people patching
the __builtin__ module to replace the functions that are good inline
candidates (range, len, id, hash and so on).  So I'm interesting in
complicating the rules here.  I'd be happy to make an explicit list of
those builtins that should not be messed with, as part of the language
definition.  Program that *do* mess with these have undefined
semantics.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Nov  3 20:50:48 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:50:48 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test/output
 test_unicodedata,1.3,1.4
References: <200011032024.MAA26242@slayer.i.sourceforge.net>
Message-ID: <3A032528.D743FC0A@lemburg.com>

Fredrik Lundh wrote:
> 
> Update of /cvsroot/python/python/dist/src/Lib/test/output
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv25791/lib/test/output
> 
> Modified Files:
>         test_unicodedata
> Log Message:
> 
> Added 38,642 missing characters to the Unicode database (first-last
> ranges) -- but thanks to the 2.0 compression scheme, this doesn't add
> a single byte to the resulting binaries (!)
> 
> Closes bug #117524

Cool :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From gmcm@hypernet.com  Fri Nov  3 20:56:56 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 15:56:56 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 03 Nov 2000 15:04:44 CST."             <14851.10348.715861.488845@beluga.mojam.com>
Message-ID: <3A02E048.17812.10062D77@localhost>

> >     Guido> If this is deemed a useful feature (for open()), we can make a
> >     Guido> rule about which built-ins you cannot override like this and
> >     Guido> which ones you can.

I think I would be satisfied with just those builtins which involve 
interfaces to the external world. Where Java allows such 
deviance, they tend to provide an API whereby you can supply 
or register a factory to override or extend the default behavior.

In principle this seems less hackish than stomping on builtins; 
in practice I doubt it makes much difference ;-).
 
> [Skip]
> > I thought we were all adults...

Snort. One of my kids will be voting on Tuesday, but I *still* 
don't know what I want to be when I grow up.

- Gordon


From trentm@ActiveState.com  Fri Nov  3 20:51:41 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Fri, 3 Nov 2000 12:51:41 -0800
Subject: 64-bit stuff on BSDI (was: Re: [Python-Dev] New Wiki-based Python 2.0 Info Area)
In-Reply-To: <20001103173107.H28658@xs4all.nl>; from thomas@xs4all.net on Fri, Nov 03, 2000 at 05:31:07PM +0100
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com> <20001103173107.H28658@xs4all.nl>
Message-ID: <20001103125141.I20329@ActiveState.com>

On Fri, Nov 03, 2000 at 05:31:07PM +0100, Thomas Wouters wrote:
> It also reminds me that we still need to fix the erroneous conclusion by
> configure that BSDI has large file support, just because it has an off_t
> type that is 64 bit. (It does have that, it just doesn't use that in any of
> the seek/tell functions available.) Trent, you were making noises about
> looking at it, when I left for ApacheCon. Did you ever get to look at it ?

No. I was in Washington D.C. for the past week at SD 2000 (and getting
occassionally verbally abused by Tim about 64-bit stuff).

> If not, I'll see if I can figure it out ;P 

I can answer questions about fixing it but you can test it, so better for you
to submit the patch.

> Once it's fixed, I think it
> should be added to CriticalPatches, but it might not be as straightforward
> as pointing to a cvsweb URL ;P
>

Sure. Yes, it will probably be more than one file (configure.in, configure,
and fileobject.c).  Whatever, I don't think it should be a problem to put up
the actually checkin patch rather than a link.

Thanks!
Trent


-- 
Trent Mick
TrentM@ActiveState.com


From gward@mems-exchange.org  Fri Nov  3 22:45:00 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 17:45:00 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>; from loewis@informatik.hu-berlin.de on Fri, Nov 03, 2000 at 07:51:49PM +0100
References: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>
Message-ID: <20001103174500.B6683@ludwig.cnri.reston.va.us>

On 03 November 2000, Martin von Loewis said:
> That happens only when using the system linker (/usr/ccs/bin/ld). The
> GNU linker won't complain, and the resulting executables will run
> fine.

I'm not sure which linker my GCC on Solaris is using.  (Even though,
umm, I built that GCC.  Errr...)  But ISTR that the choice of linker is
made when you build GCC, so it's not really an option here.  Bad enough
to tell people that they need to reinstall Python because they need some
extension; requiring them to rebuild GCC -- !

But it's not necessary at all, because...

> To make the system linker happy, you also need to compile the modules
> with -fPIC - which, according to Sun, we should have done all the time
> when building shared libraries. The linker complains about relocations
> in the text segment, which shouldn't be there - shared libraries
> should be position independent.

...compiling everything with -fPIC is exactly what the doctor ordered.
I added "-fPIC" to OPT in the Makefile and rebuilt, and everything went
smoothly when linking the extensions with "gcc -shared".  No problems
compiling or linking, and the tests are running right now without a
hitch.  Yaayyyyh!

So here's an update of the patch: this changes LDSHARED to "$(CC)
-shared" when using GCC under Solaris, and it adds "-fPIC" to OPT when
using GCC *anywhere*.  This seems like a good thing to me when building
shared objects, but if anyone is aware of a platform where "gcc
... -fPIC" is a bad idea, speak up now!

*** configure.in	2000/11/03 08:18:36	1.177
--- configure.in	2000/11/03 22:43:50
***************
*** 308,315 ****
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes";;
  		esac
  		;;
  	*)	OPT="-O";;
--- 308,315 ----
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes -fPIC";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes -fPIC";;
  		esac
  		;;
  	*)	OPT="-O";;
***************
*** 564,570 ****
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -G'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;
--- 564,570 ----
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -shared'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;


I'll go update the bug report now.  Thanks, Martin!

        Greg


From guido@python.org  Fri Nov  3 23:01:27 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 18:01:27 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
Message-ID: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>

Moshe and Andrew (the impatient youth :-) have asked what's on our
plate for Python 2.1.


Schedule
--------

The current plan for the 2.1 release is in PEP 226:

    http://python.sourceforge.net/peps/pep-0226.html

According to that PEP, the tentative schedule is:

    16-Dec-2000: 2.1 PEPs ready for review
    01-Feb-2001: First 2.1 beta release
    16-Mar-2001: 2.1 final release

But the PEP doesn't really say much about which PEPs we're going to
work on.


Overview of the PEPs
--------------------

In my recent reworking of PEP 0 (the PEP index) I created a category
of Active PEPs -- these are PEPs that I definitely want to consider in
Python 2.1:

 I    42  pep-0042.txt  Small Feature Requests                 Hylton
 S   207  pep-0207.txt  Rich Comparisons                       Ascher
 S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
 S   217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 S   222  pep-0222.txt  Web Library Enhancements               Kuchling
 I   226  pep-0226.txt  Python 2.1 Release Schedule            Hylton
 S   227  pep-0227.txt  Statically Nested Scopes               Hylton

Note that I said *consider*: I'm sure that not all feature requests
from PEP 42 (a grab-bag of stuff that may or may not be feasible to
implement) will be implemented, and I'm not sure that all the other
PEPs on the list will be (especially nested scopes still seems iffy).

Two of these (207, 208) haven't been written yet -- but I know roughly
what they will contain, so they are included in this list, and not in
the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
PEPs).  These may be reconsidered for Python 2.1 if their authors care
to follow the PEP guidelines.

There are a bunch of other PEPs that I moved to the Pie-in-the-sky
category -- these are form-complete PEPs, but they are controversial
(e.g. there are two competing matrix op PEPs); some others I think are
not important; for yet others I think it's unrealistic to expect them
to be implemented by Python 2.1.  (There are also other things I'd
like to do that fit in the pie-in-the-sky category, like breaking up
the interpreter in several replaceable pieces.)


Other issues to work on
-----------------------

These aren't PEPs yet, but I think they need to become PEPs soon --
I'd like to see work on them go into Python 2.1:

- The buffer interface needs a revamping or simplification; this was
  discussed here previously.

- A warning framework.  I've got some ideas already.

- Integer division.  If we want to play by Paul Prescod's Language
  Evolution rules (PEP 5), we better get started on the first stage.
  E.g. we could introduce a // operator in 2.1 for integer division,
  and issue a warning when / is used for integers.  Then a year later
  (i.e., March 2002!) we could change / so that it returns a floating
  point number.

- Case sensitivity???

- Class/type dichotomy???

- Weak references.  This *is* a PEP, but there's no contents yet.  We
  could also try to implement (just) weak dictionaries.

- Internationalization.  Barry knows what he wants here; I bet Martin
  von Loewis and Marc-Andre Lemburg have ideas too.

- Arbitrart attributes on functions?  This would be a generalization
  of docstrings; with the intent that you don't have to put semantics
  in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
  This could possibly lead to implementing private, protected, public
  attributes too.

- Whatever you want to work on.  If you have an idea for a PEP that
  you think should be implemented in Python 2.1, or if you want to
  revive a PEP that's currently listed in one of the "unattainable"
  categories, now's the time to make a plea!


--Guido van Rossum (home page: http://www.python.org/~guido/)


From gstein@lyra.org  Fri Nov  3 23:13:50 2000
From: gstein@lyra.org (Greg Stein)
Date: Fri, 3 Nov 2000 15:13:50 -0800
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103151350.C12266@lyra.org>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>...
> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'd like to write up a free-threading PEP, but with all of my work on
Subversion right now, I'm not sure when that writing will occur or the
coding. I'll certainly do the PEP first because it could also be used as a
development roadmap for somebody do some of the work.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gward@mems-exchange.org  Fri Nov  3 23:17:35 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 18:17:35 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
Message-ID: <20001103181734.A6809@ludwig.cnri.reston.va.us>

Hi all --

since "-Wall -Wstrict-prototypes" were added to OPT by default, a bunch
of warnings are showing up with GCC on Solaris.  (I haven't seen them on
Linux, so I assume this is OS-specific.)  See PR#121479 for a complete
list:
   https://sourceforge.net/bugs/?func=detailbug&bug_id=121479&group_id=5470)

Is anyone else looking into these?  Some of them are easy to fix,
eg. cast char to int when using 'isalnum()' and friends.  Some of them
are easy but unnecessary -- eg. sham variable initializations to shut up
the "might be used uninitialized" warning.  Others seem to require
#define'ing XOPEN_SOURCE or __EXTENSIONS__, which was a bit
controversial when it was done on Linux... but may be needed on Solaris
too.  I'm not sure yet; I've just started looking at it.  And I have to
run now.

Anyways, if anyone has guidelines/hints/suggestions for the best way to
fix these warnings, I'm all ears.  Also, if there are warnings we're not
going to worry about (eg. incorrect "might be used uninitialized"),
lemme know.

Thanks --

        Greg


From DavidA@ActiveState.com  Fri Nov  3 23:38:13 2000
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 3 Nov 2000 15:38:13 -0800 (Pacific Standard Time)
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>

>  S   207  pep-0207.txt  Rich Comparisons                       Ascher
>  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

> Two of these (207, 208) haven't been written yet -- but I know roughly
> what they will contain, so they are included in this list, and not in
> the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> PEPs).  These may be reconsidered for Python 2.1 if their authors care
> to follow the PEP guidelines.

I would _love_ to find someone to take over PEP ownership of these.  I
feel terrible that I haven't been able to find the time to do those right
(although I only feel moral ownership of 207, not 208, where others have
much more in-depth knowledge).

If someone wants to take over, please speak up and grab them.  I'll try to
find the time to share the information I have, would gladly give the very
early and now obsolete patches I wrote up.

--david
 



From akuchlin@mems-exchange.org  Sat Nov  4 01:31:05 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 3 Nov 2000 20:31:05 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103203105.A25097@kronos.cnri.reston.va.us>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>- Whatever you want to work on.  If you have an idea for a PEP that
>  you think should be implemented in Python 2.1, or if you want to

Using Distutils to compile most of the extension modules, taking the
first step to eliminating the Setup/Makefile.pre.in scheme.  I'll
begin drafting a PEP.

--amk



From guido@python.org  Sat Nov  4 03:38:17 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 22:38:17 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:38:13 PST."
 <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
Message-ID: <200011040338.WAA26703@cj20424-a.reston1.va.home.com>

> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

[David]
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).

I will take care of these myself if noone else picks them up.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Sat Nov  4 07:07:43 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Sat, 4 Nov 2000 02:07:43 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <20001103181734.A6809@ludwig.cnri.reston.va.us>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>

[Greg Ward]
> ...
> Also, if there are warnings we're not going to worry about (eg.
> incorrect "might be used uninitialized"), lemme know.

If a compiler is afraid something might be uninitialized, the code is too
clever for people to be sure it's correct at a glance too.  Note that right
before 2.0 was released, a bogus "uninitalized" msg got repaired, which
turned up a *legitimate* "uninitialized" msg that the bogus msg was covering
up.  The effort needed to fix one of these is too minor to be worth even
considering not fixing.

I'm not sure what gcc is complaining about in many of the cases; others are
quite clear (e.g., "confstr" apparently has no prototype in scope by the
time it's called in posixmodule.c, and that is indeed not good).

unix-should-be-shot-not-that-windows-shouldn't-ly y'rs  - tim




From py-dev@zadka.site.co.il  Sat Nov  4 18:00:49 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:00:49 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>

Here's a draft PEP.
Have I already mentioned how much it irks me to cater for other
editors in the PEP text itself?

PEP: Unassigned
Title: Reworking Python's Numeric Model
Version: $Revision$
Author: pep@zadka.site.co.il (Moshe Zadka)
Status: Draft
Type: Standards Track
Created: 4-Nov-2000
Post-History:

Abstract

    Today, Python's numerical model is similar to the C numeric model: 
    there are several unrelated numerical types, and when operations 
    between numerical types are requested, coercions happen. While the C 
    rational for the numerical model is that it is very similar to what
    happens on the hardware level, that rational does not apply to Python.
    So, while it is acceptable to C programmers that 2/3 == 0, it is very
    surprising to Python programmers.

Rationale

    In usability studies, one of Python hardest to learn features was
    the fact integer division returns the floor of the division. This
    makes it hard to program correctly, requiring casts to float() in
    various parts through the code. Python numerical model stems from
    C, while an easier numerical model would stem from the mathematical
    understanding of numbers.

Other Numerical Models

    Perl's numerical model is that there is one type of numbers -- floating
    point numbers. While it is consistent and superficially non-suprising,
    it tends to have subtle gotchas. One of these is that printing numbers
    is very tricky, and requires correct rounding. In Perl, there is also
    a mode where all numbers are integers. This mode also has its share of
    problems, which arise from the fact that there is not even an approximate
    way of dividing numbers and getting meaningful answers.

Suggested Interface For Python Numerical Model

    While coercion rules will remain for add-on types and classes, the built
    in type system will have exactly one Python type -- a number. There
    are several things which can be considered "number methods":

    1. isnatural()
    2. isintegral()
    3. isrational()
    4. isreal()
    5. iscomplex()

    a. isexact()

    Obviously, a number which answers m as true, also answers m+k as true.
    If "isexact()" is not true, then any answer might be wrong. (But not
    horribly wrong: it's close the truth).

    Now, there is two thing the models promises for the field operations
    (+, -, /, *): 

        If both operands satisfy isexact(), the result satisfies isexact()

        All field rules are true, except that for not-isexact() numbers,
        they might be only approximately true.      

    There is one important operation, inexact() which takes a number
    and returns an inexact number which is a good approximation.

    Several of the classical Python operations will return exact numbers
    when given inexact numbers: e.g, int().

Inexact Operations

    The functions in the "math" module will be allowed to return inexact
    results for exact values. However, they will never return a non-real
    number. The functions in the "cmath" module will return the correct
    mathematicl result.

Numerical Python Issues

    People using Numerical Python do that for high-performance
    vector operations. Therefore, NumPy should keep it's hardware
    based numeric model.

Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
-- 
Moshe Zadka <sig@zadka.site.co.il>


From mal@lemburg.com  Sat Nov  4 09:50:10 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:50:10 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
Message-ID: <3A03DBD2.7E024978@lemburg.com>

David Ascher wrote:
> 
> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
> 
> > Two of these (207, 208) haven't been written yet -- but I know roughly
> > what they will contain, so they are included in this list, and not in
> > the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> > PEPs).  These may be reconsidered for Python 2.1 if their authors care
> > to follow the PEP guidelines.
> 
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).
> 
> If someone wants to take over, please speak up and grab them.  I'll try to
> find the time to share the information I have, would gladly give the very
> early and now obsolete patches I wrote up.

I can take over the coercion PEP: I've been working
on this before (see the proposal on my Python Pages).

I would also like to know whether the PEP-0224 will be considered
for 2.1 if I update the patch to make it a little more robust
w/r to the problems mentioned in that PEP -- I'd really like
to see this in Python soon, since it makes documenting Python
programs so much easier.

Note that I won't get around to do much work on these before
January... way too busy at the moment :-/

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From py-dev@zadka.site.co.il  Sat Nov  4 17:42:48 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 19:42:48 +0200
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Fri, 03 Nov 2000 18:01:27 EST." <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <E13s7Lz-0002m1-00@darjeeling.zadka.site.co.il>

> Moshe and Andrew (the impatient youth :-) have asked what's on our
> plate for Python 2.1.

Yeah, sure. They're not making enough fun of my age where I work, <wink>.

> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

I'm working on that one, and will submit a PEP draft ASAP. 
I do want to know what kind of changes are acceptable: it seems you have
no problem with just patching over the numerical model, while I think
that solving this problem without recreating a host of others needs
total reworking of the numerical model.

> - Case sensitivity???

Writing a PEP for this seems trivial, but I thought the consensus was
that this should be solved by tools, not the language.

> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'm not sure I want to work on it, but in the past, we through around
ideas for pluggable nanny architecture. This is related to both
the case-sensitivity discussion above, and we also had lots of FAQ-stopping
nannies. (self-nanny, e.g., was designed to be a FAQ stopper)
-- 
Moshe Zadka <sig@zadka.site.co.il>


From mal@lemburg.com  Sat Nov  4 09:58:49 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:58:49 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03DDD9.8527F7BE@lemburg.com>

Moshe Zadka wrote:
> 
> Here's a draft PEP.
> Have I already mentioned how much it irks me to cater for other
> editors in the PEP text itself?
> 
> PEP: Unassigned
> Title: Reworking Python's Numeric Model
> Version: $Revision$
> Author: pep@zadka.site.co.il (Moshe Zadka)
> Status: Draft
> Type: Standards Track
> Created: 4-Nov-2000
> Post-History:
> 
> Abstract
> 
>     Today, Python's numerical model is similar to the C numeric model:
>     there are several unrelated numerical types, and when operations
>     between numerical types are requested, coercions happen. While the C
>     rational for the numerical model is that it is very similar to what
>     happens on the hardware level, that rational does not apply to Python.
>     So, while it is acceptable to C programmers that 2/3 == 0, it is very
>     surprising to Python programmers.
> 
> Rationale
> 
>     In usability studies, one of Python hardest to learn features was
>     the fact integer division returns the floor of the division. This
>     makes it hard to program correctly, requiring casts to float() in
>     various parts through the code. Python numerical model stems from
>     C, while an easier numerical model would stem from the mathematical
>     understanding of numbers.
> 
> Other Numerical Models
> 
>     Perl's numerical model is that there is one type of numbers -- floating
>     point numbers. While it is consistent and superficially non-suprising,
>     it tends to have subtle gotchas. One of these is that printing numbers
>     is very tricky, and requires correct rounding. In Perl, there is also
>     a mode where all numbers are integers. This mode also has its share of
>     problems, which arise from the fact that there is not even an approximate
>     way of dividing numbers and getting meaningful answers.
> 
> Suggested Interface For Python Numerical Model
> 
>     While coercion rules will remain for add-on types and classes, the built
>     in type system will have exactly one Python type -- a number. 

While I like the idea of having the numeric model in Python
based on a solid class hierarchy, I don't think that this model
is implementable in Python 2.x without giving away performance.

> There
>     are several things which can be considered "number methods":
> 
>     1. isnatural()
>     2. isintegral()
>     3. isrational()
>     4. isreal()
>     5. iscomplex()

+1.

I would like to see methods on Python numbers too (after having
made some really good experiences with methods on strings ;-).
There's one problem though: how would you call these on
numeric literals ? ... 1.2.isreal() ?!

>     a. isexact()
> 
>     Obviously, a number which answers m as true, also answers m+k as true.
>     If "isexact()" is not true, then any answer might be wrong. (But not
>     horribly wrong: it's close the truth).

Not sure what you mean here: perhaps .isexact() <=> can be
represented in IEEE ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From py-dev@zadka.site.co.il  Sat Nov  4 18:19:13 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:19:13 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com>
 of "Sat, 04 Nov 2000 10:58:49 +0100." <3A03DDD9.8527F7BE@lemburg.com>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com>
Message-ID: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>

> While I like the idea of having the numeric model in Python
> based on a solid class hierarchy, I don't think that this model
> is implementable in Python 2.x without giving away performance.

I think they are, using a similar trick to Fred's automorphing dictionaries.

> +1.
> 
> I would like to see methods on Python numbers too (after having
> made some really good experiences with methods on strings ;-).
> There's one problem though: how would you call these on
> numeric literals ? ... 1.2.isreal() ?!

Ummmm....how would you say you want to add 3 and 4, and multiply the result
by 5? 3+4*5?

No, you use parens:

(3+4)*5
(1.2).isreal()

> >     a. isexact()
> > 
> >     Obviously, a number which answers m as true, also answers m+k as true.
> >     If "isexact()" is not true, then any answer might be wrong. (But not
> >     horribly wrong: it's close the truth).
> 
> Not sure what you mean here: perhaps .isexact() <=> can be
> represented in IEEE ?

No, I meant "not represented exactly". The real meaning for that (one
that we might or might not promise) is that it's a float. It's a place 
where the numeric model takes the easy way out <wink>.

-- 
Moshe Zadka <sig@zadka.site.co.il>


From mal@lemburg.com  Sat Nov  4 10:21:52 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:21:52 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <3A03E340.38F78FA2@lemburg.com>

Guido van Rossum wrote:
> 
> Other issues to work on
> -----------------------
> 
> These aren't PEPs yet, but I think they need to become PEPs soon --
> I'd like to see work on them go into Python 2.1:
> 
> - The buffer interface needs a revamping or simplification; this was
>   discussed here previously.
> 
> - A warning framework.  I've got some ideas already.

This would be a *cool* thing. I have a need for such a framework
in mx.ODBC where ODBC often issues warnings. Currently these
turn out as exceptions which is not really all that useful
because some warnings can safely be ignored.
 
> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

+0... and then only, if there will be a tool to check Python
source code for integer divides.
 
> - Case sensitivity???

Should be left to Py3k. It could then be implemented by using
a special dictionary subclass as instance dictionary.
 
> - Class/type dichotomy???

One thing that would probably be implementable is a way to
maintain "instance" dictionaries for types (which are created
on-demand whenever an assignment is made). This would enable
extending types with new methods and attributes. "Subclassing"
could then be emulated by using new contructors which add the
new or changed methods to each created type instance, e.g.

class myclose:

    def __init__(self, object, basemethod):
        self.object = object
        self.basemethod = basemethod

    def __call__(self):
        print 'Closed file %s' % self.object
        self.basemethod()

def myfile(filename):
    f = open(filename)
    # add/override attributes
    f.newattribute = 1
    # add/override methods
    f.close = myclose(f, f.close)
    return f

Types would have to be made aware of this possibility. Python
could provide some helping APIs to make life easier for the
programmer.

> - Weak references.  This *is* a PEP, but there's no contents yet.  We
>   could also try to implement (just) weak dictionaries.

These already exist... http://www.handshake.de/~dieter/weakdict.html

mx.Proxy also has an implementation which support weak references.

BTW, are these still needed now that we have GC ?
 
> - Internationalization.  Barry knows what he wants here; I bet Martin
>   von Loewis and Marc-Andre Lemburg have ideas too.

We'd need a few more codecs, support for the Unicode compression,
normalization and collation algorithms.

> - Arbitrart attributes on functions?  This would be a generalization
>   of docstrings; with the intent that you don't have to put semantics
>   in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
>   This could possibly lead to implementing private, protected, public
>   attributes too.

Perhaps the above "trick" could help with this ?!
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sat Nov  4 10:31:01 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:31:01 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03E565.9D23559D@lemburg.com>

Moshe Zadka wrote:
> 
> > While I like the idea of having the numeric model in Python
> > based on a solid class hierarchy, I don't think that this model
> > is implementable in Python 2.x without giving away performance.
> 
> I think they are, using a similar trick to Fred's automorphing dictionaries.

You mean numbers "morph" to become floats, complex numbers, etc. 
on demand ? E.g. math.sqrt(-1) would return 1j ?!

> > +1.
> >
> > I would like to see methods on Python numbers too (after having
> > made some really good experiences with methods on strings ;-).
> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!
> 
> Ummmm....how would you say you want to add 3 and 4, and multiply the result
> by 5? 3+4*5?
> 
> No, you use parens:
> 
> (3+4)*5
> (1.2).isreal()

Ah. Of course :-) Cool !
 
> > >     a. isexact()
> > >
> > >     Obviously, a number which answers m as true, also answers m+k as true.
> > >     If "isexact()" is not true, then any answer might be wrong. (But not
> > >     horribly wrong: it's close the truth).
> >
> > Not sure what you mean here: perhaps .isexact() <=> can be
> > represented in IEEE ?
> 
> No, I meant "not represented exactly". The real meaning for that (one
> that we might or might not promise) is that it's a float. It's a place
> where the numeric model takes the easy way out <wink>.

Uhm, that's what I meant. I don't see much use for this though:
the whole meaning of "exact" is void w/r to floats. It should
be replaced by "accurate to n digits".

Note that there is a whole mathematical theory that tries to
deal with this problem: interval calculus. A package to support
this would probably make sense... a nice side-effect of 
interval calculus is that it allows "automatic" optimization
of functions within certain bounds. Numbers are replaced with
intervals and calculus is then done on the interval bounds.

This is just about as close as you can get to floating point
values with computer machinery ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From moshez@zadka.site.co.il  Sat Nov  4 18:52:35 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:52:35 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com>
 of "Sat, 04 Nov 2000 11:31:01 +0100." <3A03E565.9D23559D@lemburg.com>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <3A03E565.9D23559D@lemburg.com>
Message-ID: <E13s8Qf-0002ok-00@darjeeling.zadka.site.co.il>

[Moshe Zadka, about efficient and honest numbers]
> I think they are, using a similar trick to Fred's automorphing dictionaries

[MAL]
> You mean numbers "morph" to become floats, complex numbers, etc. 
> on demand ? E.g. math.sqrt(-1) would return 1j ?!

math.sqrt has been dealt elsewhere in the PEP. It has been suggested
that math.foo will accept and return real numbers, and that
cmath.foo will accept and return complex numbers. If you 
want to always deal with complex numbers, put this in your site.py

import cmath
import sys
sys.modules['math']=cmath

> > No, I meant "not represented exactly". The real meaning for that (one
> > that we might or might not promise) is that it's a float. It's a place
> > where the numeric model takes the easy way out <wink>.
> 
> Uhm, that's what I meant. I don't see much use for this though:
> the whole meaning of "exact" is void w/r to floats. It should
> be replaced by "accurate to n digits".

I'm just promising that floats are inexact. I don't see a need for
"accuracy to n digits" (interval mathematics, etc.) in core Python.
This should be a new module if anyone needs it. Since inexact numbers
will only come about via external modules, you can just use:

import imath # interval math module
import sys
sys.modules['math']=imath.

I'm not repeating myself.

> This is just about as close as you can get to floating point
> values with computer machinery ;-)

I thought floats are the way to get to floating point values with computer
machinery?
-- 
Moshe Zadka <sig@zadka.site.co.il>


From martin@loewis.home.cs.tu-berlin.de  Sat Nov  4 11:12:22 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 12:12:22 +0100
Subject: [Python-Dev] 2.1 tasks (Was: statically nested scopes)
Message-ID: <200011041112.MAA01135@loewis.home.cs.tu-berlin.de>

>  * Work on something CPAN-like.  This may or may not have repercussions for
>    the core; I don't know.

At a minimum, I think we need to include somethingCPANlike.py into the
core (library) to make something CPAN-like useful.

Regards,
Martin



From thomas@xs4all.net  Sat Nov  4 13:54:17 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 14:54:17 +0100
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>; from tim_one@email.msn.com on Sat, Nov 04, 2000 at 02:07:43AM -0500
References: <20001103181734.A6809@ludwig.cnri.reston.va.us> <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>
Message-ID: <20001104145417.I28658@xs4all.nl>

On Sat, Nov 04, 2000 at 02:07:43AM -0500, Tim Peters wrote:
> [Greg Ward]
> > ...
> > Also, if there are warnings we're not going to worry about (eg.
> > incorrect "might be used uninitialized"), lemme know.

> If a compiler is afraid something might be uninitialized, the code is too
> clever for people to be sure it's correct at a glance too.

> I'm not sure what gcc is complaining about in many of the cases; others are
> quite clear (e.g., "confstr" apparently has no prototype in scope by the
> time it's called in posixmodule.c, and that is indeed not good).

There are a few messages that certainly should be looked at. The
'uninitialized usage' messages, for instances, might seriously be problems.
In this case, though, the "'ord' might be used uninitialized" warning isn't
a real problem. 'ord' is indeed only set when 'size == 1' is true, but it's
also only used if 'size == 1', and size isn't changed inbetween those
checks. Whether it should be fixed or not is another issue, but at least
it isn't causing problems.

The 'subscript has type char' message I'm not so sure about -- what is the
problem with those ? I assume it has something to do with char's signedness
being undefined, but I'm not sure. I also thought 'up'casting, such as in
functioncalls (function needs int, you give it char) was done automatically,
as part of the language, and thus shouldn't be a problem.

But the -Wstrict-prototypes seems to detect a lot more 'errors' in system
headerfiles than in Python. For instance, all the "function declaration
isn't a prototype" warnings in signalmodule.c and intrcheck.c seem to be
caused by the SIG_ERR, SIG_DFL and SIG_IGN #defines, which Python can do
nothing about. (those SIG_ #defines are apparently defined as function
declarations without prototypes.) I've seen the same effect on BSDI 4.0.1,
where a few system include files define or declare functions without
prototypes. We can't fix those errors, except by complaining to the OS
vendor. Maybe we should just disable -Wstrict-prototypes (but not -Wall) for
releases, to avoid confusion. (Developers should still use
-Wstrict-prototypes, to catch warnings Python *can* do something about, me
thinks.)

And the 'confstr' message, well, ewww ;) The manpage on Linux seems to
indicate you need to define either _POSIX_C_SOURCE as 2, or define
_XOPEN_SOURCE, for it to work. But I've never understood how those defines
are supposed to interact, so I'm hesitant to go there ;P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Sat Nov  4 14:08:43 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:08:43 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>; from py-dev@zadka.site.co.il on Sat, Nov 04, 2000 at 08:19:13PM +0200
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <20001104150842.J28658@xs4all.nl>

On Sat, Nov 04, 2000 at 08:19:13PM +0200, Moshe Zadka wrote:

> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!

> you use parens:
> (1.2).isreal()

Why ? There is exactly no problem with this example :)

>>> 1.2.isreal()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'float' object has no attribute 'isreal'

If float objects had attributes, it would already work. The real problem
isn't with floats, but with nonfloats:

>>> 1.isreal()
  File "<stdin>", line 1
    1.isreal()
           ^
SyntaxError: invalid syntax

And the limitation is just the parser, currently. Whether we want to allow
that syntax is something that will have to be figured out. And the parser
would have to be fixed (either rewritten into at least an LL(2) parser, or
the metagrammar hacked so that 'NUMBER' doesn't eat the . after a number if
it isn't followed by another number or whitespace.) (At least IMHO, this
isn't an insurmountable problem, or even a medium-sized problem. It was just
never necessary to fix it.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Sat Nov  4 14:46:00 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:46:00 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104150842.J28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:08:43PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl>
Message-ID: <20001104154559.L28658@xs4all.nl>

On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:

> And the parser would have to be fixed (either rewritten into at least an
> LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> after a number if it isn't followed by another number or whitespace.) (At
> least IMHO, this isn't an insurmountable problem, or even a medium-sized
> problem. It was just never necessary to fix it.)

Actually, no, it isn't easily fixable, if at all. The problem is mostly the
scientific notation:

1.e5

Even if it was parsed properly, it is definately going to confuse people.
They wouldn't be able to say, for instance, 

1.e()

;P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From cgw@fnal.gov  Sat Nov  4 15:21:02 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:21:02 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
 <3A03DDD9.8527F7BE@lemburg.com>
 <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <14852.10590.837218.382614@buffalo.fnal.gov>

Moshe Zadka writes:

 > MAL > Not sure what you mean here: perhaps .isexact() <=> can be
 > MAL > represented in IEEE ?
 > 
 > No, I meant "not represented exactly". The real meaning for that (one
 > that we might or might not promise) is that it's a float. It's a place 
 > where the numeric model takes the easy way out <wink>.

Hmm, I like almost everything about your proposal.  The above point
bothers me slightly.  Are you saying (1.0).isexact() == 0?

Also, how about long integers?  Will they, under your new proposal, be
indistinguisable from regular ints?  While this has some appeal to it
it would be problematic for C extension modules.

Finally, although I'm no Schemer, the hierarchy you suggest sounds
very Schemish to me - I know they have a similar hierarchy of numeric
types with some of these same predicates to test for integrality,
rationality, reality, exactness - maybe there is something to be
learned by studying the Scheme model?




From cgw@fnal.gov  Sat Nov  4 15:25:37 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:25:37 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
 <3A03DDD9.8527F7BE@lemburg.com>
 <mal@lemburg.com>
 <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
 <20001104150842.J28658@xs4all.nl>
 <20001104154559.L28658@xs4all.nl>
Message-ID: <14852.10865.96590.740569@buffalo.fnal.gov>

Thomas Wouters writes:

 > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
 > scientific notation:
 > 
 > 1.e5

You could strongly encourage people to spell this 1.0e5


From bckfnn@worldonline.dk  Sat Nov  4 16:21:48 2000
From: bckfnn@worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 16:21:48 GMT
Subject: [Python-Dev] Three argument slices.
Message-ID: <3a04376a.28016074@smtp.worldonline.dk>

Hi,

While updating the difference page in the Jython documentation, I came
across this:

- JPython sequences support three argument slices. i.e. 
  range(3)[::-1] == [2,1,0]. 
  CPython should be fixed.

Is this actually true? Should (and will cpython) change in this respect?

regards,
finn


From py-dev@zadka.site.co.il  Sun Nov  5 00:57:12 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 02:57:12 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from Charles G Waldman <cgw@fnal.gov>
 of "Sat, 04 Nov 2000 09:21:02 CST." <14852.10590.837218.382614@buffalo.fnal.gov>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <14852.10590.837218.382614@buffalo.fnal.gov>
Message-ID: <E13sE7V-0002v1-00@darjeeling.zadka.site.co.il>

> Hmm, I like almost everything about your proposal.  The above point
> bothers me slightly.  Are you saying (1.0).isexact() == 0?

Yes. 1.0 is not an exact number. What's wrong with that?
(Consider stuff like 0.333333333*3: this shouldn't be exact!)

> Also, how about long integers?  Will they, under your new proposal, be
> indistinguisable from regular ints?

Yes.

> While this has some appeal to it
> it would be problematic for C extension modules.

I haven't mentioned anything about implementation, so I haven't dealt
with the C level at all. Currently, a Python-level API is under 
consideration. I believe I can keep current day C API almost unchanged.

> Finally, although I'm no Schemer, the hierarchy you suggest sounds
> very Schemish to me

I shamelessly stole it from Scheme, with only minor changes -- most
of them about hardening some things Scheme left for implementations
to decide.
-- 
Moshe Zadka <sig@zadka.site.co.il>


From mwh21@cam.ac.uk  Sat Nov  4 16:47:31 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 04 Nov 2000 16:47:31 +0000
Subject: [Python-Dev] Three argument slices.
In-Reply-To: bckfnn@worldonline.dk's message of "Sat, 04 Nov 2000 16:21:48 GMT"
References: <3a04376a.28016074@smtp.worldonline.dk>
Message-ID: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>

bckfnn@worldonline.dk (Finn Bock) writes:

> Hi,
> 
> While updating the difference page in the Jython documentation, I
> came across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this
> respect?

Well, there's a patch I wrote on sf to add this to CPython, but it was
too late for 2.0 and it got postponed.

Does J[P]ython allow

l = range(10)
l[::3] = range(4)

?  That's one of the odder bits of the behaviour of my patch.

awaiting-pronouncement-(or-even-consensus)-ly y'rs
m.

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From bckfnn@worldonline.dk  Sat Nov  4 18:41:59 2000
From: bckfnn@worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 18:41:59 GMT
Subject: [Python-Dev] Three argument slices.
In-Reply-To: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
References: <3a04376a.28016074@smtp.worldonline.dk> <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3a045808.36365811@smtp.worldonline.dk>

>> - JPython sequences support three argument slices. i.e. 
>>   range(3)[::-1] == [2,1,0]. 
>>   CPython should be fixed.
>> 
>> Is this actually true? Should (and will cpython) change in this
>> respect?
>
>Well, there's a patch I wrote on sf to add this to CPython, but it was
>too late for 2.0 and it got postponed.
>
>Does J[P]ython allow
>
>l = range(10)
>l[::3] = range(4)
>
>?  That's one of the odder bits of the behaviour of my patch.

No:

Jython 2.0 pre-alpha on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> l = range(10)
>>> l[::3] = range(4)
Traceback (innermost last):
  File "<console>", line 1, in ?
ValueError: step size must be 1 for setting list slice

regards,
finn


From gstein@lyra.org  Sat Nov  4 18:56:08 2000
From: gstein@lyra.org (Greg Stein)
Date: Sat, 4 Nov 2000 10:56:08 -0800
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:46:00PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl>
Message-ID: <20001104105608.A10135@lyra.org>

Oh, this is just simple:

  (1.2).isreal()
  1.2 .isreal()

As Thomas said, fixing the grammar/parser would be rather difficult, so just
expect people to use parens or an extra space if they want to use it on a
constant.

[ of course, it is very silly to make *any* changes to the grammar just to
  allow people to use these on a constant; that is quite a silly "usage"
  that we don't need to pander to; the above "workarounds", if you will, are
  sufficient for the bozos who use it on a constant. ]

Cheers,
-g

On Sat, Nov 04, 2000 at 03:46:00PM +0100, Thomas Wouters wrote:
> On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:
> 
> > And the parser would have to be fixed (either rewritten into at least an
> > LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> > after a number if it isn't followed by another number or whitespace.) (At
> > least IMHO, this isn't an insurmountable problem, or even a medium-sized
> > problem. It was just never necessary to fix it.)
> 
> Actually, no, it isn't easily fixable, if at all. The problem is mostly the
> scientific notation:
> 
> 1.e5
> 
> Even if it was parsed properly, it is definately going to confuse people.
> They wouldn't be able to say, for instance, 
> 
> 1.e()
> 
> ;P
> 
> -- 
> Thomas Wouters <thomas@xs4all.net>
> 
> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/


From thomas@xs4all.net  Sat Nov  4 21:29:28 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:29:28 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <14852.10865.96590.740569@buffalo.fnal.gov>; from cgw@fnal.gov on Sat, Nov 04, 2000 at 09:25:37AM -0600
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <14852.10865.96590.740569@buffalo.fnal.gov>
Message-ID: <20001104222928.M28658@xs4all.nl>

On Sat, Nov 04, 2000 at 09:25:37AM -0600, Charles G Waldman wrote:
> Thomas Wouters writes:

>  > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
>  > scientific notation:

>  > 1.e5

> You could strongly encourage people to spell this 1.0e5

Oh, sure, but that isn't going to solve anything, unless you are proposing
to break the common practice of not requiring zeros before or after decimal
points entirely, and thus breaking gobs of code. The problem is simply that
the syntax is truly ambiguous, and there is no way to tell whether the
statement

  x = 1.e5

is meant to assign 100000 (as a float) to 'x', or assign the 'e5' attribute
of the object '1' to 'x'. Not even full context-based parsing is going to
solve that. This is an edge case, and not likely to happen in real life, but
I don't think it's really worth the trouble, all in all. We'd have to
rewrite the parser into something other than a look-ahead parser to be able
to correctly parse the cases where the syntax isn't really ambiguous, such
as

x = 1.e42e

or some such, and that would still leave unparseable syntax. And all that
just to avoid forcing people to use parentheses around 'normal' integer
literals when directly following them with an attribute dereference. After
all,

 t = 1
 x = t.e5

is perfectly valid, and probably a lot more common. Who needs to run a
method on a literal anyway [don't mention the .join() war!]

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Sat Nov  4 21:32:39 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:32:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104105608.A10135@lyra.org>; from gstein@lyra.org on Sat, Nov 04, 2000 at 10:56:08AM -0800
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <20001104105608.A10135@lyra.org>
Message-ID: <20001104223239.N28658@xs4all.nl>

On Sat, Nov 04, 2000 at 10:56:08AM -0800, Greg Stein wrote:
> Oh, this is just simple:

>   (1.2).isreal()
>   1.2 .isreal()

Very observant, Greg ! I hadn't even realized that, but it makes perfect
sense if you think about it ;) '1.e5' is a single token, to the parser (a
NUMBER), so it can't have whitespace inbetween. But '1 .e5' is naturally
broken up into at least two tokens (three, in this case), and thus is
'correctly' parsed even in todays parser:

>>> 1 .e5
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'int' object has no attribute 'e5'

Y'all just need to quit thinking about floats as the problem... the problem
is ints, not floats ;)

I-shut-up-now-ly y'rs,
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From martin@loewis.home.cs.tu-berlin.de  Sat Nov  4 21:15:13 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 22:15:13 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>

> Yes. 1.0 is not an exact number. What's wrong with that?
> (Consider stuff like 0.333333333*3: this shouldn't be exact!)

It's not inherently wrong. It just points out an omission in the PEP:
it doesn't talk about the meaning of number literals. Since the new
model is going to follow algebraic principles more closely, I had
expected that

  0.333333333 == 333333333 / 1000000000

where, as I understand the proposal, the right-hand side is an exact
number (so 0.333333333*3 would be 999999999 / 1000000000).

One of the more-frequent questions on python-help is confusion about
floating-point numbers, e.g. why is the result of 1.99+4.99 printed
as 6.9800000000000004; users often report that as a bug.

Of course, spelling the number you had in mind as inexact(0.333333333)
is hardly acceptable, either.

Regards,
Martin


From skip@mojam.com (Skip Montanaro)  Sat Nov  4 22:55:44 2000
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Sat, 4 Nov 2000 16:55:44 -0600 (CST)
Subject: [Python-Dev] Accessing RH 7.0 on SF?
Message-ID: <14852.37872.843069.514782@beluga.mojam.com>

Someone posted a bug about the bsddb config stuff related to RH7.0, which I
don't have direct access to.  I've seen others access different Linux
dialects on the SF site.  Can someone explain how I can access RH7.0 there?
All I really need to do at the moment is peruse the /usr/include/db3 stuff.

Thx,

Skip


From thomas@xs4all.net  Sat Nov  4 22:10:32 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 23:10:32 +0100
Subject: [Python-Dev] Accessing RH 7.0 on SF?
In-Reply-To: <14852.37872.843069.514782@beluga.mojam.com>; from skip@mojam.com on Sat, Nov 04, 2000 at 04:55:44PM -0600
References: <14852.37872.843069.514782@beluga.mojam.com>
Message-ID: <20001104231031.P28658@xs4all.nl>

On Sat, Nov 04, 2000 at 04:55:44PM -0600, Skip Montanaro wrote:
> Someone posted a bug about the bsddb config stuff related to RH7.0, which I
> don't have direct access to.  I've seen others access different Linux
> dialects on the SF site.  Can someone explain how I can access RH7.0 there?
> All I really need to do at the moment is peruse the /usr/include/db3 stuff.

I don't think they have it, yet. RH7 is pretty new after all. I can give you
an account on my home machine if you want, though, it's RH7. Only accessible
through ssh currently, but if you want I can turn on telnet. Just send me an
email with login/pw you want ;) Or if you'd rather I peruse the db3 stuff
for you, that's fine too.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From moshez@zadka.site.co.il  Sun Nov  5 08:20:45 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 10:20:45 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
 of "Sat, 04 Nov 2000 22:15:13 +0100." <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
Message-ID: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> It's not inherently wrong. It just points out an omission in the PEP:
> it doesn't talk about the meaning of number literals.

That's right -- but I did mean that floating-point literals will be
inexact.

> Since the new
> model is going to follow algebraic principles more closely, I had
> expected that
> 
>   0.333333333 == 333333333 / 1000000000
> 
> where, as I understand the proposal, the right-hand side is an exact
> number (so 0.333333333*3 would be 999999999 / 1000000000).
> 
> One of the more-frequent questions on python-help is confusion about
> floating-point numbers, e.g. why is the result of 1.99+4.99 printed
> as 6.9800000000000004; users often report that as a bug.

That's one thing my PEP is not meant to help with -- floating point
numbers will remain hard. Hopefully, people will use them less often
when they'll have rational arithmetic.

-- 
Moshe Zadka <sig@zadka.site.co.il>


From martin@loewis.home.cs.tu-berlin.de  Sun Nov  5 08:28:06 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 09:28:06 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> (message from
 Moshe Zadka on Sun, 05 Nov 2000 10:20:45 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
Message-ID: <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>

> > It's not inherently wrong. It just points out an omission in the PEP:
> > it doesn't talk about the meaning of number literals.
> 
> That's right -- but I did mean that floating-point literals will be
> inexact.

Remind you that your model has no notion of floating-point numbers -
then what the heck are floating-point literals? The numbers that you
can write in a base-10 notation are all rational numbers, and the
point doesn't really float in them...

Regards,
Martin


From py-dev@zadka.site.co.il  Sun Nov  5 17:25:33 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 19:25:33 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
 of "Sun, 05 Nov 2000 09:28:06 +0100." <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
Message-ID: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> Remind you that your model has no notion of floating-point numbers -
> then what the heck are floating-point literals? The numbers that you
> can write in a base-10 notation are all rational numbers, and the
> point doesn't really float in them...

Well, of course they are rational numbers. The only question is whether 1.0
should be inexact or exact. While that is not specified in the PEP (which
was meant for Barry to assign me a PEP number primarily...), I think
the principle of least suprise would be to treat 1.0 as inexact.
(IOW, not to promise that (1.0/3.0)*3.0 == 1.0)
-- 
Moshe Zadka <sig@zadka.site.co.il>


From martin@loewis.home.cs.tu-berlin.de  Sun Nov  5 10:24:47 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 11:24:47 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> (message from
 Moshe Zadka on Sun, 05 Nov 2000 19:25:33 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>

> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

To long-term Python users, that would be the least surprise.

To new users, the entire notion of inexact numbers is surprising; more
so that something as simple as 1.0 is inexact.

To computer numerics fanatics, it is surprising that 1.0 is inexact,
since the common representations of floating point numbers are well
capable of representing it exactly.

Regards,
Martin


From py-dev@zadka.site.co.il  Sun Nov  5 19:04:10 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 21:04:10 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
 of "Sun, 05 Nov 2000 11:24:47 +0100." <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
Message-ID: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>

[Moshe Zadka]
> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

[Martin v. Loewis]
> To long-term Python users, that would be the least surprise.

And to long term users of C/Perl/etc., once they map the numerical 
concepts correctly. But I hardly thing we should be arguing about this
at this stage: I'm willing to leave this as an open issue in the PEP,
if this is all you find wrong with it...

And a request to all Python-Devvers: please direct comments directly to
me, and I promise I'll summarize them all in the PEP. As soon as I
get a PEP number, I'll publish an updated version, with all objections and open
issues sumarized.
-- 
Moshe Zadka <sig@zadka.site.co.il>


From martin@loewis.home.cs.tu-berlin.de  Sun Nov  5 11:21:14 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 12:21:14 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> (message from
 Moshe Zadka on Sun, 05 Nov 2000 21:04:10 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>

> And to long term users of C/Perl/etc., once they map the numerical 
> concepts correctly. But I hardly thing we should be arguing about this
> at this stage: I'm willing to leave this as an open issue in the PEP,
> if this is all you find wrong with it...

Sorry I didn't mention it: Overall, I like your proposal very well.
I'm missing the section on implementation strategies, though.

> And a request to all Python-Devvers: please direct comments directly
> to me, and I promise I'll summarize them all in the PEP. As soon as
> I get a PEP number, I'll publish an updated version, with all
> objections and open issues sumarized.

Yes, that is a tricky part of the PEP procedure: not commenting in the
public initially. I think PEP authors can contribute by not posting
the text of their PEP publically.

Regards,
Martin



From gstein@lyra.org  Sun Nov  5 11:40:44 2000
From: gstein@lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 03:40:44 -0800
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sun, Nov 05, 2000 at 12:21:14PM +0100
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <20001105034044.M10135@lyra.org>

On Sun, Nov 05, 2000 at 12:21:14PM +0100, Martin v. Loewis wrote:
>...
> > And a request to all Python-Devvers: please direct comments directly
> > to me, and I promise I'll summarize them all in the PEP. As soon as
> > I get a PEP number,

Euh... is process getting in the way of progress?

Allocate yourself a PEP number and publish the darn thing. If you don't feel
comfortable grabbing a PEP number, then just post it to the list or
something.

One of the worst things that can happen is to feel locked in by a cumbersome
process. Open Source works because people can flow at their own speed,
independent of what is happening with others. If Guido and company are too
busy to update syncmail... no problem! Thomas has time and inclination and
jumps in to fix it. Somebody too busy to revamp the headers for ANSI C? No
worries... we have a lot of volunteers just ready and able to do that.

But you throw in the gates? The locks? The process? It halts.

> > I'll publish an updated version, with all
> > objections and open issues sumarized.
> 
> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

I'm not sure what you're saying here. That a PEP author should develop the
PEP entirely in private? Only when it is fully-cooked, that it should be
published?

Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
a first draft. Revise. Revise. Revise.

Comments on PEPs are generated when people feel their are listened to. If a
PEP author develops a PEP entirely in secret, then the feedback is going to
be diminished because it is hard for people to really know if their comments
and ideas are being captured and considered. When you have that feedback
loop and the positive reinforcement, then you will engender more commentary.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From nhodgson@bigpond.net.au  Sun Nov  5 11:59:03 2000
From: nhodgson@bigpond.net.au (Neil Hodgson)
Date: Sun, 5 Nov 2000 22:59:03 +1100
Subject: [Python-Dev] What to choose to replace Tkinter?
Message-ID: <049d01c0471f$d7899450$8119fea9@neil>

Andrew Kuchling:

> I believe the GNOME (not GTk's, but GNOME's) canvas widget began as a
> fork of the Tk widget that was then substantially enhanced to be a
> general-purpose display engine, with antialiasing, alpha compositing,
> more attention to performance, and other fancy features.  I don't know
> if the corresponding text widget (which is Pango, www.pango.org, I
> think) is equally featureful.

   There is a port of the Tk text widget to GTK+ by Havoc Pennington which
doesn't require Pango. Its aims are a little muddled as a high quality Pango
based text widget is also under long term development.

   Pango isn't just a text widget but a layered set of capabilities allowing
development of internationalised layout and rendering (the equivalent of
Microsoft's Uniscribe). Scintilla for GTK+ will use Pango to implement
Unicode once Pango is released.

   Neil



From Moshe Zadka <moshez@math.huji.ac.il>  Sun Nov  5 12:12:09 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:12:09 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>

On Sun, 5 Nov 2000, Martin v. Loewis wrote:

> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

Perhaps, but that is part of the PEP flow: public post->number
Well, thanks a lot for the feedback.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From Moshe Zadka <moshez@math.huji.ac.il>  Sun Nov  5 12:16:46 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:16:46 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
Message-ID: <Pine.GSO.4.10.10011051412530.28025-100000@sundial>

On Sun, 5 Nov 2000, Greg Stein wrote:

> Euh... is process getting in the way of progress?

Perhaps. Well, I won't let it get in the way more then a few more hours --
I'll allocate myself a PEP. It's  easier to get forgiveness then
permission <wink>

> I'm not sure what you're saying here. That a PEP author should develop the
> PEP entirely in private? Only when it is fully-cooked, that it should be
> published?

Far from it. Only tht the discussion tends to clutter up Python-Dev too
much, so I want to moderate it by way of private mail to me-> checking
to the PEP. I hope everyone here trust me to be honest enough not to
censor competing points.

> Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
> a first draft. Revise. Revise. Revise.

I intend to. 

> Comments on PEPs are generated when people feel their are listened to. If a
> PEP author develops a PEP entirely in secret

All future revisions will be in the Python CVS. Only security through
obscurity can keep me secret there <wink>
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From fdrake@acm.org  Sun Nov  5 16:45:55 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 11:45:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
Message-ID: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Perhaps, but that is part of the PEP flow: public post->number
 > Well, thanks a lot for the feedback.

  Don't hesitate to publish an updated version just because Barry
hasn't assigned a number.  If you don't want to publish the full text
too often, assign a number to yourself in the PEP index and check that
in, then add the actual PEP with the right number.
  (Make sure you run "./pep2html.py -i 0 <num>" to update the index
and your PEP on python.sourceforge.net once you've made the checkins.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From thomas@xs4all.net  Sun Nov  5 19:18:51 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 20:18:51 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <200011051655.IAA13536@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Sun, Nov 05, 2000 at 08:55:27AM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
Message-ID: <20001105201851.C27208@xs4all.nl>

On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:

> Added Files:
> 	pep-0228.txt 
> Log Message:
> Added first revision of numerical model pep.

> ***** Error reading new file: (2, 'No such file or directory')
> ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461

Thank you, Moshe ;) Now to figure out why it does work for some people, or
in some cases ;P I'm guessing it's a race condition of some sort, but damned
if I know what triggers it.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From gstein@lyra.org  Sun Nov  5 21:54:39 2000
From: gstein@lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 13:54:39 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105201851.C27208@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 08:18:51PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl>
Message-ID: <20001105135439.V10135@lyra.org>

On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:
> On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:
> 
> > Added Files:
> > 	pep-0228.txt 
> > Log Message:
> > Added first revision of numerical model pep.
> 
> > ***** Error reading new file: (2, 'No such file or directory')
> > ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461
> 
> Thank you, Moshe ;) Now to figure out why it does work for some people, or
> in some cases ;P I'm guessing it's a race condition of some sort, but damned
> if I know what triggers it.

It certainly could be a race condition. Just look at blast_mail(). It forks
off the operation, and it might run before the file arrives in the
repository. (??)

Or is it possible that the script looks for pep-0228.txt rather than getting
a copy from pep-0228.txt,v ?? My script does "cvs -Qn update -p -r1.1 FILE"
for new files, and pipes that to the output.

Note that the log_accum.pl script that I use for my CVS repository (which
came from Apache, which came from BSD) does not fork. It seems to work quite
fine with added files.

[ and the point in blast_mail() about holding the CVS lock too long is
  rather silly given the horsepower of the SourceForge boxes ]

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From thomas@xs4all.net  Sun Nov  5 22:12:35 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:12:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105135439.V10135@lyra.org>; from gstein@lyra.org on Sun, Nov 05, 2000 at 01:54:39PM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org>
Message-ID: <20001105231235.X12776@xs4all.nl>

On Sun, Nov 05, 2000 at 01:54:39PM -0800, Greg Stein wrote:
> On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:

> It certainly could be a race condition. Just look at blast_mail(). It forks
> off the operation, and it might run before the file arrives in the
> repository. (??)

No... that's not it.

> Or is it possible that the script looks for pep-0228.txt rather than getting
> a copy from pep-0228.txt,v ??

Almost correct :) The problem is that the loginfo process is run in the
server-specific /tmp dir, and the filename is not prefixed by the path to
the CVSROOT or some such. I guess it's pure coincidence that the file is
still there when the syncmail script arrives at the 'open()' call.

> My script does "cvs -Qn update -p -r1.1 FILE" for new files, and pipes
> that to the output.

Yes... I just wrote a small patch to syncmail which does exactly that
(actually, it uses 'newrev' rather than a hardcoded '1.1', and it uses -f
and -n but not -Q -- if we add -Q, we should add it to the diff commands as
well, and it might confuse developers that are used to reading the debug
info ;)

The main reason I'm delaying the checkin is to test it on the only CVS
repository I can play with, which is over a slow link to an american
highschool. Now if only you had mailed an hour earlier, Greg, I wouldn't
have had to go through that trouble ;)

> [ and the point in blast_mail() about holding the CVS lock too long is
>   rather silly given the horsepower of the SourceForge boxes ]

Well, syncmail was written to manage the Python CVS tree on a slow Sun (I
believe) and did an rsync-over-ssh to another machine as well. That can
definately take long ;) If we just remove the fork, the rest of syncmail
might just work, even with new files. In the mean time, I'll check in my
change. It might be the best thing to do anyway, since it shouldn't
interfere unless the file isn't there.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From jeremy@alum.mit.edu  Sun Nov  5 21:08:55 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:08:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
 <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
 <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
 <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
 <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
 <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
 <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <20001105034044.M10135@lyra.org>
Message-ID: <14853.52327.355764.528782@bitdiddle.concentric.net>

[Comments from Greg and Moshe on the PEP process.]

I don't see the PEP process creating any impediments here.  There is
almost no process -- Barry assigns numbers to PEPs and Guido rules on
them.  We've got some rules about what must be in the PEP before it is
approved, but almost none about how the PEP is created.

There is nothing about the PEP process that prevents a healthy
discussion of issues, in private mail or on a mailing list (python-dev
or otherwise).  We had lots of comments on the statically nested
scopes PEP before Barry assigned it a number.  An entire PEP could be
created and discussed before it gets a number.

Someone may want to work on a PEP in privacy and wait to share it
until the entire document is complete; that's fine too, provided that
revisions are made based on feedback.

One goal we had when setting up the PEP process was to limit the
amount of repeated discussion on an issue.  It's not too uncommon for
email discussions to go in circles or to endlessly rehash the same few
issues.  We hoped that PEP authors would incorporate a discussion of
such issues in the PEP and reduce the amount of wasted bandwidth on
repetitive discussion.

Let's not waste time discussing how to create PEPs and instead
actually create them.  The clock is ticking for new features in 2.1;
the tentative deadline for new PEPs is mid-December.

Jeremy


From jeremy@alum.mit.edu  Sun Nov  5 21:14:36 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:14:36 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
 <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
Message-ID: <14853.52668.117844.28459@bitdiddle.concentric.net>

>>>>> "FLD" == Fred L Drake, <fdrake@acm.org> writes:

  FLD> Moshe Zadka writes:
  >> Perhaps, but that is part of the PEP flow: public post->number
  >> Well, thanks a lot for the feedback.

  FLD>   Don't hesitate to publish an updated version just because
  FLD>   Barry
  FLD> hasn't assigned a number.  If you don't want to publish the
  FLD> full text too often, assign a number to yourself in the PEP
  FLD> index and check that in, then add the actual PEP with the right
  FLD> number.

I thought we discussed this earlier and agreed that a little bit of
control over the process was healthy.  I would prefer to see all PEP
creation go through Barry.  We can circulate drafts in email before
that.

Jeremy


From thomas@xs4all.net  Sun Nov  5 22:29:21 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:29:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105231235.X12776@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 11:12:35PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org> <20001105231235.X12776@xs4all.nl>
Message-ID: <20001105232921.E27208@xs4all.nl>

On Sun, Nov 05, 2000 at 11:12:35PM +0100, Thomas Wouters wrote:

> In the mean time, I'll check in my change. It might be the best thing to
> do anyway, since it shouldn't interfere unless the file isn't there.

Since changes to files in CVSROOT go to python-checkins-admin rather than
python-checkins, here's the diff I just checked in:

Index: syncmail
===================================================================
RCS file: /cvsroot/python/CVSROOT/syncmail,v
retrieving revision 3.14
retrieving revision 3.15
diff -c -c -r3.14 -r3.15
*** syncmail    2000/11/02 21:44:32     3.14
--- syncmail    2000/11/05 22:24:29     3.15
***************
*** 85,91 ****
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             fp = open(file)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')
--- 85,95 ----
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             if os.path.exists(file):
!                 fp = open(file)
!             else:
!                 update_cmd = 'cvs -fn update -r %s -p %s' % (newrev, file)
!                 fp = os.popen(update_cmd)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')

See the manpage for 'cvs' for an explanation of the options ;) This should
fix 99.95% or so of the problem (there is still a tiny window for the file
being removed inbetween the os.path.exists and the actual open) and is
probably best even if we do remove the fork() from syncmail.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From greg@cosc.canterbury.ac.nz  Mon Nov  6 00:56:16 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 13:56:16 +1300 (NZDT)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
Message-ID: <200011060056.NAA29965@s454.cosc.canterbury.ac.nz>

"Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>:

> To computer numerics fanatics, it is surprising that 1.0 is inexact,
> since the common representations of floating point numbers are well
> capable of representing it exactly.

I suppose in principle one could meticulously keep track
of which floating point numbers in a calculation were
exact and which weren't. But you'd lose the property that
any arithmetic operation on exact operands produces an
exact result.

Also, it would be very tedious and inefficient to have
to keep track of exactness so exactly!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Nov  6 01:03:06 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:03:06 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
Message-ID: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz>

Guido:

> I'd be happy to make an explicit list of
> those builtins that should not be messed with

There's a precedent for this in Scheme, which has a notion
of "integrable procedures".

As for the rest, with static scoping it will be possible to
make access to builtins just as efficient as locals, while
still allowing them to be rebound, so there's no reason why
__builtin__.__dict__.open = foo can't continue to work,
if so desired.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Nov  6 01:16:27 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:16:27 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A02B489.89EF108C@lemburg.com>
Message-ID: <200011060116.OAA29972@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal@lemburg.com>:

> Nested scopes will introduce cycles in all frame objects.

It doesn't have to be that way. A static link is only
needed if a function actually uses any variables from
an outer scope. In the majority of cases, it won't.

And it's possible to do even better than that. You can
separate out variables referred to in an inner scope
and store them separately from the rests of the frame,
so you only keep what's really needed alive.

> This means that with GC turned off, frame objects will live
> forever

Don't allow GC to be turned off, then! (Presumably this
feature would only be considered once GC has become a
permanent feature of Python.)

> BTW, Python's GC only works for a few builtin types (frames
> are not among the supported types)

But they could become so if necessary, surely?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Nov  6 01:19:02 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:19:02 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Sat, 04 Nov 2000 20:00:49 +0200."
 <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
Message-ID: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>

So I go offline for a couple of days to entertain guests and have my
body kicked around in a dance class, and I have 25 messages discussing
Python's numeric model waiting for me...

I was hoping that Tim would chime in, but he's apparently taken the
weekend off -- very much out of character. :-)

I like the idea of a PEP to rework the numeric model.  I think that
Moshe, being a mathematician by training, will make a good editor.  I
think that this has a snowball's chance in hell to make it into Python
2.1 however -- there are many deep issues, and it's a bad idea to
experiment too much in the released version of the language.

Some of the issues are more usability issues than of a mathematical
nature.  For example, Tim has conjectured that using binary floating
point will always be a problem for the "unwashed masses" -- the only
thing they might understand is decimal floating point, not rational
numbers.  There are at least two issues here:

(1) As long as the internal representation is not the same as what is
    commonly printed, there will be surprises -- with rationals just
    as much as with floating point.  There are issues with decimal
    floating point too, but they are only the issues having to do with
    loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
    1.0) and not with loss of precision in the printing, where most of
    the "bug reports" we get seem to concentrate.

(2) Rational numbers have the unpleasant property of growing
    unboundedly during many innocent calculations, thereby using up
    exorbitant amounts of memory and slowing down the calculation --
    often mysteriously, because what is displayed is truncated.

Another issue that I might bring up is that there are no inexact
numbers (each floating point number is perfectly exact and rational)
-- there are only inexact operations.  I'm not sure what to do with
this though.

If we take its meaning literally, the isreal() function should only
return true for numbers for which isrational() is also true:
mathematically speaking, real numbers that aren't also rational don't
have an easy finite representation, since they are numbers like
sqrt(2) or pi.

I'll leave it to Tim to explain why inexact results may not be close
to the truth.

Tim may also break a lance for IEEE 754.

Finally, the PEP doesn't talk about how the numeric model can be
extended, and how other numeric types can be blended in.  E.g. I've
heard of wild floating point representations that make multiplication
and division really cheap but addition a pain, rather than the other
way around; some people may want to have long ints implemented using
the GNU mp library, and so on.  Such custom types should be supported
as well as native types -- like they are now.

--Guido van Rossum (home page: http://www.python.org/~guido/)

PS.  The "1.isreal()" problem is a non-problem.  This question is only
interesting to ask about variables.


From guido@python.org  Mon Nov  6 01:25:08 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:25:08 -0500
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
In-Reply-To: Your message of "Sat, 04 Nov 2000 10:50:10 +0100."
 <3A03DBD2.7E024978@lemburg.com>
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
 <3A03DBD2.7E024978@lemburg.com>
Message-ID: <200011060125.UAA03986@cj20424-a.reston1.va.home.com>

Marc-Andre:
> I can take over the coercion PEP: I've been working
> on this before (see the proposal on my Python Pages).

Thanks, excellent (although I haven't seen your proposal yet).

> I would also like to know whether the PEP-0224 will be considered
> for 2.1 if I update the patch to make it a little more robust
> w/r to the problems mentioned in that PEP -- I'd really like
> to see this in Python soon, since it makes documenting Python
> programs so much easier.

I "kinda" like the idea of having attribute docstrings (meaning it's
not of great importance to me) but there are two things I don't like
in your current proposal:

1. The syntax you propose is too ambiguous: as you say, stand-alone
   string literal are used for other purposes and could suddenly
   become attribute docstrings.

2. I don't like the access method either (__doc_<attrname>__).

> Note that I won't get around to do much work on these before
> January... way too busy at the moment :-/

That's a problem -- we really want to have the PEPs ready for review
by mid December.  This will also be a problem for the coercion PEP --
if you think you won't be able to work on it before then, I'd prefer
to find another (co-)author.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 02:05:20 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:05:20 -0500
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:03:06 +1300."
 <200011060103.OAA29969@s454.cosc.canterbury.ac.nz>
References: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz>
Message-ID: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>

> Guido:
> > I'd be happy to make an explicit list of
> > those builtins that should not be messed with

[Greg Ewing]
> There's a precedent for this in Scheme, which has a notion
> of "integrable procedures".

Good!

> As for the rest, with static scoping it will be possible to
> make access to builtins just as efficient as locals, while
> still allowing them to be rebound, so there's no reason why
> __builtin__.__dict__.open = foo can't continue to work,
> if so desired.

I'm not sure what you mean.  With integrable procedures (whatever they
may be :-) I believe this is possible.  Without them, the lookup in
globals() can be skipped for builtins, but a local is accessed with
*zero* dict lookups -- how would you do this while still supporting
__builtin__.__dict__.open = foo?  have "hookable" dictionaries?
(Those would solve a bunch of problems, but are not under
consideration at the moment.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gmcm@hypernet.com  Mon Nov  6 02:21:47 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Sun, 5 Nov 2000 21:21:47 -0500
Subject: [Python-Dev] Stackless pages
Message-ID: <3A05CF6B.32107.2CAF777@localhost>

I have put up 6 pages of information about stackless at

http://www.mcmillan-inc.com/stackless.html

The first page attempts to give a conceptual overview of 
stackless. Notice I said "conceptual" - I make no attempt to 
be technically accurate!

Next follow 4 pages of tutorial. Mainly this is a discussion of 
implementing generators and coroutines through the 
continuation module. It includes rewrites of 2 samples that 
Tim used to demonstrate his coroutines-implemented-via-
threads madness.

Finally, the last page is about SelectDispatcher, which is kind 
of Medusa using coroutines. Included as a demonstration is a 
full FTPServer that will run on Windows. This is not just demo 
quality code - it's at the core of a couple commercial apps I'm 
doing for clients, at least one of which will make something of 
a splash in its (large, prosperous) industry. SelectDispatcher 
and friends are released under the McMillan Enterprises 4 line 
license (do what thou wilt; maintain the copyright notice; no 
warranty).

While these are not the PEPs I owe on stackless, they are 
part of the background material for those PEPs, particularly in 
demonstrating why some of us are so interested in seeing 
these facilities within core Python.

I apologize in advance to Christian for any misunderstandings 
or misinformation these pages may contain.

Enjoy!

- Gordon


From guido@python.org  Mon Nov  6 02:34:40 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:34:40 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
 <3A03E340.38F78FA2@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
Message-ID: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>

[Guido]
> > - Integer division.  If we want to play by Paul Prescod's Language
> >   Evolution rules (PEP 5), we better get started on the first stage.
> >   E.g. we could introduce a // operator in 2.1 for integer division,
> >   and issue a warning when / is used for integers.  Then a year later
> >   (i.e., March 2002!) we could change / so that it returns a floating
> >   point number.

[MAL]
> +0... and then only, if there will be a tool to check Python
> source code for integer divides.

Hm.  I don't believe it's possible to write a tool to check for
integer divides by inspection of the source code only -- you have to
actually execute the code (with specific input, etc.).

However, with the right warnings framework in place (I'll post some
ideas about this under a separate subject), the Python interpreter
itself can be the perfect tool to do the checking.  Given that it's
pretty uncontroversial that 1/2 in Py3K should equal 0.5, I'd rather
get this started sooner than later.

Let me state some requirements:

- We don't want to break code in 2.1 that works in 2.0.

- It's okay to issue warnings though (my warnings proposal will limit
  the warnings to once per source line).

- In Py3K, 1/2 will yield 0.5 and users must use a different way to
  spell floor(x/y).

- Starting in 2.1, we want to issue warnings that encourage users to
  make their code Py3K-ready.

- We want (almost) all users to have converted their code to using
  1//2 instead of 1/2 by the time 2.n (the last 2.x version before
  Py3K is released) comes out, because unchanged code will silently
  change its meaning at the Py3K transition.

- Code that is Py3K-ready (in this respect) should trigger no warnings
  in Python 2.1.

Note: it is open for debate whether the result of x/y for integer (or
long integer) arguments should yield an integer (or long integer) in
those cases where the result *is* representable as such (e.g. 4/2).
It is possible that the numeric tower will render this problem moot --
but that depends on what happens to Moshe's PEP 228, and that's a much
longer story.  However, I think we can decide on the transition path
from / to // independent from the outcome of that discussion, since in
all cases it is clear that 1/2 will change in meaning.

Here's a concrete proposal (could be PEPped pretty easily):

- In Py3K, there will be two division operators:

  - x/y will always yield the mathematically expected result (possibly
    inexact, depending on how the numeric model is changed).

  - x//y will always yield the floor of the mathematical result, even
    for float arguments.  For complex arguments, this should raise an
    exception, just as int(1j) does today.

- Starting with Python 2.1, x//y will do the right thing (1//2 == 0).

- In 2.1, using x/y for ints and longs will issue a warning.

- In Py3K, there *might* be a special "backward incompatibility
  warning mode" that issues warnings when x/y is used for ints; 1/2
  will still yield 0.5 in this mode, but a warning will be issued.

This is a minimal proposal.  If Python were to grow a syntax for
pragmas, it would be nice to have a pragma saying "I want int/int to
do float division" in code that is Py3K ready; otherwise, this always
has to be spelled as float(x)/y to ensure proper working in 2.x as
well as in Py3K.

David Scherer proposed to spell this pragma as a "magical import"
(http://www.python.org/pipermail/idle-dev/2000-April/000138.html). This
is an OK idea (+0) because it can be assumed to fail in pre-2.1
installations and doesn't require new syntax.  I don't give it a +1
because it's a hack -- "import" doesn't quite convey the intention.
(Perl's "use" is better for this purpose!)

Tim didn't seem to like this idea much
(http://www.python.org/pipermail/python-dev/2000-April/010029.html).
His dislike seems based on the assumption that such annotations would
mention specific language (or interpreter) version numbers, which
could be interpreted as resisting progress (one moment 1.7 is a
forward looking version, but the next moment it is backward looking).
However if we use directives (I don't want to call them pragmas
because pragmas are supposed to be ignorable) to select specify
specific features, especially features for which there are only two
versions (the old way and the new way) then it seems okay to use such
a mechanism -- if we can agree on a syntax for directives.

Hm, reading Tim's post again it seems he's mostly objecting against
defaulting to an old version.  I have to agree with him there.
However what I'm proposing here is defaulting to the current version,
and allowing a way to select a "future version" as an alternative.

If we don't adopt directives, all we need to do (in Python 2.1) is add
a new opcode for //, keeping the opcode for / unchanged.  If we do
adopt directives, we'll need to introduce two new opcodes: one for the
new (always float) /, one for the new //, still keeping the old /
opcode with the 2.0 meaning.  The latter is what David Scherer
proposes (and what he needs for his students).  Note that it would be
okay to introduce directives in a later 2.x version -- this won't
break any previous code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 02:44:57 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:44:57 -0500
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
 <3A03E340.38F78FA2@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
Message-ID: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>

[me]
> > - Class/type dichotomy???

[MAL]
> One thing that would probably be implementable is a way to
> maintain "instance" dictionaries for types (which are created
> on-demand whenever an assignment is made).
> 
> This would enable
> extending types with new methods and attributes. "Subclassing"
> could then be emulated by using new contructors which add the
> new or changed methods to each created type instance, e.g.
> 
> class myclose:
> 
>     def __init__(self, object, basemethod):
>         self.object = object
>         self.basemethod = basemethod
> 
>     def __call__(self):
>         print 'Closed file %s' % self.object
>         self.basemethod()
> 
> def myfile(filename):
>     f = open(filename)
>     # add/override attributes
>     f.newattribute = 1
>     # add/override methods
>     f.close = myclose(f, f.close)
>     return f
> 
> Types would have to be made aware of this possibility. Python
> could provide some helping APIs to make life easier for the
> programmer.

But this would require an extra pointer field for *all* built-in
types.  That would seriously impact the space requirements for ints
and floats!

As long as we're proposing hacks like this that don't allow smooth
subclassing yet but let you get at least some of the desired effects,
I'd rather propose to introduce some kind of metaclass that will allow
you to use a class statement to define this.  Thinking aloud:

import types
filemetaclass = metaclass(types.FileType)

class myfile(filemetaclass):

      def __init__(self, filename):
	  filemetaclass.__init__(filename)
	  self.newattribute = 1

      def close(self):
          myclose(self)
	  filemetaclass.close(self)

I'm not quite sure what kind of object "filemetaclass" here should be
or what exactly "metaclass()" should do, but it could create a new
type that has the lay-out of an existing file object, with an instance
dictionary (for newattribute) tacked on the end.  Later maybe (I'm
going to brainstorm with Jim Fulton about types and classes).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 02:48:47 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:48:47 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
 <3A03E340.38F78FA2@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
Message-ID: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>

[me]
> > - Weak references.  This *is* a PEP, but there's no contents yet.  We
> >   could also try to implement (just) weak dictionaries.

[MAL]
> These already exist... http://www.handshake.de/~dieter/weakdict.html
> 
> mx.Proxy also has an implementation which support weak references.

Thanks.  For Fred to read...

> BTW, are these still needed now that we have GC ?

Yes, definitely.  Weak dicts are sometimes needed for situations where
a regular dict would keep objects alive forever.  E.g. we were made
aware of a "leak" in JPython that could only be fixed with weak dicts:
the Swing wrapper code has a global dict mapping widgets to callback
functions, and this keeps all widgets alive forever.  The Java GC
doesn't destroy the widgets, because they are still referenced from
the dict.  A weak dict solves this problem nicely (if it weren't that
JDK 1.1 doesn't support weak dicts).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Nov  6 02:49:25 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 21:49:25 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.52668.117844.28459@bitdiddle.concentric.net>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
 <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
 <14853.52668.117844.28459@bitdiddle.concentric.net>
Message-ID: <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I thought we discussed this earlier and agreed that a little bit of
 > control over the process was healthy.  I would prefer to see all PEP
 > creation go through Barry.  We can circulate drafts in email before

  I think I hadn't actually noticed some of that email, or perhaps
there was a conversation I've forgotten.  Fine.  I still don't see a
problem for people creating PEPs; there's always email and the files
can be pubished at alternate locations before a number has been
assigned.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fdrake@acm.org  Mon Nov  6 03:06:29 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 22:06:29 -0500 (EST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
 <200011060248.VAA04306@cj20424-a.reston1.va.home.com>
Message-ID: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > Yes, definitely.  Weak dicts are sometimes needed for situations where
 > a regular dict would keep objects alive forever.  E.g. we were made
 > aware of a "leak" in JPython that could only be fixed with weak dicts:
 > the Swing wrapper code has a global dict mapping widgets to callback

  That's a perfect example.  I've started working on some text
describing the motivation; hopefully I'll have that fleshed out and
checked in later this week.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From greg@cosc.canterbury.ac.nz  Mon Nov  6 03:19:28 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:19:28 +1300 (NZDT)
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <200011060319.QAA00004@s454.cosc.canterbury.ac.nz>

Guido:

> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types

> But this would require an extra pointer field for *all* built-in
> types.

Ruby has an interesting solution to this. It keeps such
"extra" instance variables in a global data structure.

The Python version of this would be to have a special global
dict which maps instances of built-in types to dicts holding
their extra instance variables.

The keys in this dict would have to be weak references,
so that they wouldn't keep the objects alive. A flag would
be set in the object header so that, when the object was
deleted, the corresponding entry in the global dict
could be cleaned up.

The overhead would then be one bit in each object, and
one extra test when deleting an object.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Nov  6 03:20:37 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:20:37 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <200011060320.QAA00007@s454.cosc.canterbury.ac.nz>

Guido:

> Here's a concrete proposal (could be PEPped pretty easily):

Looks good to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Nov  6 03:35:26 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:35:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
 <3A03E340.38F78FA2@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
Message-ID: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>

Before I fall asleep let me write up my ideas about the warning
framework.


Requirements:

- A C-level API that lets C code issue a warning with a single call
  taking one or two arguments, e.g. Py_Warning(level, message).  (The
  'level' argument is an example only; I'm not sure what if any we
  need.)

- After the first time a specific warning is issued for a given source
  code location, the overhead of calling Py_Warning() should be
  minimal.

- An equivalent Python level API, e.g. sys.warning(level, message).

- Flexible control over which warnings are printed or not; there
  should be a way to set this up from within the Python program but
  also from the command line or possible using an environment
  variable.

- Control over the disposition of warnings; by default they should be
  printed to sys.stderr but an alternative disposition should be
  supported (the mechanism could be either a different file or a
  different callback function).

- By default, a warning is printed once (the first time it is issued)
  for each source line where it is issued.

- For a specific warning at a specific source code location, it should
  be possible to specify the following alternatives:

  - Turn it into an exception

  - Don't print it at all

  - Print it each time it is issued

- It should also be possible to specify these alternatives:

  - For all warnings

  - For all warnings in a specific module or file

  - For all warnings at a specific source code location

  - For a specific warning everywhere in a specific module or file

  - For a specific warning everywhere in the program

  - For all warnings at/above/below (?) a specific level, if we use
    warning levels


Possible implementation:

- Each module can has a dictionary __warnings__ in its global
  __dict__, which records the state of warnings.  It is created as an
  emprt dict if it doesn't exist when it is needed.  The keys are
  (message, linenumber) tuples (the module or file is implicit through
  the use of the module's __dict__).  The value is None if no more
  action is needed for this particular warning and location.  Some
  other values may indicate the options "always print warning" (1?)
  and "raise an exception" (-1?).

- There's a list of "filters" in the sys module
  (e.g. sys.warningfilters) that is checked whenever a warning doesn't
  have a hit in the __warnings__ dict.  Entries in the filter list are
  (file, line, message, action) tuples.  (If we decide to implement
  warning levels, these must also be represented here somewhere.)

  - The file must be None or a shell matching pattern, e.g. "*foo";
    the ".py" suffix is optional; a partial pathname may be given too.
    So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
    "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
    the filter applies regardless of the file.

  - The line must be None or an integer.  If the file is None or "*"
    (indicating all files) the line must be None and is ignored.

  - The message must be a None or a string.  If it is None, the filter
    applies to all messages.  The message string may end in "*" to
    match all messages with the given text (up to the "*").

  - The action must be one of the following strings:

    - "ignore" -- the warning is never printed

    - "always" -- the warning is always printed

    - "once" -- the warning is printed for the first occurrence
      matching the filter

    - "module" -- the warning is printed for the first occurrence in
      each module matching the filter

    - "location" -- the warning is printed for the first occurrence at
      each source code location (module + line) matching the filter

    - "exception" -- the warning is turned into an exception whenever
      it matches the filter

    Note: implementation of "once" and "module" require additional
    state per filter entry; I'm not sure if that's worth the effort.

  - When the warning system decides to print a warning, it is given to
    sys.displaywarning(file, line, message), which by default does
    something like print >>sys.stderr, file, ":", line, ":", message

  - There should be a function sys.addwarningfilter(file, line,
    message, action) that appends items to sys.warningfilters after
    some sanity checking.

  - There should be command line options to specify the most common
    filtering actions, which I expect to include at least:

    - suppress all warnings

    - suppress a particular warning message everywhere

    - suppress all warnings in a particular module

    - turn all warnings into exceptions

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Mon Nov  6 03:34:26 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:34:26 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>
Message-ID: <200011060334.QAA00011@s454.cosc.canterbury.ac.nz>

Guido:

> the lookup in
> globals() can be skipped for builtins, but a local is accessed with
> *zero* dict lookups -- how would you do this while still supporting
> __builtin__.__dict__.open = foo?  have "hookable" dictionaries?

With fully static scoping, I envisage that all three kinds
of scope (local, module and builtin) would be implemented
in essentially the same way, i.e. as arrays indexed by
integers.

That being the case, all you need to do is arrange for the
__builtin__ module and the global scope to be one and the
same thing, and __builtin__.open = foo will work just fine
(assuming open() isn't one of the special inlinable functions).

Getting __builtin__.__dict__['open'] = foo to work as well
may require some kind of special dictionary-like object.
But then you're going to need that anyway if you want to
continue to support accessing module namespaces as if they
are dictionaries.

Whether it's worth continuing to support that in Py3k is
something that can be debated separately.

> integrable procedures (whatever they may be :-)

In the Revised^n Report, some builtin procedures are
declared to be "integrable", meaning that the compiler is
allowed to assume that they have their usual definitions
and optimise accordingly.

(This is quite important in Scheme, even more so than
in Python, when you consider that almost every operation in
Scheme, including '+', is a procedure call!)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Mon Nov  6 03:40:33 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:40:33 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
 <3A03E340.38F78FA2@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com>
Message-ID: <200011060340.WAA04479@cj20424-a.reston1.va.home.com>

[me]
> > - Internationalization.  Barry knows what he wants here; I bet Martin
> >   von Loewis and Marc-Andre Lemburg have ideas too.

[MAL]
> We'd need a few more codecs, support for the Unicode compression,
> normalization and collation algorithms.

Hm...  There's also the problem that there's no easy way to do Unicode
I/O.  I'd like to have a way to turn a particular file into a Unicode
output device (where the actual encoding might be UTF-8 or UTF-16 or a
local encoding), which should mean that writing Unicode objects to the
file should "do the right thing" (in particular should not try to
coerce it to an 8-bit string using the default encoding first, like
print and str() currently do) and that writing 8-bit string objects to
it should first convert them to Unicode using the default encoding
(meaning that at least ASCII strings can be written to a Unicode file
without having to specify a conversion).  I support that reading from
a "Unicode file" should always return a Unicode string object (even if
the actual characters read all happen to fall in the ASCII range).

This requires some serious changes to the current I/O mechanisms; in
particular str() needs to be fixed, or perhaps a ustr() needs to be
added that it used in certain cases.  Tricky, tricky!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 03:48:22 2000
From: guido@python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:48:22 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: Your message of "Sun, 05 Nov 2000 21:21:47 EST."
 <3A05CF6B.32107.2CAF777@localhost>
References: <3A05CF6B.32107.2CAF777@localhost>
Message-ID: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>

> I have put up 6 pages of information about stackless at
> 
> http://www.mcmillan-inc.com/stackless.html

Gordon, thanks for doing this.  I still have a review of Stackless on
my TODO list.  It takes a serious chunk of my time to do it justice,
and this continues to be a problem, but the existience of your
overview certainly helps.  I still think that the current Stackless
implementation is too complex, and that continuations aren't worth the
insanity they seem to require (or cause :-), but that microthreads and
coroutines *are* worth having and that something not completely unlike
Stackless will be one day the way to get there...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Mon Nov  6 03:55:17 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 22:55:17 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
References: <3A05CF6B.32107.2CAF777@localhost>
 <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <14854.11173.601039.883893@bitdiddle.concentric.net>

[Changed discussion list from general python-list to specific
stackless.] 

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  >> I have put up 6 pages of information about stackless at
  >> http://www.mcmillan-inc.com/stackless.html

  GvR> Gordon, thanks for doing this.  I still have a review of
  GvR> Stackless on my TODO list.  It takes a serious chunk of my time
  GvR> to do it justice, and this continues to be a problem, but the
  GvR> existience of your overview certainly helps.  I still think
  GvR> that the current Stackless implementation is too complex, and
  GvR> that continuations aren't worth the insanity they seem to
  GvR> require (or cause :-), but that microthreads and coroutines
  GvR> *are* worth having and that something not completely unlike
  GvR> Stackless will be one day the way to get there...

I tend to agree with you, Guido.  I think we would do well to
purposefully omit continuations from the Python language.  There seems
to be little need for a facility to implement arbitrary control
structures in Python.  If Python support coroutines and microthreads,
I am not sure what else would be needed.

It would be very helpful if the PEPs on Stackless could address this
issue.  One way to address it is to ask these questions: What new
control structures do users want in Python?  How best can they be
implemented?  Are continuations necessary to implement them or are
there other options?

The sort of implementation complexity that I worry about with
Stackless is, e.g. clean interaction with the C stack.  If a Python C
API call is made that pushes a C stack frame, e.g. PyObject_Compare,
then a continuation stored before that call can no longer be invokved.
The problem is that continuations break the notion a C API call will
always return an answer; they create a situation in which the C call
that is made should never return, because control is transferred to
the continuation.  I assume Stackless raises an error in this case,
but this seems pretty messy: How do we right a language spec that
explains when an error will occur without appealing to the
language implementation?

Jeremy



From petrilli@amber.org  Mon Nov  6 04:06:35 2000
From: petrilli@amber.org (Christopher Petrilli)
Date: Sun, 5 Nov 2000 23:06:35 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Sun, Nov 05, 2000 at 10:06:29PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060248.VAA04306@cj20424-a.reston1.va.home.com> <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>
Message-ID: <20001105230635.A18694@trump.amber.org>

Fred L. Drake, Jr. [fdrake@acm.org] wrote:
> 
> Guido van Rossum writes:
>  > Yes, definitely.  Weak dicts are sometimes needed for situations where
>  > a regular dict would keep objects alive forever.  E.g. we were made
>  > aware of a "leak" in JPython that could only be fixed with weak dicts:
>  > the Swing wrapper code has a global dict mapping widgets to callback
> 
>   That's a perfect example.  I've started working on some text
> describing the motivation; hopefully I'll have that fleshed out and
> checked in later this week.

Another example is some of the things in Zope use back-references for
ease of traversability (or worse keep weird counts hanging around).
Alot of these are negated by ZODB's ability to break cycles, but... a
lot of data structures would be hugely better from an architecture
perspective if we had a native weak reference.  

Chris
-- 
| Christopher Petrilli
| petrilli@amber.org


From est@hyperreal.org  Mon Nov  6 04:27:52 2000
From: est@hyperreal.org (est@hyperreal.org)
Date: Sun, 5 Nov 2000 20:27:52 -0800 (PST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 5, 2000 09:48:47 pm"
Message-ID: <20001106042752.25360.qmail@hyperreal.org>

Guido van Rossum discourseth:
> [MAL]
> > These already exist... http://www.handshake.de/~dieter/weakdict.html
> > 
> > mx.Proxy also has an implementation which support weak references.
> 
> Thanks.  For Fred to read...

He may want to also see my http://www.hyperreal.org/~est/python/weak
(also registered in the vaults), an implementation of weak dicts and
proxy maps that doesn't require the contained objects to be
subclassed.  It even has documentation in standard format (!)
including some words about motivation.

One simple implementation approach to make it work for all objects
might be to have:

int PyObject_PushWeakHandler(PyObject *o,
                             void (*handler)(PyObject *o, PyObject *data),
                             PyObject *data);

When an object is deallocated all its handlers would be called on the
object and the data that was registered with the handler.  I believe
this would make weak dicts a very simple extension module.  With this
approach I suspect DATA should be incref'd by the call to
PyObject_PushWeakHandler() and decref'd after the associated handler
is called.

Best,

Eric


From est@hyperreal.org  Mon Nov  6 04:39:29 2000
From: est@hyperreal.org (est@hyperreal.org)
Date: Sun, 5 Nov 2000 20:39:29 -0800 (PST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net> from Jeremy
 Hylton at "Nov 5, 2000 10:55:17 pm"
Message-ID: <20001106043929.2515.qmail@hyperreal.org>

Jeremy Hylton discourseth:
> 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

This point is particularly worrisome to me because of a common pattern
I see in my own Python development work.  I'll define a class which is
parameterized with some callbacks.  Sometimes, when profiling reveals
the need, I'll move these classes to C.  If the client of the class is
using continuations via its callbacks, it may suddenly break.  This
seems a step back in the modularity I can count on in my program
components.

..and I say this as a long-time, die-hard Schemer. :)

I definitely pine for micro-threads in some of my application domains
though.

Eric


From greg@cosc.canterbury.ac.nz  Mon Nov  6 04:50:09 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 17:50:09 +1300 (NZDT)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy@alum.mit.edu>:

> Are continuations necessary to implement them or are
> there other options?

I think you'll find that any implementation of microthreads
or coroutines or whatever you want to call them, that
doesn't rely on playing nonportable tricks with the C
stack, will be just as mindbending as Stackless.

> The problem is that continuations break the notion a C API call will
> always return an answer;

So do threads or coroutines. As soon as you have multiple
threads of control, you have the chance that one of them
will switch to another and never get back.

> I assume Stackless raises an error in this case,
> but this seems pretty messy

This messiness isn't the fault of Stackless itself, but of the
large amount of code which *hasn't* been converted to the
Stackless Way. If the whole of Python and everything that
it calls were made truly stackless, the problem would not
arise.

Doing so, however, would not be fun. It wouldn't be fun
for any future extension writers, either.

I can't see any way out of this. Continuations/coroutines/
microthreads are all basically the same thing underneath, and
they imply an execution model that just doesn't fit well
with C.

Maybe we need to reimplement Python in Scheme, and then
feed it through a good Scheme compiler. SPython, anyone? 

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From moshez@zadka.site.co.il  Mon Nov  6 13:52:55 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Mon, 06 Nov 2000 15:52:55 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Sun, 05 Nov 2000 20:19:02 EST." <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
Message-ID: <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>

[GvR]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

I think the solution is obvious -- stop going offline to entertain guests.

[GvR]
> I was hoping that Tim would chime in

Me too. I even tried to drag him in by mentioning 754.

[GvR]
> I like the idea of a PEP to rework the numeric model.  I think that
> Moshe, being a mathematician by training, will make a good editor.  I
> think that this has a snowball's chance in hell to make it into Python
> 2.1 however -- there are many deep issues, and it's a bad idea to
> experiment too much in the released version of the language.

The reason the PEP was written now was because you started making sounds
of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
to show a less dangerous (and less innovative) way of getting similar
usability.


[GvR]
> (1) As long as the internal representation is not the same as what is
>     commonly printed, there will be surprises -- with rationals just
>     as much as with floating point.  There are issues with decimal
>     floating point too, but they are only the issues having to do with
>     loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
>     1.0) and not with loss of precision in the printing, where most of
>     the "bug reports" we get seem to concentrate.

My PEP does not yet deal with either written or inputted representation.

[GvR]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

Those calculations, if performed with floating points, would often build
up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
to be wildly inaccurate.

[GvR]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

You'll note that my PEP does not mention floating point explicitly --
and once again I mention that my PEP does not yet deal with number
literals. All it allows (not requires) is for things like math.sqrt()
to return inexact results. Naive implementations (which we might use)
would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
course, good implementations of math.sqrt() would realize that 1 has
an exact root, but that might be too hard to do for not enough gain.

[GvR]
> If we take its meaning literally, the isreal() function should only
> return true for numbers for which isrational() is also true:

Correct -- in my current model. If you later add things like constructive
reals, that is no longer true: if I have a constructive Pi, it's not 
rational. 

[GvR]
> mathematically speaking, real numbers that aren't also rational don't
> have an easy finite representation, since they are numbers like
> sqrt(2) or pi.

But numbers don't have to have a finite (periodic really) representation
to be representable in the computer: what about infinite continued fractions,
for example?

[GvR]
> Finally, the PEP doesn't talk about how the numeric model can be
> extended

That's because its rich enough not to need it. 
Let me explain exactly what I mean: as long as all field operations between
Python numbers give honest to god Python numbers, then everything else
can be solved with Python's current model of coercions, and can be solved
well when the coercion PEP will be written.

[GvR]
> I've
> heard of wild floating point representations that make multiplication
> and division really cheap but addition a pain, rather than the other
> way around;

Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
that instead of inexact() and math.{...}. How these numbers interact
with builtin Python numbers is your responsibility -- and that's what
the coercions are for.

Same goes for gmp: as long as you're not expecting to be able to change
10000000000+10000000000 to be a gmp long rather then a Python long, then
there shouldn't be a problem.
-- 
Moshe Zadka <sig@zadka.site.co.il>


From paulp@ActiveState.com  Mon Nov  6 06:24:27 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Sun, 5 Nov 2000 22:24:27 -0800 (PST)
Subject: [Python-Dev] Warnings PEP
Message-ID: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com>

Abstract

    This PEP describes a generalized warning mechanism for Python 2.1. The
    primary purpose of this mechanism is to alert the programmer or user
    of a program to potential or likely errors which, for whatever reason,
    are not considered exception-worthy. For example, this might be done
    to keep old code working during a transitional period or to alert the
    programmer or user of a recoverable error.

Syntax

    assert >> cls, test[[[, arg], arg]...]

    "cls" may be any callable object that takes a list as a single
    argument argument list and returns an object with the required
    attributes "get_action" and "format_message"
    
       * get_action() -> "warn"|"error"|"suppress"
       * format_message() -> string

    A provided base class implements these methods in a reusable
    fashion. Warning creators are encouraged to merely subclass.

    This extended form of the assertion statement calls the assertion
    handler code in the new "assertions" module. 

    The semantics of the built-in assertion handler are defined by the
    following code. It should be exposed in a new "assertions" module.

    def handle_assertion(cls, message = ""):
        "This code is called when an assertion fails and cls is not None"

	obj = cls(message)
	action = obj.get_action()
            
        if action=="error":
            *** existing assertion code ***
        elif action=="warn":
	    sys.stderr.write(obj.format_message())
        elif action=="suppress":
            pass
        else:
            assert action in ["warn","error","suppress"]

    Even if handle_assertion is implemented in C, it should be exposed as
    assertions.handle_assertion so that it may be overriden. 

    The generic warning base class is defined below:

    class Assertion:
        def __init__(self, *args):
	    if len(args) == 1:
	       self.args = args[0]
	    else:
	        self.args = args

	def format_message(self):
            sys.stderr.write("%s: %s" %(obj.__name__, self.args))

	def get_action(self):
	    return (self.get_user_request(self.__class__) 
                                or sys.default_warning_action)

        def get_user_request(self, cls):
            if cls.__name__ in sys.errors:
                return "error"
	    elif cls.__name__ in sys.warnings:
                return "warn"
            elif cls.__name__ in sys.disabled_warnings:
                return "suppress"
	    for base in cls.__bases__:
	        rc = self.get_user_request(base)
		if rc:
		    return rc
            else:
                return None

    The Assertion class should be implemented in Python so that it can be 
    used as a base class.

    Because this code inherently implements "warning state inheritance",
    it would be rare to override any of the methods, but this is possible
    in exceptional circumstances.

Command line

    By default the special variables have the following contents:

    sys.warnings = []
    sys.errors = []
    sys.suppress = []
    sys.default_warning_action = "warn"
    
    These variables may be changed from the command line. The command line
    arguments are interpreted as described below:

    -w XXX => sys.warnings.append("XXX")
    -e XXX => sys.errors.append("XXX")
    -no-w XXX => sys.suppress.append("XXX")
    -wall => sys.default_warning_action => "warn"
    -eall => sys.default_warning_action => "error"
    -no-wall => sys.default_warning_action => "suppress"

    As per the code above, errors take precedence over warnings and
    warnings over suppressions unless a particular assertion class
    specifies otherwise.

Built-in warning objects:

    class exception_in_del(Assertion):
        "An exception was ignored in an __del__ method"

    class deprecation(Assertion):
        "This feature may be removed in a future version of Python."

    class dubious(Assertion):
        "There is a common error associated with this feature."

    These class definitions are part of the "Assertion" module. They
    should only ever be used when there exists a way for the programmer to 
    accomplish the same thing without triggering the warning. For instance
    the way to suppress the deletion exception is to trap exceptions in
    __del__ methods with a try/except block.




From tim_one@email.msn.com  Mon Nov  6 06:34:38 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 01:34:38 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>

[Guido]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

The scary thing is which one of those you clearly enjoyed more <wink>.

> I was hoping that Tim would chime in, but he's apparently taken the
> weekend off -- very much out of character. :-)

Exactly in character, alas:  I was obsessed with my new cable modem
connection.  I had years of stuff to learn about firewalls in two days --
not to mention years of pornography to download in one <wink>.

Some quickies for now:

+ Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
numeric model forever.

+ Everyone has IEEE-754 fp hardware today; some people actually want to use
it; Moshe doesn't, but whatever revamping we get needs to allow others their
delusions too.

> ...
> For example, Tim has conjectured that using binary floating point will
> always be a problem for the "unwashed masses" -- the only thing they
> might understand is decimal floating point,

At first glance, yes.  Complaints traced to the "binary" part of "binary fp"
vastly outnumber complaints due to the "fp" part *and* integer division
combined, on both Python-Help and the Tutor list.  So if we want to know
what actually trips up newbies, they've been telling us for years.  Decimal
fp would silence most of those complaints; but rationals would silence them
too (provided they're *displayed* in rounded decimal fp notation (restart
"str" vs "repr" rant, and that the interactive prompt uses the wrong one,
and ditto str(container))), plus a few more (non-obvious example:

    (1/49)*49

does not equal 1 in either decimal or IEEE-754 binary double fp, but does
equal 1 using rationals).

Note that Mike Cowlishaw (REXX's dad) has been working on a scheme to merge
REXX's decimal fp with IEEE-854 (the decimal variant of IEEE-754):

    http://www2.hursley.ibm.com/decimal/

I'll talk to Jim Fulton about that, since Cowlishaw is pushing a BCD variant
and Jim was wondering about that (around the change of the year, for use-- I
presume --in Zope).

Note also IBM's enhanced BigDecimal class for Java:

    http://www2.hursley.ibm.com/decimalj/

> ...
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)
> -- there are only inexact operations.  I'm not sure what to do with
> this though.

IEEE-754 defines exactly what to do with this, for binary floats (and your
hardware has an "inexact result" flag set or not after every fp operation).
Conversion of the string "1.0" to float must not set it; conversion of "0.1"
must set it; and similarly for + - * / sqrt:  "inexact result" gets set
whenever the infinitely precise result differs from the computed result.  So
inexactness there is neither a property of types nor of numbers, but of
specific computations.  Extreme example:

    x = 1./3.   # inexact
    y = x-x     # exact result (from inexact inputs!)

I know that this version (operation-based) of inexactness can be useful.  I
see almost no earthly use for calling every number of a given type inexact.
Tagging individual numbers with an exact/inexact bit is an extremely crude
form of interval arithmetic (where the intervals are single points or
infinite).

> ...
> I'll leave it to Tim to explain why inexact results may not be close
> to the truth.

> Tim may also break a lance for IEEE 754.

Somebody else on c.l.py offered to write a 754 PEP; delighted to let them
have it.

if-you-ever-approximate-people-will-get-confused-
    but-if-you-don't-they'll-run-out-of-time-or-memory-ly y'rs  - tim




From paul@prescod.net  Mon Nov  6 07:17:52 2000
From: paul@prescod.net (Paul Prescod)
Date: Sun, 05 Nov 2000 23:17:52 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A065B20.BBD1C1E3@prescod.net>

It's just coincidence that I was working on warnings at the same time
you were.

Our proposals seem to have almost no overlap. I think mine does less,
but is also much simpler. I'm always nervous about over-engineering
rather than incremental development.

-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From pf@artcom-gmbh.de  Mon Nov  6 09:05:12 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Mon, 6 Nov 2000 10:05:12 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 5, 2000  9:34:40 pm"
Message-ID: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

[Guido]:
> David Scherer proposed to spell this pragma as a "magical import"
> (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
at module level as a backward compatible method to introduce module level
pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
I still like David Scherers proposal very much.

[David]:
> I actually implemented 1/2==0.5 in Python 1.5.2, complete with a
> module-level backward compatibility flag.  The flag made an unusual use of
> the "global" statement, which appears to be accepted at toplevel by 1.5.2
> without any effect.  Therefore a statement like "global olddivision" will be
> silently ignored by 1.5.2 and earlier, and will result in the old behavior
> under my patch.  "global" even has the right sort of sound for module-level
> options :)
> 
> An outline of what I did:
> 
> 1.  Add new opcode BINARY_FRACTION to opcode.h and dis.py
> 2.  Add new flag "int c_olddivision" to struct compiling in compile.c
> 3.  Set c_olddivision to base->c_olddivision or 0 in jcompile
> 4.  Check for "global olddivision" outside a function definition in
> com_global_stmt, and set c_olddivision=1
> 5.  Check c_olddivision in com_term, and generate either BINARY_DIVISION or
> BINARY_FRACTION
> 6.  Add PyNumber_Fraction to abstract.h, and define it in abstract.c to
> explicitly check for a pair of integers and do float division
> 7.  Add a BINARY_FRACTION case to ceval.c, which calls PyNumber_Fraction
> instead of PyNumber_Divide.

BTW:  I think the "symbol" '//' is incredible ugly and starting with
IBMs JCL years ago all languages I encountered, that used this symbol
for something, did suck in some way or another.  I would appreaciate
very much, if it could be avoided alltogether to add a symbol '//'
to Python.  '//' will look like a comment delimiter to most people today.

Using a new keyword like 'div' in the tradition of languages like 
Modula-2 looks far more attractive to me.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From mal@lemburg.com  Mon Nov  6 09:14:12 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 10:14:12 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>
Message-ID: <3A067664.22D09D03@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Internationalization.  Barry knows what he wants here; I bet Martin
> > >   von Loewis and Marc-Andre Lemburg have ideas too.
> 
> [MAL]
> > We'd need a few more codecs, support for the Unicode compression,
> > normalization and collation algorithms.
> 
> Hm...  There's also the problem that there's no easy way to do Unicode
> I/O.  I'd like to have a way to turn a particular file into a Unicode
> output device (where the actual encoding might be UTF-8 or UTF-16 or a
> local encoding), which should mean that writing Unicode objects to the
> file should "do the right thing" (in particular should not try to
> coerce it to an 8-bit string using the default encoding first, like
> print and str() currently do) and that writing 8-bit string objects to
> it should first convert them to Unicode using the default encoding
> (meaning that at least ASCII strings can be written to a Unicode file
> without having to specify a conversion).  I support that reading from
> a "Unicode file" should always return a Unicode string object (even if
> the actual characters read all happen to fall in the ASCII range).
> 
> This requires some serious changes to the current I/O mechanisms; in
> particular str() needs to be fixed, or perhaps a ustr() needs to be
> added that it used in certain cases.  Tricky, tricky!

It's not all that tricky since you can write a StreamRecoder
subclass which implements this. AFAIR, I posted such an implementation
on i18n-sig.

BTW, one of my patches on SF adds unistr(). Could be that it's
time to apply it :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Mon Nov  6 09:45:21 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Mon, 6 Nov 2000 11:45:21 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011061142050.4175-100000@sundial>

On Sun, 5 Nov 2000, Guido van Rossum wrote:

>   - The file must be None or a shell matching pattern, e.g. "*foo";
>     the ".py" suffix is optional; a partial pathname may be given too.
>     So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
>     "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
>     the filter applies regardless of the file.

How about "file" must be None or a callable, and if it's a callable,
it will be called to check whether to print? If I'll want fnmatch,
I know where to find it.

>   - The message must be a None or a string.  If it is None, the filter
>     applies to all messages.  The message string may end in "*" to
>     match all messages with the given text (up to the "*").

Same remark. If I want re, I know where to find it.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From Moshe Zadka <moshez@math.huji.ac.il>  Mon Nov  6 10:42:34 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Mon, 6 Nov 2000 12:42:34 +0200 (IST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>

[Tim Peters]
> Some quickies for now:
> 
> + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
> numeric model forever.

OK, I hope this e-mail address reaches him: I got it off his webpage.

[Tim Peters]
> + Everyone has IEEE-754 fp hardware today; some people actually want to use
> it; Moshe doesn't, but whatever revamping we get needs to allow others their
> delusions too.

My proposal has nothing *against* 754 either. For example, "all inexact
operations are done compliant to 754" is a perfectly acceptable addition.

[Tim Peters, about rationals]
> (provided they're *displayed* in rounded decimal fp notation (restart
> "str" vs "repr" rant, and that the interactive prompt uses the wrong one,
> and ditto str(container))),

Tim, that's the other PEP <wink>

[Tim Peters]
> IEEE-754 defines exactly what to do with this, for binary floats (and your
> hardware has an "inexact result" flag set or not after every fp operation).

Cool! I didn't know about that.

[Tim Peters]
> Conversion of the string "1.0" to float must not set it; conversion of "0.1"
> must set it; and similarly for + - * / sqrt:  "inexact result" gets set
> whenever the infinitely precise result differs from the computed result.

Is there some API for it in C? If not, we might as well assume that any
floating point number is inexact.

[Tim Peters]
> So
> inexactness there is neither a property of types nor of numbers, but of
> specific computations.  Extreme example:
> 
>     x = 1./3.   # inexact
>     y = x-x     # exact result (from inexact inputs!)
> 
> I know that this version (operation-based) of inexactness can be useful.  I
> see almost no earthly use for calling every number of a given type inexact.
> Tagging individual numbers with an exact/inexact bit is an extremely crude
> form of interval arithmetic (where the intervals are single points or
> infinite).

The tagging is whether you *want* exact operations or inexact operations.

I.e.:

1.0/3 --> inexact
1/3 --> exact.

Well, this is the classical definition of "type": what do the operations
mean? Which is why I need the inexact() function in my PEP.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Mon Nov  6 12:13:06 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:13:06 +0100
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
 <3A03DBD2.7E024978@lemburg.com> <200011060125.UAA03986@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A052.1990A512@lemburg.com>

Guido van Rossum wrote:
> 
> Marc-Andre:
> > I can take over the coercion PEP: I've been working
> > on this before (see the proposal on my Python Pages).
> 
> Thanks, excellent (although I haven't seen your proposal yet).
> 
> > I would also like to know whether the PEP-0224 will be considered
> > for 2.1 if I update the patch to make it a little more robust
> > w/r to the problems mentioned in that PEP -- I'd really like
> > to see this in Python soon, since it makes documenting Python
> > programs so much easier.
> 
> I "kinda" like the idea of having attribute docstrings (meaning it's
> not of great importance to me) but there are two things I don't like
> in your current proposal:
> 
> 1. The syntax you propose is too ambiguous: as you say, stand-alone
>    string literal are used for other purposes and could suddenly
>    become attribute docstrings.

This can be fixed by introducing some extra checks in the
compiler to reset the "doc attribute" flag in the compiler
struct.
 
> 2. I don't like the access method either (__doc_<attrname>__).

Any other name will do. It will only have to match these
criteria:

* must start with two underscores (to match __doc__)
* must be extractable using some form of inspection (e.g. by using
  a naming convention which includes some fixed name part)
* must be compatible with class inheritence (i.e. should be
  stored as attribute)

> > Note that I won't get around to do much work on these before
> > January... way too busy at the moment :-/
> 
> That's a problem -- we really want to have the PEPs ready for review
> by mid December.  This will also be a problem for the coercion PEP --
> if you think you won't be able to work on it before then, I'd prefer
> to find another (co-)author.

I think I'll need a co-author for the PEPs -- I have high-priority
project running which has a deadline in mid-December too.

Much food-for-thought is already available (see the PEP 214 and
the coercion proposal on my Python Pages) and I should find
time for some review.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Mon Nov  6 12:25:26 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:25:26 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A336.5FD1EBC7@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Class/type dichotomy???
> 
> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types (which are created
> > on-demand whenever an assignment is made).
> >
> > This would enable
> > extending types with new methods and attributes. "Subclassing"
> > could then be emulated by using new contructors which add the
> > new or changed methods to each created type instance, e.g.
> >
> > class myclose:
> >
> >     def __init__(self, object, basemethod):
> >         self.object = object
> >         self.basemethod = basemethod
> >
> >     def __call__(self):
> >         print 'Closed file %s' % self.object
> >         self.basemethod()
> >
> > def myfile(filename):
> >     f = open(filename)
> >     # add/override attributes
> >     f.newattribute = 1
> >     # add/override methods
> >     f.close = myclose(f, f.close)
> >     return f
> >
> > Types would have to be made aware of this possibility. Python
> > could provide some helping APIs to make life easier for the
> > programmer.
> 
> But this would require an extra pointer field for *all* built-in
> types.  That would seriously impact the space requirements for ints
> and floats!

True.

> As long as we're proposing hacks like this that don't allow smooth
> subclassing yet but let you get at least some of the desired effects,
> I'd rather propose to introduce some kind of metaclass that will allow
> you to use a class statement to define this.  Thinking aloud:
> 
> import types
> filemetaclass = metaclass(types.FileType)
> 
> class myfile(filemetaclass):
> 
>       def __init__(self, filename):
>           filemetaclass.__init__(filename)
>           self.newattribute = 1
> 
>       def close(self):
>           myclose(self)
>           filemetaclass.close(self)
> 
> I'm not quite sure what kind of object "filemetaclass" here should be
> or what exactly "metaclass()" should do, but it could create a new
> type that has the lay-out of an existing file object, with an instance
> dictionary (for newattribute) tacked on the end.  Later maybe (I'm
> going to brainstorm with Jim Fulton about types and classes).

I think the problem we currently have with subclassing types
is strongly related to the fact that all Py<type>_Check()
macros only work on a address compare basis.

If we could find a way to change this to some kind of (very)
fast different lookup scheme we'd open a door which could
lead to making subclassing of types a whole lot easier.

<Brainstorming>

Perhaps a simple indirection could help...
instead of obj->ob_type == PyInteger_Type we'd write
obj->ob_type->base_type == PyInteger_Type_ID.

A subclass could then identify itself as integer subclass
by setting the base_type id to PyInteger_Type_ID. It would
of course have to publish the same internal structure in
order to remain compatible to the PyInteger_*() API, but there
would be a possibility to extend the object struct and
slots could also be replaced with new ones.

</>

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From jim@interet.com  Mon Nov  6 13:11:25 2000
From: jim@interet.com (James C. Ahlstrom)
Date: Mon, 06 Nov 2000 08:11:25 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com>
Message-ID: <3A06ADFD.5C735BBA@interet.com>

"M.-A. Lemburg" wrote:

I am happy to modify zipfile.py as desired by this group.
 
> I'm having trouble opening ZIP files created using InfoZIP's
> zip utility (which uses zlib) with zipfile.py:
> 
> >>> x = z.read('README')
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
>   File "/home/lemburg/lib/zipfile.py", line 242, in read
>     bytes = dc.decompress(bytes)
> zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> 
> Is this due to the installed zlib on my system being incompatible,
> or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> zip version 2.2.

I "borrowed" the zlib code from someone else, and it uses
undocumented features.  Since WinZip seems happy with the compression
I left it alone.  I wouldn't be surprised if other utilities have
problems, especially if they use zlib.  I am not sure what to
do about this.  My starting view is that there is only one
compression method, and if WinZip accepts it, then InfoZip
is wrong.  Is there perhaps a version difference in zlib
in Python vs InfoZip?

Could you send me an InfoZip zip file?

> Also, I wonder why zipfile forces the mode flag to be 'r',
> 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> to the mode flag instead ?!

The 'b' is not necessary, as it is always added by zipfile.py
when the file is opened.  The [rwa] each do something
different, and aren't really the same as a file open.  I
am not sure what you mean here.
 
> The ZipFile is also missing some kind of method which
> extracts files in the ZIP archive to a file-like object. This
> would be very useful for extracting large files from a ZIP
> archive without having to first read in the whole file into
> memory.

The "read(self, name)" method returns the bytes, which you can
write to a file if you want.  What new method would you like?

JimA


From gvwilson@nevex.com  Mon Nov  6 13:39:37 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Mon, 6 Nov 2000 08:39:37 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.10.10011060823080.26810-100000@akbar.nevex.com>

> Jeremy wrote:
> I tend to agree with you, Guido.  I think we would do well to purposefully
> omit continuations from the Python language.  There seems to be little need
> for a facility to implement arbitrary control structures in Python.  If
> Python support coroutines and microthreads, I am not sure what else would be
> needed.

I just finished reading Thomas and Hunt's "Programming Ruby" (the first book in
English on the language).  It's pretty clear that their favorite language
feature in Ruby is the block, which is any group of statements inside either
braces or do...end.  Blocks are invoked using the 'yield' construct, and can
take any number of arguments (enclosed in bars):

    def fibUpTo(max)
      i1, i2 = 1, 1
      while i1 <= max
        yield i1                      # 'call' the block
        i1, i2 = i2, i1 + i2
      end
    end

    fibUpTo(1000) { |f| print f, " " }

Most built-in types have iterators that understand blocks:

    [1, 3, 5].each { |i| puts i }     # prints 1, 3, and 5 on separate lines

Programmers can use blocks and 'yield' to create new control structures, subject
to the limitation that a statement can only be given one block (which means that
a multi-way interleaving loop can't be built out of blocks).  It would be
interesting to see how many of their examples can be done (easily) with
stackless...

Thanks,
Greg




From mal@lemburg.com  Mon Nov  6 14:00:33 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 15:00:33 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com>
Message-ID: <3A06B981.B3D73091@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> I am happy to modify zipfile.py as desired by this group.
> 
> > I'm having trouble opening ZIP files created using InfoZIP's
> > zip utility (which uses zlib) with zipfile.py:
> >
> > >>> x = z.read('README')
> > Traceback (innermost last):
> >   File "<stdin>", line 1, in ?
> >   File "/home/lemburg/lib/zipfile.py", line 242, in read
> >     bytes = dc.decompress(bytes)
> > zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> >
> > Is this due to the installed zlib on my system being incompatible,
> > or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> > zip version 2.2.
> 
> I "borrowed" the zlib code from someone else, and it uses
> undocumented features.  Since WinZip seems happy with the compression
> I left it alone.  I wouldn't be surprised if other utilities have
> problems, especially if they use zlib.  I am not sure what to
> do about this.  My starting view is that there is only one
> compression method, and if WinZip accepts it, then InfoZip
> is wrong.  Is there perhaps a version difference in zlib
> in Python vs InfoZip?
> 
> Could you send me an InfoZip zip file?

As it turned out it was a false alarm: Python must have picked
up an old zlibmodule.so from somewhere which caused the
problem. A clean rebuild made zipfile work with my InfoZIP
files (all my Python Pages ZIP-archives are built using InfoZIP 2.2).

Now I get this error after working in interactive Python
mode with zipfile:

Exception exceptions.AttributeError: 
"ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

I guess the __del__ method has to be a bit more careful about
what it expects to find... sometimes the builtins may have
already been garbage collected.

> > Also, I wonder why zipfile forces the mode flag to be 'r',
> > 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> > to the mode flag instead ?!
> 
> The 'b' is not necessary, as it is always added by zipfile.py
> when the file is opened.  The [rwa] each do something
> different, and aren't really the same as a file open.  I
> am not sure what you mean here.

Sorry, I was only reading the code and got the impression that
mode testing was done on the complete string you pass to the
constructor ... I missed the line "key = mode[0]".
 
> > The ZipFile is also missing some kind of method which
> > extracts files in the ZIP archive to a file-like object. This
> > would be very useful for extracting large files from a ZIP
> > archive without having to first read in the whole file into
> > memory.
> 
> The "read(self, name)" method returns the bytes, which you can
> write to a file if you want.  What new method would you like?

I would like a method .copy(self, name, output) which
reads the file name from the ZIP archive and writes it directly to
the file-like object output. This should copy the file in chunks
of say 64kB in order to reduce memory load.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From nas@arctrix.com  Mon Nov  6 07:44:53 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Sun, 5 Nov 2000 23:44:53 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06A336.5FD1EBC7@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 01:25:26PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com>
Message-ID: <20001105234453.A8255@glacier.fnational.com>

On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> I think the problem we currently have with subclassing types
> is strongly related to the fact that all Py<type>_Check()
> macros only work on a address compare basis.

I don't think this is the problem, although it is closely
related.  The problem is that the interpreter uses these type
checks to special case the handling of certain types.
PyInstance_Check() is a big offender.

Behavior should be based solely on the type structure.  Extension
types would then be able to behave exactly like any other builtin
type.  Your coercion proposal and David's rich comparisions both
remove some of this special casing based on type.

  Neil


From gmcm@hypernet.com  Mon Nov  6 14:54:57 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 09:54:57 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
References: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <3A067FF1.10498.57C83EB@localhost>

Jeremy wrote:

> ...  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.

I'm not sure what you mean here. So far, no one has asked for 
anything to get added to the *language* (although generators 
and coroutines could be made nicer with a bit of keyword 
support, it's not necessary).
 
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?  How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?

If by "control structure" you mean for / while / do type things, 
well, Python's not a functional language, and stackless / 
continuations won't grow you any new ones.

OTOH, if threads added new "control structures", then yes, 
continuations adds new ones (although from a language 
viewpoint, they don't look like anything new).
 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.

You mean when C code (called by Python) ends up calling 
eval_code2, and the Python code so invoked wants to use a 
continuation which "resides" in some other invocation of 
eval_code2? Yes, Christian gives you an error at that point (I 
believe; I've never stumbled on this personally).

> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation. 

Well, sys.exit breaks that notion too :-). If an API says that a 
return is required, it's a programmer error not to return 
something.

I think it's universally agreed that there should be a "coroutine" 
API. Perhaps it is best done from scratch, or perhaps it is to 
the continuation module as threading is to thread. But I think 
Greg Ewing is right - it doesn't make much difference as far as 
stackless is concerned.

> I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

I'm not understanding. 

In practical terms, I toasted my machine more times and more 
thoroughly playing with the thread module than I have with the 
continuation module. In that respect, continuations are a good 
deal safer - they don't have any effect on the state of your OS.

If you're playing with the raw "go-to" features of the 
continuation module, it's very easy to screw up. That's simply 
because code has to "balance", somehow. Call and return are 
the universally accepted way of doing that in the procedural 
community. But they're just a protocol based on go-to. The go-
to is still there, even if it's in the chip's microcode.

Christian's continuation module exposes a bunch of primitives. 
Most of the material on the pages I put up is concerned with 
how to mix and match those to get useful results (one 
example is coded 6 different ways). No one thinks this is a 
good "end state", just like the thread module was a lousy "end 
state".  But we can't steal Java's API here, and we can't steal 
from functional languages either, 'cause Python just ain't 
functional.

Everybody wants to see safer ways of using this stuff, but 
there's still a lot of experimenting to do. And I, for one, don't 
want to slam the door after one or two safe ways have been 
found. (And personally, I've found most of the attempts at, e.g. 
writing a "coroutine" class, less comprehensible than using 
the primitives directly.)



- Gordon


From tismer@tismer.com  Mon Nov  6 14:23:48 2000
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 16:23:48 +0200
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>
Message-ID: <3A06BEF4.95B773BD@tismer.com>


Greg Ewing wrote:
> 
> Jeremy Hylton <jeremy@alum.mit.edu>:
> 
> > Are continuations necessary to implement them or are
> > there other options?
> 
> I think you'll find that any implementation of microthreads
> or coroutines or whatever you want to call them, that
> doesn't rely on playing nonportable tricks with the C
> stack, will be just as mindbending as Stackless.
> 
> > The problem is that continuations break the notion a C API call will
> > always return an answer;
> 
> So do threads or coroutines. As soon as you have multiple
> threads of control, you have the chance that one of them
> will switch to another and never get back.

This is correct!
The only "special" thing with my continuation implementation
is that frames are armed to be able to accept a return
from a callee multiple times.
This little piece on top of frames turns them into full
continuations.

Without doing this, the API doesn't change, just the implementation
gets a little simpler.
What remains is still what Greg says:
The guarantee of stack-like frame execution no longer holds,
and every stackless C extension must either provide a way
to handle calls in a tail-recursive manner, *or* it must
enforce stack-like execution, like it is done today.

The *only* thing out of (coroutines, generators, uthreads)
which can be built without breaking this assumption are
the simple ICON-style generators.
But those can be implemented without becoming stackless
at all.

If you want full coroutines, or uthreads, the non-trivial
change of execution-order which Stackless permits *is*
necessary.

The step from there to supporting full continuations
is tiny, can be done easily or left completely.
The order of complexity of the Python implementation
is not increased by continuations.
In fact, supporting uthreads and coroutines and not only
stack-based generators impose the real problem.

That's one of my reasons to support continuations:
Making Python completely coroutine aware, without
tricking the C stack, is 90 percent of the problem.
But after walking that far, there is no reason
to leave the other 10 percent alone.

...
Greg:
> I can't see any way out of this. Continuations/coroutines/
> microthreads are all basically the same thing underneath, and
> they imply an execution model that just doesn't fit well
> with C.

Yes, you understood it completely.

> Maybe we need to reimplement Python in Scheme, and then
> feed it through a good Scheme compiler. SPython, anyone?

I believe the current implementation is quite pleasant
for a lot of people, while it isn't perfectly stackless.

A lot of internal operations could be made stackless by
introduction of a couple more opcodes, turning these
operations into bytecode. This would probably not cost
much speed, since only those code branches are a problem
which cause an interpreter recursion today, anyway.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From guido@python.org  Mon Nov  6 15:41:02 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 10:41:02 -0500
Subject: [Python-Dev] Three argument slices.
In-Reply-To: Your message of "Sat, 04 Nov 2000 16:21:48 GMT."
 <3a04376a.28016074@smtp.worldonline.dk>
References: <3a04376a.28016074@smtp.worldonline.dk>
Message-ID: <200011061541.KAA07351@cj20424-a.reston1.va.home.com>

> While updating the difference page in the Jython documentation, I came
> across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this respect?

Good question.  I haven't pronounced on Michael's patch

http://sourceforge.net/patch/?func=detailpatch&patch_id=100998&group_id=5470

because I don't know whether this is really a good idea.  I don't know
that I ever *will* pronounce uness someone points out how useful it is
for some practical application.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@wooz.org  Mon Nov  6 15:50:35 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:50:35 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
 <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
 <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
 <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <14854.54091.14275.140381@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <py-dev@zadka.site.co.il> writes:

    MZ> (which was meant for Barry to assign me a PEP number
    MZ> primarily...)

Looks like Guido beat me to it this morning and assigned 228 to your
PEP.  It really pisses me off though, him stepping on my toes there,
because clearly the PEP should have been assigned PEP .3/1.e-3

:)

-Barry


From barry@wooz.org  Mon Nov  6 15:59:40 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:59:40 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
 <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
 <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
 <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
 <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
 <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
 <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <20001105034044.M10135@lyra.org>
Message-ID: <14854.54636.49888.781754@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

    GS> Allocate yourself a PEP number and publish the darn thing. If
    GS> you don't feel comfortable grabbing a PEP number, then just
    GS> post it to the list or something.

A general rule is that meta-peps usually get assigned numbers < 100
and standards track peps just get the next free number above 200.
Pretty simple really.

However, I'm trying to encourage two things: only serious proposals
get pep'd, and peps ought to have a consistent style.  I want to avoid
approving peps for incomplete proposals (such as 207, not to purposely
pick on David).  There are enough good peps that you should largely be
able to figure out the style to use.  I can clean up any minor issues
after the fact.

I think the way Moshe did this one was fine.  He wrote it up and
posted it, and got assigned a pep number fairly quickly.

-Barry


From rrdn60@email.sps.mot.com  Mon Nov  6 16:04:41 2000
From: rrdn60@email.sps.mot.com (Norman Shelley (rrdn60))
Date: Mon, 06 Nov 2000 09:04:41 -0700
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <3A06D699.D3AFAC0@email.sps.mot.com>

Jeremy Hylton wrote:

> [Changed discussion list from general python-list to specific
> stackless.]
>
> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
>
>   >> I have put up 6 pages of information about stackless at
>   >> http://www.mcmillan-inc.com/stackless.html
>
>   GvR> Gordon, thanks for doing this.  I still have a review of
>   GvR> Stackless on my TODO list.  It takes a serious chunk of my time
>   GvR> to do it justice, and this continues to be a problem, but the
>   GvR> existience of your overview certainly helps.  I still think
>   GvR> that the current Stackless implementation is too complex, and
>   GvR> that continuations aren't worth the insanity they seem to
>   GvR> require (or cause :-), but that microthreads and coroutines
>   GvR> *are* worth having and that something not completely unlike
>   GvR> Stackless will be one day the way to get there...
>
> I tend to agree with you, Guido.  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.
>
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?

This kind of question/thought disturbs me.  It presumes we can determine
apriori all the ways one might wish to use the features that Stackless
provides.  Closing off or bounding innovation just because we can't
answer the question as to how it will be used will just cause future
forks in Python or promote or  non-Python choices.

> How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?
>
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?
>
> Jeremy
>
> _______________________________________________
> Stackless mailing list
> Stackless@starship.python.net
> http://starship.python.net/mailman/listinfo/stackless



From barry@wooz.org  Mon Nov  6 16:05:51 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:05:51 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
 <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
 <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
 <14853.52668.117844.28459@bitdiddle.concentric.net>
 <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>
Message-ID: <14854.55007.188953.545215@anthem.concentric.net>

BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
unreasonable that he'd have to wait until Monday to get a PEP number.
I reserve the right to apply Warsaw's 2nd Law to these cases. :)

-Barry

Warsaw's Second Law: Unbending Law of Commit Scheduling 
    Never change anything after 3pm on a Friday. 


From hinsen@cnrs-orleans.fr  Mon Nov  6 16:12:39 2000
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 6 Nov 2000 17:12:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011061235400.4175-100000@sundial> (message from
 Moshe Zadka on Mon, 6 Nov 2000 12:42:34 +0200 (IST))
References: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>
Message-ID: <200011061612.RAA05582@chinon.cnrs-orleans.fr>

> > + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"

I'd like to know what I am about to be sucked into here ;-)

> OK, I hope this e-mail address reaches him: I got it off his webpage.

That's fine, my web page always know how to reach me.

From your mail I get the impression that the discussion is about
some PEP. If you tell me which one I'll try to have a look at it.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------


From barry@wooz.org  Mon Nov  6 16:23:43 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:23:43 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
 <20001105201851.C27208@xs4all.nl>
 <20001105135439.V10135@lyra.org>
 <20001105231235.X12776@xs4all.nl>
Message-ID: <14854.56079.737563.729305@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas@xs4all.net> writes:

    TW> Well, syncmail was written to manage the Python CVS tree on a
    TW> slow Sun (I believe)

Correct.
    
    TW> and did an rsync-over-ssh to another machine as well. That can
    TW> definately take long ;) If we just remove the fork, the rest
    TW> of syncmail might just work, even with new files. In the mean
    TW> time, I'll check in my change. It might be the best thing to
    TW> do anyway, since it shouldn't interfere unless the file isn't
    TW> there.

Your patch looks good Thomas, thanks for tracking this down.  IIRC,
without the fork, the checkin would actually deadlock trying to get
the diff.

-Barry


From guido@python.org  Mon Nov  6 16:28:50 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 11:28:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 15:52:55 +0200."
 <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
 <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>
Message-ID: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
Algol-60 did this.  Fortran does this.  And rational numbers are less
innovative?

And what's dangerous about 1/2 yielding 0.5?

> My PEP does not yet deal with either written or inputted representation.

These affect the numeric model though (see Tim's posts) so should be
considered.

> [GvR]
> > (2) Rational numbers have the unpleasant property of growing
> >     unboundedly during many innocent calculations, thereby using up
> >     exorbitant amounts of memory and slowing down the calculation --
> >     often mysteriously, because what is displayed is truncated.
> 
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
is clearly an issue open for debate, but note that I speak from
experience: ABC used rationals unless you forced it to use reals, and
the rationals *did* cause real users to complain about how slow ABC
was.

> [GvR]
> > Another issue that I might bring up is that there are no inexact
> > numbers (each floating point number is perfectly exact and rational)
> 
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

Without a clear implementation plan your PEP is incomplete, PEP
guidelines notwithstanding.  Quality of implementation is important
for such a basic feature!

> [GvR]
> > If we take its meaning literally, the isreal() function should only
> > return true for numbers for which isrational() is also true:
> 
> Correct -- in my current model. If you later add things like constructive
> reals, that is no longer true: if I have a constructive Pi, it's not 
> rational. 
> 
> [GvR]
> > mathematically speaking, real numbers that aren't also rational don't
> > have an easy finite representation, since they are numbers like
> > sqrt(2) or pi.
> 
> But numbers don't have to have a finite (periodic really) representation
> to be representable in the computer: what about infinite continued fractions,
> for example?

So the question is, what implementation do you have in mind?  You
can't just go prescribe idealistic semantics and hope it gets
implemented by magic (even Tim can't do that :-).

> [GvR]
> > Finally, the PEP doesn't talk about how the numeric model can be
> > extended
> 
> That's because its rich enough not to need it. 
> Let me explain exactly what I mean: as long as all field operations between
> Python numbers give honest to god Python numbers, then everything else
> can be solved with Python's current model of coercions, and can be solved
> well when the coercion PEP will be written.

I think this deserves very explicit mention in your PEP.  An example
of how I would go about implementing my own Rational class or
extension type would help.

Also, the coercions PEP is still in need of an author.  Maybe you want
to take this on too?  It will help your numeric proposal if you can
write down how you think coercions should work.

> [GvR]
> > I've
> > heard of wild floating point representations that make multiplication
> > and division really cheap but addition a pain, rather than the other
> > way around;
> 
> Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
> that instead of inexact() and math.{...}. How these numbers interact
> with builtin Python numbers is your responsibility -- and that's what
> the coercions are for.

So explain how to do the coercions.  This will force you to be
explicit about implementation details.  (Both from Python and from C.)

> Same goes for gmp: as long as you're not expecting to be able to change
> 10000000000+10000000000 to be a gmp long rather then a Python long, then
> there shouldn't be a problem.

Fair enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Nov  6 17:02:51 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:02:51 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com>
Message-ID: <3A06E43B.D95EC267@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> > I think the problem we currently have with subclassing types
> > is strongly related to the fact that all Py<type>_Check()
> > macros only work on a address compare basis.
> 
> I don't think this is the problem, although it is closely
> related.  The problem is that the interpreter uses these type
> checks to special case the handling of certain types.
> PyInstance_Check() is a big offender.
> 
> Behavior should be based solely on the type structure.  Extension
> types would then be able to behave exactly like any other builtin
> type.  Your coercion proposal and David's rich comparisions both
> remove some of this special casing based on type.

Even though this could remove some of the problems, it doesn't
help much with a common use of Py<Type>_Check(): that of
using fast access macros and native Py<Type>_*() APIs once an
object has been classified as being of a certain type.
This usually improves performance.

By changing the simple address compare to a type handle
system, we might be able to add some more flexibility to
the system while keeping b/w compatibility.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Mon Nov  6 17:17:22 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:17:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."
 <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>
References: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>

> [Guido]:
> > David Scherer proposed to spell this pragma as a "magical import"
> > (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

[Peter Funk]
> Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
> at module level as a backward compatible method to introduce module level
> pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
> I still like David Scherers proposal very much.

Oops, you're right.  That URL mentions "global olddivision".  The
"import floatdivision" proposal came from a more recent private mail
from Bruce Sherwood.  I maintain that neither seems the right way to
spell "directive".  I think "import" is slightly better if the import
is supposed to enable a feature that is not supported by previous
versions, because the import will cause a clear failure on systems
that don't have the new feature (rather than silently giving wrong
results sometimes).

> BTW:  I think the "symbol" '//' is incredible ugly and starting with
> IBMs JCL years ago all languages I encountered, that used this symbol
> for something, did suck in some way or another.  I would appreaciate
> very much, if it could be avoided alltogether to add a symbol '//'
> to Python.  '//' will look like a comment delimiter to most people today.

Good point.  I've just used // as a placeholder for a new way to spell
integer division.

> Using a new keyword like 'div' in the tradition of languages like 
> Modula-2 looks far more attractive to me.

That's a possibility too.  It's a new keyword though, which has a much
higher threshold for acceptance than a new two-character operator
symbol.  We could spell it as a built-in function: div(a, b),
(analogous to divmod(a, b)) but that's not very user-friendly either.

Keep looking...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 17:20:03 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:20:03 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:14:12 +0100."
 <3A067664.22D09D03@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>
 <3A067664.22D09D03@lemburg.com>
Message-ID: <200011061720.MAA07862@cj20424-a.reston1.va.home.com>

[GvR]
> > Hm...  There's also the problem that there's no easy way to do Unicode
> > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > local encoding), which should mean that writing Unicode objects to the
> > file should "do the right thing" (in particular should not try to
> > coerce it to an 8-bit string using the default encoding first, like
> > print and str() currently do) and that writing 8-bit string objects to
> > it should first convert them to Unicode using the default encoding
> > (meaning that at least ASCII strings can be written to a Unicode file
> > without having to specify a conversion).  I support that reading from
> > a "Unicode file" should always return a Unicode string object (even if
> > the actual characters read all happen to fall in the ASCII range).
> > 
> > This requires some serious changes to the current I/O mechanisms; in
> > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > added that it used in certain cases.  Tricky, tricky!

[MAL]
> It's not all that tricky since you can write a StreamRecoder
> subclass which implements this. AFAIR, I posted such an implementation
> on i18n-sig.
> 
> BTW, one of my patches on SF adds unistr(). Could be that it's
> time to apply it :-)

Adding unistr() and StreamRecoder isn't enough.  The problem is that
when you set sys.stdout to a StreamRecoder, the print statement
doesn't do the right thing!  Try it.  print u"foo" will work, but
print u"\u1234" will fail because print always applies the default
encoding.

The required changes to print are what's tricky.  Whether we even need
unistr() depends on the solution we find there.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Mon Nov  6 16:19:21 2000
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 18:19:21 +0200
Subject: [Python-Dev] cgi.py and huge uploads problem
Message-ID: <3A06DA09.143D2D9@tismer.com>

Howdy,

there is a problem with the cgi.py implementation of Python 1.5.2
and uploading of huge files.
(found out by Joachim Rudolph, Virtual Photonics)

Class FieldStorage of cgi.py has a couple of methods which
add accumulated lines to a self.lines array. This array fills and
fills until the whole upload is done, with the side effect of
loading the whole file into memory. The memory is freed
after the whole upload is done.

This is no problem, until a company like VPI uses cgi.py to
upload whole distributions of 100 MB and more, via Zope. :-)

Looking into cgi.py, I can't find a reason why this happens.
Is this possibly just a debugging feature which is no longer
needed?
While cgi.py was modified for cosmetic resons, I didn't find
changes for Python 2.0 on this topic.

Does it make sense to use a debug flag for this, or should
the feature vanish completely?
Do you want a patch?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From moshez@zadka.site.co.il  Tue Nov  7 01:30:05 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:30:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Mon, 06 Nov 2000 11:28:50 EST." <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>  <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
Message-ID: <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

OK, I take back the innovation claim.

[Guido]
> And what's dangerous about 1/2 yielding 0.5?

Nothing, but then people would expect 1/3 to yield 0.333333333....,
while simultaneusly expecting (1/3)*3 == 1.
IOW, shoving floating points in people's faces is not user-friendly.

[Moshe]
> My PEP does not yet deal with either written or inputted representation.

[Guido]
> These affect the numeric model though (see Tim's posts) so should be
> considered.

I agree. That's why they are in the "unresolved issues" of the PEP.

[Guido]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

[Moshe]
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

[Guido]
> That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> is clearly an issue open for debate, but note that I speak from
> experience: ABC used rationals unless you forced it to use reals, and
> the rationals *did* cause real users to complain about how slow ABC
> was.

Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
list, and nobody ever complained. Besides, if "forcing" is simply saying
either inexact() (and note that inexact-exact operations are performed
inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
be), it might be easier. I don't know ABC well enough to say how the
forcing mechanism worked.

[Guido]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

[Moshe]
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

[Guido]
> Without a clear implementation plan your PEP is incomplete, PEP
> guidelines notwithstanding.  Quality of implementation is important
> for such a basic feature!

Well, I agree. When I set down to complete the PEP, I'll go over all
the math/cmath functions and remark how the implementation should change.
I also plan to draft an implementation design. I just wanted to throw
the idea out into the open, to get some feedback -- in no way can the
PEP be considered complete.

[Guido]
> So the question is, what implementation do you have in mind?  You
> can't just go prescribe idealistic semantics and hope it gets
> implemented by magic (even Tim can't do that :-).

Well, for constructive reals, none. It's too much of pain to implement,
and too little to gain. That wouldn't preclude a later day implementation,
in case it turns out not to be the case. Besides, the tower wouldn't
look clean without it.

[Guido]
> I think this deserves very explicit mention in your PEP.  An example
> of how I would go about implementing my own Rational class or
> extension type would help.

Well, I don't see why there should be a difference from what happens
currently. The thing is, the model will not support you telling it
at runtime what the results for operations on types it already knows
should be: that would make us Scheme, not Python. 

[Guido]
> Also, the coercions PEP is still in need of an author.  Maybe you want
> to take this on too?  It will help your numeric proposal if you can
> write down how you think coercions should work.

Smooth, real smooth. OK, sold to the highest bidder -- I take it.
I'll update it and 0000, and start reading MAL's pages.

[Guido]
> So explain how to do the coercions.  This will force you to be
> explicit about implementation details.  (Both from Python and from C.)

Again this is no different from current day Python, modulu other PEPs. <wink> 
-- 
Moshe Zadka <sig@zadka.site.co.il>


From gmcm@hypernet.com  Mon Nov  6 17:27:22 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 12:27:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."             <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <3A06A3AA.1401.6080F51@localhost>

[Peter Funk]
> > Using a new keyword like 'div' in the tradition of languages like 
> > Modula-2 looks far more attractive to me.
[Guido]:
> That's a possibility too.  It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.  We could spell it as a built-in function: div(a, b),
> (analogous to divmod(a, b)) but that's not very user-friendly either.
> 
> Keep looking...

FWIW, I remember as a newbie being sure that integer 
division was spelled "a div b". In fact when it didn't work, I went 
digging through PP, IPWP and the docs in dumbfounded 
disbelief.

this-is-your-brain-on-Modula-2-ly y'rs

- Gordon


From nas@arctrix.com  Mon Nov  6 10:35:29 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Mon, 6 Nov 2000 02:35:29 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06E43B.D95EC267@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 06:02:51PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com>
Message-ID: <20001106023529.B8639@glacier.fnational.com>

On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> Neil Schemenauer wrote:
> > Behavior should be based solely on the type structure.  Extension
> > types would then be able to behave exactly like any other builtin
> > type.  Your coercion proposal and David's rich comparisions both
> > remove some of this special casing based on type.
> 
> Even though this could remove some of the problems, it doesn't
> help much with a common use of Py<Type>_Check(): that of
> using fast access macros and native Py<Type>_*() APIs once an
> object has been classified as being of a certain type.
> This usually improves performance.

Can you clarify what you mean by "it doesn't help much"?  Do you
mean that extension types will not be able to perform as well as
types that get special treatment by the interpreter?  I think the
major problem that extension types _cannot_ behave the same as
the builtin types.

> By changing the simple address compare to a type handle
> system, we might be able to add some more flexibility to
> the system while keeping b/w compatibility.

I don't see what this buys us.  The Python interpreter shouldn't
care about which type object it is dealing with.  Can you give an
example of where you think this would be useful?

  Neil


From moshez@zadka.site.co.il  Tue Nov  7 01:40:30 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:40:30 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from barry@wooz.org (Barry A. Warsaw)
 of "Mon, 06 Nov 2000 11:05:51 EST." <14854.55007.188953.545215@anthem.concentric.net>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de> <Pine.GSO.4.10.10011051411170.28025-100000@sundial> <14853.36547.748584.450976@cj42289-a.reston1.va.home.com> <14853.52668.117844.28459@bitdiddle.concentric.net> <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>  <14854.55007.188953.545215@anthem.concentric.net>
Message-ID: <E13sxkU-0005PL-00@darjeeling.zadka.site.co.il>

[Barry]
> BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
> unreasonable that he'd have to wait until Monday to get a PEP number.

It is to me. If I work on Sunday, I might as well do some Python work too...
Problems of truly international open source projects <wink>
-- 
Moshe Zadka <sig@zadka.site.co.il>


From guido@python.org  Mon Nov  6 17:43:07 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:43:07 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:19:21 +0200."
 <3A06DA09.143D2D9@tismer.com>
References: <3A06DA09.143D2D9@tismer.com>
Message-ID: <200011061743.MAA08031@cj20424-a.reston1.va.home.com>

> there is a problem with the cgi.py implementation of Python 1.5.2
> and uploading of huge files.
> (found out by Joachim Rudolph, Virtual Photonics)
> 
> Class FieldStorage of cgi.py has a couple of methods which
> add accumulated lines to a self.lines array. This array fills and
> fills until the whole upload is done, with the side effect of
> loading the whole file into memory. The memory is freed
> after the whole upload is done.
> 
> This is no problem, until a company like VPI uses cgi.py to
> upload whole distributions of 100 MB and more, via Zope. :-)
> 
> Looking into cgi.py, I can't find a reason why this happens.
> Is this possibly just a debugging feature which is no longer
> needed?
> While cgi.py was modified for cosmetic resons, I didn't find
> changes for Python 2.0 on this topic.
> 
> Does it make sense to use a debug flag for this, or should
> the feature vanish completely?
> Do you want a patch?

You know, I have *no* idea why this is.  I have looked through various
revisions (this feature is as old as cgi.py:1.8) and cannot find any
use of or need for self.lines!  It just gets appended to.

So I think it's safe to toss all the references to self.lines and see
who complains.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Nov  6 17:36:54 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:36:54 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>
 <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>
Message-ID: <3A06EC36.61B32B4D@lemburg.com>

Guido van Rossum wrote:
> 
> [GvR]
> > > Hm...  There's also the problem that there's no easy way to do Unicode
> > > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > > local encoding), which should mean that writing Unicode objects to the
> > > file should "do the right thing" (in particular should not try to
> > > coerce it to an 8-bit string using the default encoding first, like
> > > print and str() currently do) and that writing 8-bit string objects to
> > > it should first convert them to Unicode using the default encoding
> > > (meaning that at least ASCII strings can be written to a Unicode file
> > > without having to specify a conversion).  I support that reading from
> > > a "Unicode file" should always return a Unicode string object (even if
> > > the actual characters read all happen to fall in the ASCII range).
> > >
> > > This requires some serious changes to the current I/O mechanisms; in
> > > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > > added that it used in certain cases.  Tricky, tricky!
> 
> [MAL]
> > It's not all that tricky since you can write a StreamRecoder
> > subclass which implements this. AFAIR, I posted such an implementation
> > on i18n-sig.
> >
> > BTW, one of my patches on SF adds unistr(). Could be that it's
> > time to apply it :-)
> 
> Adding unistr() and StreamRecoder isn't enough.  The problem is that
> when you set sys.stdout to a StreamRecoder, the print statement
> doesn't do the right thing!  Try it.  print u"foo" will work, but
> print u"\u1234" will fail because print always applies the default
> encoding.

Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
Perhaps we ought to let it call PyObject_Unicode() (which you
find in the patch on SF) instead for Unicode objects. That way
the file-like .write() method will be given a Unicode object
and StreamRecoder could then do the trick.

Haven't tried this, but it could work (the paths objects take
through Python to get printed are somewhat strange at times
-- there are just so many different possiblities and special
cases that it becomes hard telling from just looking at the
code).

> The required changes to print are what's tricky.  Whether we even need
> unistr() depends on the solution we find there.

I think we'll need PyObject_Unicode() and unistr() one way
or another. Those two APIs simply complement PyObject_Str()
and str() in that they always return Unicode objects and
do the necessary conversion based on the input object type.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Mon Nov  6 17:49:35 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:49:35 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Tue, 07 Nov 2000 03:30:05 +0200."
 <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
 <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>
Message-ID: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>

> [Guido]
> > That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> > is clearly an issue open for debate, but note that I speak from
> > experience: ABC used rationals unless you forced it to use reals, and
> > the rationals *did* cause real users to complain about how slow ABC
> > was.

[Moshe]
> Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
> list, and nobody ever complained. Besides, if "forcing" is simply saying
> either inexact() (and note that inexact-exact operations are performed
> inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
> be), it might be easier. I don't know ABC well enough to say how the
> forcing mechanism worked.

Scheme has no real users -- only CS types. :-)

ABC's forcing was as simple as writing ~x or mixing inexact numbers.
We did have the notion that 1.0 was an exact literal (to me it looks
exact!) so maybe that was an additional problem.

> [Guido]
> > I think this deserves very explicit mention in your PEP.  An example
> > of how I would go about implementing my own Rational class or
> > extension type would help.
> 
> Well, I don't see why there should be a difference from what happens
> currently. The thing is, the model will not support you telling it
> at runtime what the results for operations on types it already knows
> should be: that would make us Scheme, not Python. 

Agreed.  I think what makes me feel uneasy is that your proposal
assumes that there is One True Numeric Type, and all the rest are
second-class numbers.  Python currently makes no such claim: there are
many numeric types built in and you can add your own types that play
along.  The only thing that makes a difference is that there are
literals for the built-in types and not for the extension types; but
apart from that they have all the same privileges, and the coercion
rules work in everybody's favor (except when they don't :-). Placing
more emphasis on One True Numeric Type runs a risk of discriminating
against the others.

> [Guido]
> > Also, the coercions PEP is still in need of an author.  Maybe you want
> > to take this on too?  It will help your numeric proposal if you can
> > write down how you think coercions should work.
> 
> Smooth, real smooth. OK, sold to the highest bidder -- I take it.
> I'll update it and 0000, and start reading MAL's pages.

OK, it's a deal.  But make yourself a co-author with MAL.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov  6 17:53:36 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:53:36 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:36:54 +0100."
 <3A06EC36.61B32B4D@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>
 <3A06EC36.61B32B4D@lemburg.com>
Message-ID: <200011061753.MAA08112@cj20424-a.reston1.va.home.com>

[Guido]
> > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > when you set sys.stdout to a StreamRecoder, the print statement
> > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > print u"\u1234" will fail because print always applies the default
> > encoding.

[MAL]
> Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> Perhaps we ought to let it call PyObject_Unicode() (which you
> find in the patch on SF) instead for Unicode objects. That way
> the file-like .write() method will be given a Unicode object
> and StreamRecoder could then do the trick.

That's still not enough. Classes and types should be able to have a
__str__ (or tp_str) that yields Unicode too.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From moshez@zadka.site.co.il  Tue Nov  7 02:08:05 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 04:08:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Mon, 06 Nov 2000 12:49:35 EST." <200011061749.MAA08063@cj20424-a.reston1.va.home.com>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com> <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>  <200011061749.MAA08063@cj20424-a.reston1.va.home.com>
Message-ID: <E13syBB-0005Zr-00@darjeeling.zadka.site.co.il>

[Guido]
> Scheme has no real users -- only CS types. :-)

I resent that! I'm not a CS type: I never studied CS, and I refuse
to stand for that kind of mud slinging.

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

Let me clarify: exactness *is* in the operations. Since Python is
an OO language, an object decides semantics of operations on it. "1.0"
decides to have inexact operations on it, no matter how exact it
looks to you.

[Guido]
> Agreed.  I think what makes me feel uneasy is that your proposal
> assumes that there is One True Numeric Type, and all the rest are
> second-class numbers.  Python currently makes no such claim: there are
> many numeric types built in and you can add your own types that play
> along.

My proposal doesn't change that, as far as it's true. (And it isn't
really true: I cant have my new types as literals, or as results of
existing operations). It changes the number of built-in numeric types
to 1.
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From mal@lemburg.com  Mon Nov  6 18:15:27 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 19:15:27 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>
 <3A06EC36.61B32B4D@lemburg.com> <200011061753.MAA08112@cj20424-a.reston1.va.home.com>
Message-ID: <3A06F53F.2CF6760@lemburg.com>

Guido van Rossum wrote:
> 
> [Guido]
> > > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > > when you set sys.stdout to a StreamRecoder, the print statement
> > > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > > print u"\u1234" will fail because print always applies the default
> > > encoding.
> 
> [MAL]
> > Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> > Perhaps we ought to let it call PyObject_Unicode() (which you
> > find in the patch on SF) instead for Unicode objects. That way
> > the file-like .write() method will be given a Unicode object
> > and StreamRecoder could then do the trick.
> 
> That's still not enough. Classes and types should be able to have a
> __str__ (or tp_str) that yields Unicode too.

Instances are allowed to return Unicode through their __str__
method and PyObject_Unicode() will pass it along. PyObject_Str()
will still convert it to an 8-bit string though because there's
too much code out there which expects a string object (without
checking !) ... even the Python core.

So if you print an instance which returns Unicode through __str__,
the wrapper should see a real Unicode object at its end... at least
I think we're getting closer ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From barry@wooz.org  Mon Nov  6 18:32:44 2000
From: barry@wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 13:32:44 -0500 (EST)
Subject: [Python-Dev] cgi.py and huge uploads problem
References: <3A06DA09.143D2D9@tismer.com>
 <200011061743.MAA08031@cj20424-a.reston1.va.home.com>
Message-ID: <14854.63820.201069.434450@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> You know, I have *no* idea why this is.  I have looked
    GvR> through various revisions (this feature is as old as
    GvR> cgi.py:1.8) and cannot find any use of or need for
    GvR> self.lines!  It just gets appended to.

    GvR> So I think it's safe to toss all the references to self.lines
    GvR> and see who complains.

There are two bug reports related to this in the SF database.  The
first one was 110674, which we closed after adding the feature request
to PEP 42.  The second is 119806, which looks fairly new, but wasn't
submitted by Chris or Joachim.

I came to the same conclusion Guido does above when I looked at 110674
in the Python 2.0 time frame, but didn't feel comfortable making that
change for 2.0.  I think it's correct to make the change now.

I will do the following:

- remove self.lines from cgi.py
- close bug #119806
- update pep 42

Attached below is the patch.

-Barry

-------------------- snip snip --------------------
Index: cgi.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/cgi.py,v
retrieving revision 1.55
diff -u -r1.55 cgi.py
--- cgi.py	2000/10/03 13:51:09	1.55
+++ cgi.py	2000/11/06 18:32:18
@@ -497,7 +497,6 @@
 
         self.list = self.file = None
         self.done = 0
-        self.lines = []
         if ctype == 'application/x-www-form-urlencoded':
             self.read_urlencoded()
         elif ctype[:10] == 'multipart/':
@@ -633,7 +632,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             self.file.write(line)
 
     def read_lines_to_outerboundary(self):
@@ -646,7 +644,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:
@@ -676,7 +673,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:


From guido@python.org  Mon Nov  6 19:09:26 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 14:09:26 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 13:32:44 EST."
 <14854.63820.201069.434450@anthem.concentric.net>
References: <3A06DA09.143D2D9@tismer.com> <200011061743.MAA08031@cj20424-a.reston1.va.home.com>
 <14854.63820.201069.434450@anthem.concentric.net>
Message-ID: <200011061909.OAA08466@cj20424-a.reston1.va.home.com>

>     GvR> You know, I have *no* idea why this is.  I have looked
>     GvR> through various revisions (this feature is as old as
>     GvR> cgi.py:1.8) and cannot find any use of or need for
>     GvR> self.lines!  It just gets appended to.
> 
>     GvR> So I think it's safe to toss all the references to self.lines
>     GvR> and see who complains.

[Barry]
> There are two bug reports related to this in the SF database.  The
> first one was 110674, which we closed after adding the feature request
> to PEP 42.  The second is 119806, which looks fairly new, but wasn't
> submitted by Chris or Joachim.
> 
> I came to the same conclusion Guido does above when I looked at 110674
> in the Python 2.0 time frame, but didn't feel comfortable making that
> change for 2.0.  I think it's correct to make the change now.
> 
> I will do the following:
> 
> - remove self.lines from cgi.py
> - close bug #119806
> - update pep 42
> 
> Attached below is the patch.

Patch looks good.  Go for it.

Somehow when I skimmed the original bug reports I never really
understood what was going on -- thanks to Christian for pointing out
*exactly* what was the problem in words I could understand. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Nov  6 20:01:40 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 21:01:40 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>
Message-ID: <3A070E24.B07FA7D7@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> > Neil Schemenauer wrote:
> > > Behavior should be based solely on the type structure.  Extension
> > > types would then be able to behave exactly like any other builtin
> > > type.  Your coercion proposal and David's rich comparisions both
> > > remove some of this special casing based on type.
> >
> > Even though this could remove some of the problems, it doesn't
> > help much with a common use of Py<Type>_Check(): that of
> > using fast access macros and native Py<Type>_*() APIs once an
> > object has been classified as being of a certain type.
> > This usually improves performance.
> 
> Can you clarify what you mean by "it doesn't help much"?  Do you
> mean that extension types will not be able to perform as well as
> types that get special treatment by the interpreter?  I think the
> major problem that extension types _cannot_ behave the same as
> the builtin types.

If you only define types by interface, the interpreter will have
to apply the interface availability checks every time it calls
a slot. This would cause a major performance hit which would not
be acceptable.

The "doesn't help much" refers to the fact that once an object
has been identified as say Integer you are free to use whatever
access macro or function you choose.

> > By changing the simple address compare to a type handle
> > system, we might be able to add some more flexibility to
> > the system while keeping b/w compatibility.
> 
> I don't see what this buys us.  The Python interpreter shouldn't
> care about which type object it is dealing with.  Can you give an
> example of where you think this would be useful?

Take e.g. dictionaries: you could easily add a new dictionary
type which uses case-insensitive string keys by extending the
existing dictionary type.

The new type would reuse most of the slots of the original
type and only replace the ones needed for lookup with the
new logic for case-insensitivity.

Then it sets the type id to PyDict_TypeID and Python will
use it as if it were an original dictionary object. The
underlying type objects would be different though (and also
the type object address which is currently used to identify
a builtin type).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Mon Nov  6 21:02:27 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 16:02:27 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Sun, 05 Nov 2000 22:24:27 PST."
 <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com>
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com>
Message-ID: <200011062102.QAA09294@cj20424-a.reston1.va.home.com>

Paul, thanks for submitting a warnings framework.  I'd like to give
some feedback, comparing it to my own proposal.  Please also give
explicit feedback on mine!

[Paul Prescod]
> Abstract
> 
>     This PEP describes a generalized warning mechanism for Python 2.1. The
>     primary purpose of this mechanism is to alert the programmer or user
>     of a program to potential or likely errors which, for whatever reason,
>     are not considered exception-worthy. For example, this might be done
>     to keep old code working during a transitional period or to alert the
>     programmer or user of a recoverable error.

What's missing here is a set of requirements.  Comparing your proposal
to my requirements, I find that you are not addressing two
requirements that are important in my mind: a convenient and efficient
C API, and a mechanism that prints a warning message only the first
time that the warning is issued.  These are important to me, because I
expect that most warnings will probably be generated by C code, and in
order to be useful we must avoid mindless repetition.  If a function
divides two integers using the / operator, this is being detected by C
code (the int or long division implementation) and we only want to
print the warning once per program execution and per source location.

My expectation is that if a single occurrence (in the program) of a
warning condition caused an endless sequence of warnings to be spit
out, people would quickly grow a habit of disabling all warnings, thus
defeating the purposes.

Warnings are more a human engineering issue than a technical issue!
That's also why I am emphasizing a C API -- I want to make it real
easy to ussue quality warnings in the runtime.  It's also why I
specify a rich (probably too rich!) filtering mechanism.

> Syntax
> 
>     assert >> cls, test[[[, arg], arg]...]

I have several problems with this.  First of all, using "assert" means
that in "optimizing" mode (python -O) you won't get *any* warnings.  I
think that the decision to disable all warnings should be independent
from the decision to "optimize".  Second, you're hypergeneralizing the
extended print syntax.  Just because I think it's okay to add >>file
to the print syntax doesn't mean that it's now okay to add >>object
syntax to all statements!

I also don't see what warnings have to do with assertions.  Assertions
are a mechanism to check for error conditions.  What happens if the
error is detected is of less importance -- it could raise an exception
(Python), issue a fatal error (C), or do nothing (in -O mode).

With warnings I believe the issue is not so much the detection of the
condition (for which a regular 'if' statement does just fine) but the
reporting.  Again, this is motivated by the fact that I expect that
flexible filtering is essential for a successful warning mechanism.

>     "cls" may be any callable object that takes a list as a single
>     argument argument list and returns an object with the required
>     attributes "get_action" and "format_message"
>     
>        * get_action() -> "warn"|"error"|"suppress"
>        * format_message() -> string
> 
>     A provided base class implements these methods in a reusable
>     fashion. Warning creators are encouraged to merely subclass.

This is just a matter of exposition, but when I first read your PEP I
had a hard time figuring out the purpose of the cls object.  It wasn't
until I got to the very end where I saw your example classes that I
realized what it is: it represents a specific warning condition or a
group of related warning conditions.

>     This extended form of the assertion statement calls the assertion
>     handler code in the new "assertions" module. 

I won't harp on this each time, but I'd like to point out once more
that "assertion" is the wrong name for a warning feature.  Although it
isn't part of the Zen of Python (by Tim Peters), it should be: a
suggestive name for a feature is worth half a spec!

>     The semantics of the built-in assertion handler are defined by the
>     following code. It should be exposed in a new "assertions" module.
> 
>     def handle_assertion(cls, message = ""):
>         "This code is called when an assertion fails and cls is not None"
> 
>         obj = cls(message)
>         action = obj.get_action()
>             
>         if action=="error":
>             *** existing assertion code ***

That's just

              raise AssertionError, message

Right?

>         elif action=="warn":
>             sys.stderr.write(obj.format_message())
>         elif action=="suppress":
>             pass
>         else:
>             assert action in ["warn","error","suppress"]
> 
>     Even if handle_assertion is implemented in C, it should be exposed as
>     assertions.handle_assertion so that it may be overriden. 

Suggestion: have separate warning and error handlers, so that if I
want to override these branches of the if statement I don't have to
repeat the entire handler.

>     The generic warning base class is defined below:
> 
>     class Assertion:
>       def __init__(self, *args):
>           if len(args) == 1:
>               self.args = args[0]
>           else:
>               self.args = args
> 
>       def format_message(self):
>           sys.stderr.write("%s: %s" %(obj.__name__, self.args))
> 
>       def get_action(self):
>           return (self.get_user_request(self.__class__) 
>                   or sys.default_warning_action)
> 
>       def get_user_request(self, cls):
>           if cls.__name__ in sys.errors:
>               return "error"
>           elif cls.__name__ in sys.warnings:
>               return "warn"
>           elif cls.__name__ in sys.disabled_warnings:

I see no definition of sys.disabled_warnings.  Instead of
sys.disabled_warnings you meant sys.suppress, right?

>               return "suppress"
>           for base in cls.__bases__:
>               rc = self.get_user_request(base)
>               if rc:
>                   return rc
>           else:
>               return None

This approach (searching for the class name or the name of one of its
base classes in a list) doesn't look very object-oriented.  It would
make more sense to store the desired return value as a class or
instance attribute.  The default warning action could be stored on the
base class.

>     The Assertion class should be implemented in Python so that it can be 
>     used as a base class.
> 
>     Because this code inherently implements "warning state inheritance",
>     it would be rare to override any of the methods, but this is possible
>     in exceptional circumstances.
> 
> Command line
> 
>     By default the special variables have the following contents:
> 
>     sys.warnings = []
>     sys.errors = []
>     sys.suppress = []
>     sys.default_warning_action = "warn"
>     
>     These variables may be changed from the command line. The command line
>     arguments are interpreted as described below:
> 
>     -w XXX => sys.warnings.append("XXX")
>     -e XXX => sys.errors.append("XXX")
>     -no-w XXX => sys.suppress.append("XXX")
>     -wall => sys.default_warning_action => "warn"
>     -eall => sys.default_warning_action => "error"
>     -no-wall => sys.default_warning_action => "suppress"

Python doesn't support long options (I don't *like* long options so I
doubt that this is a good occasion to start lobbying for them :-).  We
can come up with different options though.

>     As per the code above, errors take precedence over warnings and
>     warnings over suppressions unless a particular assertion class
>     specifies otherwise.

I would use a different precedence scheme: a more specific filter
takes precedence over a more general filter.  So -eall -wdubious would
mean that "dubious" class warnings are warnings but all others are
errors, and -wall -edubious would mean the opposite.

> Built-in warning objects:
> 
>     class exception_in_del(Assertion):
>         "An exception was ignored in an __del__ method"
> 
>     class deprecation(Assertion):
>         "This feature may be removed in a future version of Python."
> 
>     class dubious(Assertion):
>         "There is a common error associated with this feature."
> 
>     These class definitions are part of the "Assertion" module. They
>     should only ever be used when there exists a way for the programmer to 
>     accomplish the same thing without triggering the warning. For instance
>     the way to suppress the deletion exception is to trap exceptions in
>     __del__ methods with a try/except block.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Mon Nov  6 21:36:49 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:49 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com>

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
new rules, int div always truncates).

So far as innovation goes, no choice on the table so far is innovative
(neither mathematically nor in programming languages), so there's no basis
for choosing there.

Guido, *why* did ABC use rationals by default?  Was that driven by usability
studies?




From tim_one@email.msn.com  Mon Nov  6 21:36:50 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPHHOAA.tim_one@email.msn.com>

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

I seriously wonder whether that was *the* problem with ABC:  not only was
1.0 treated as an exact rational in ABC, so was 6.02e23 and 3.14159e-314
etc.  At least for me, this caused rationals to get used in many places I
*intended* to use floats.  I assume many others got burned by this too, as
I'd say it's impossible for users coming from other languages not to see
6.02e23 etc as float literals.




From paulp@ActiveState.com  Mon Nov  6 22:01:27 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Mon, 06 Nov 2000 14:01:27 -0800
Subject: [Python-Dev] Warnings PEP
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>
Message-ID: <3A072A37.C2CAFED1@activestate.com>

Guido van Rossum wrote:
> 
> Paul, thanks for submitting a warnings framework.  I'd like to give
> some feedback, comparing it to my own proposal.  Please also give
> explicit feedback on mine!

Okay, but I'm a little bit worried about getting into a science project
-- in terms of my time to work on it. I'm perfectly happy with the two
requirements you mentioned but I haven't thought enough about filters to
be able to whip off ideas about them quickly.

> What's missing here is a set of requirements.  Comparing your proposal
> to my requirements, I find that you are not addressing two
> requirements that are important in my mind: a convenient and efficient
> C API, and a mechanism that prints a warning message only the first
> time that the warning is issued.  

I agree that these are both important things and I will add them to my
proposal. Also, I like your requirement that the target for warnings
should be easily customizable.

> Warnings are more a human engineering issue than a technical issue!
> That's also why I am emphasizing a C API -- I want to make it real
> easy to ussue quality warnings in the runtime.  It's also why I
> specify a rich (probably too rich!) filtering mechanism.

Let me put out the strawman proposal that "grep" is a nicely orthogonal
filtering mechanism. <duck!>

I guess my thinking is that you turn off warnings at the source. If you
get a warning about __del__ exceptions then you put in a try/except. If
you get a warning about an unused variable then you assign it to itself.
If you get a warning about integer division then you should pass in a
float and so forth.

> > Syntax
> >
> >     assert >> cls, test[[[, arg], arg]...]
> 
> I have several problems with this.  First of all, using "assert" means
> that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> think that the decision to disable all warnings should be independent
> from the decision to "optimize".  

Arguable. I see it as "release" and "debug" configurations. python and
python_d.

> Second, you're hypergeneralizing the
> extended print syntax.  Just because I think it's okay to add >>file
> to the print syntax doesn't mean that it's now okay to add >>object
> syntax to all statements!

Well getting new syntax into Python is really, really hard so we've got
to squeeze out as much value out of what we have as possible. But
anyhow, assertions are not allowed to 

You and I agree that there is an important sociological dimension to
this. We can't require:

  import warnings

  warnings.warn("foo", "bar")

I prefer:

  warn Foo, "Bar"

just like:

  raise Foo, "Bar"

> I also don't see what warnings have to do with assertions.  Assertions
> are a mechanism to check for error conditions.  What happens if the
> error is detected is of less importance -- it could raise an exception
> (Python), issue a fatal error (C), or do nothing (in -O mode).

Warnings are issued when an error or dubious construct is detected.
Assertions are "fatal warnings". You agree that it is appropriate for
some "warnings" to kill the app in some circumstances. Isn't it just a
hop-skip-and-a-jump to say that warnings and errors are just points on
the spectrum:

Report Once
Report Always
Report and Die

Python trained me to think of function calls and object constructions as
being basically the same thing -- and keyword arguments and variable
names being basically the same thing. etc.

> With warnings I believe the issue is not so much the detection of the
> condition (for which a regular 'if' statement does just fine) but the
> reporting.  Again, this is motivated by the fact that I expect that
> flexible filtering is essential for a successful warning mechanism.

I don't see why assertions need special syntax but warnings do not! I
would have been happy for a "warn" keyword with assert-like syntax but I
don't think I'll see that in my lifetime.

> This is just a matter of exposition, but when I first read your PEP I
> had a hard time figuring out the purpose of the cls object.  It wasn't
> until I got to the very end where I saw your example classes that I
> realized what it is: it represents a specific warning condition or a
> group of related warning conditions.

It sort of evolved into an exception-like mechanism in that the class is
instantiated with arguments just as exception classes are.

> >         if action=="error":
> >             *** existing assertion code ***
> 
> That's just
> 
>               raise AssertionError, message
> 
> Right?

Well I'd like the traceback to emanate from the caller's position not
the warning handler's position. Python doesn't really have a way to say
that simply. This may well be implemented in C so it might not matter.

> Suggestion: have separate warning and error handlers, so that if I
> want to override these branches of the if statement I don't have to
> repeat the entire handler.

Good idea.

> I see no definition of sys.disabled_warnings.  Instead of
> sys.disabled_warnings you meant sys.suppress, right?

Right.

> >               return "suppress"
> >           for base in cls.__bases__:
> >               rc = self.get_user_request(base)
> >               if rc:
> >                   return rc
> >           else:
> >               return None
> 
> This approach (searching for the class name or the name of one of its
> base classes in a list) doesn't look very object-oriented.  It would
> make more sense to store the desired return value as a class or
> instance attribute.  The default warning action could be stored on the
> base class.

The way I came to this odd structure is I wanted most subclasses to be
just no-ops as many exceptions are:

class ActiveStateLintWarning(Warning): pass

class CodeLooksLikePerl(ActiveStateLintWarning): pass

class TooMuchOnOneLine(CodeLooksLikePerl): pass

So what __init__ could I write that would look up -wTooMuchOnOneLine and
then if it failed that, look up -wCodeLooksLikePerl and so forth?

It gets pretty mind-bending because you sort of want one method to call
another and yet you want it to be the *same* inherited method (i.e.
don't have to code it each time). E.g. you want to run the *same method*
at each level of the hierarchy. So I just do that directly.

> >     -w XXX => sys.warnings.append("XXX")
> >     -e XXX => sys.errors.append("XXX")
> >     -no-w XXX => sys.suppress.append("XXX")
> >     -wall => sys.default_warning_action => "warn"
> >     -eall => sys.default_warning_action => "error"
> >     -no-wall => sys.default_warning_action => "suppress"
> 
> Python doesn't support long options (I don't *like* long options so I
> doubt that this is a good occasion to start lobbying for them :-).  We
> can come up with different options though.

Are these long options? Or just overloaded behavior on -w, -e, -n .
Think of "all" as the base class for warnings or something.

> >     As per the code above, errors take precedence over warnings and
> >     warnings over suppressions unless a particular assertion class
> >     specifies otherwise.
> 
> I would use a different precedence scheme: a more specific filter
> takes precedence over a more general filter.  So -eall -wdubious would
> mean that "dubious" class warnings are warnings but all others are
> errors, and -wall -edubious would mean the opposite.

Good idea. I think the code may work that way already. :)

 Paul Prescod


From guido@python.org  Mon Nov  6 22:16:29 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 17:16:29 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:36:49 EST."
 <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com>
Message-ID: <200011062216.RAA09669@cj20424-a.reston1.va.home.com>

> [Guido]
> > 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> > Algol-60 did this.  Fortran does this.  And rational numbers are less
> > innovative?

[Tim]
> Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
> new rules, int div always truncates).

I stand corrected -- the idea is only 40 years old, not 44. :-)

> So far as innovation goes, no choice on the table so far is innovative
> (neither mathematically nor in programming languages), so there's no basis
> for choosing there.
> 
> Guido, *why* did ABC use rationals by default?  Was that driven by usability
> studies?

I assume that the use of rationals for exact numbers was driven by
usability studies -- like us, the ABC designers were tired of
explaining the vagaries of floating point to novices.

I remember that I pushed for using rationals for 1E1000 and 1E-1000,
probably out of a mistaken sense of consistency.  I don't think I'm
responsible for 1.0 being exact -- in "The B Programmer's Handbook"
(CWI, 1985) 1.0 is exact and 1E10 is approximate.  In "The ABC
Progammer's Handbook (Prentice Hall, 1990) these are all exact.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@loewis.home.cs.tu-berlin.de  Tue Nov  7 00:49:34 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 01:49:34 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070049.BAA01485@loewis.home.cs.tu-berlin.de>

> Is there some API for it in C?

In C99, you have access to the floating-point environment:

               #include <fenv.h>
               /* ... */
               {
                       #pragma STDC FENV_ACCESS ON
                       int set_excepts;
                       feclearexcept(FE_INEXACT | FE_OVERFLOW);
                       // maybe raise exceptions
                       set_excepts = fetestexcept(FE_INEXACT | FE_OVERFLOW);
                       if (set_excepts & FE_INEXACT) f();
                       if (set_excepts & FE_OVERFLOW) g();
                       /* ... */
               }

It defines the following symbolic exception constants:

               FE_DIVBYZERO
               FE_INEXACT
               FE_INVALID
               FE_OVERFLOW
               FE_UNDERFLOW

Few compilers support that, though.

Regards,
Martin


From greg@cosc.canterbury.ac.nz  Tue Nov  7 01:06:04 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 14:06:04 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
Message-ID: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>

> It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.

It could be non-reserved, since a div b is currently
a syntax error.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From martin@loewis.home.cs.tu-berlin.de  Tue Nov  7 01:13:04 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 02:13:04 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070113.CAA01767@loewis.home.cs.tu-berlin.de>

> I seriously wonder whether that was *the* problem with ABC: not only
> was 1.0 treated as an exact rational in ABC, so was 6.02e23 and
> 3.14159e-314 etc.  At least for me, this caused rationals to get
> used in many places I *intended* to use floats.  I assume many
> others got burned by this too, as I'd say it's impossible for users
> coming from other languages not to see 6.02e23 etc as float
> literals.

There seems to be a long tradition in Python of annotating literals to
get them interpreted in a different way; I think it would be
reasonable to tell apart floating point literals and rational literals
(with a power-of-ten denominator).

Specifically, the "scientific notation" could be used: 1.1 would be
exactly the same as 11/10, 1.1e0 would be binary floating point, and
only approximately equal to 11/10.

Regards,
Martin


From guido@python.org  Tue Nov  7 03:07:49 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 22:07:49 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:01:27 PST."
 <3A072A37.C2CAFED1@activestate.com>
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>
 <3A072A37.C2CAFED1@activestate.com>
Message-ID: <200011070307.WAA10555@cj20424-a.reston1.va.home.com>

[Guido]
> > Warnings are more a human engineering issue than a technical issue!
> > That's also why I am emphasizing a C API -- I want to make it real
> > easy to ussue quality warnings in the runtime.  It's also why I
> > specify a rich (probably too rich!) filtering mechanism.

[Paul]
> Let me put out the strawman proposal that "grep" is a nicely orthogonal
> filtering mechanism. <duck!>
> 
> I guess my thinking is that you turn off warnings at the source. If you
> get a warning about __del__ exceptions then you put in a try/except. If
> you get a warning about an unused variable then you assign it to itself.
> If you get a warning about integer division then you should pass in a
> float and so forth.

I'm thinking of a different situation, where the user isn't very Unix
or Python sophisticated or the code generating warnings isn't easily
accessible for editing, or the fix isn't obvious (which I expect to be
often the case for __del__ and int division warnings).

Just as with exception handlers, where unqualified except clauses are
bad because of the risk that they mask real errors, I want to avoid
the likelihood that end users (those who can't or shouldn't fix the
errors) have to turn off all warnings.

> > > Syntax
> > >
> > >     assert >> cls, test[[[, arg], arg]...]
> > 
> > I have several problems with this.  First of all, using "assert" means
> > that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> > think that the decision to disable all warnings should be independent
> > from the decision to "optimize".  
> 
> Arguable. I see it as "release" and "debug" configurations. python and
> python_d.
> 
> > Second, you're hypergeneralizing the
> > extended print syntax.  Just because I think it's okay to add >>file
> > to the print syntax doesn't mean that it's now okay to add >>object
> > syntax to all statements!
> 
> Well getting new syntax into Python is really, really hard so we've got
> to squeeze out as much value out of what we have as possible. But
> anyhow, assertions are not allowed to 

(Did you forget to complete a sentence there?)

I say there's no need for new syntax, this can just be a function.

> You and I agree that there is an important sociological dimension to
> this. We can't require:
> 
>   import warnings
> 
>   warnings.warn("foo", "bar")
> 
> I prefer:
> 
>   warn Foo, "Bar"
> 
> just like:
> 
>   raise Foo, "Bar"

Hm.  Raise was made a statement because (1) Modula-3 did so, and (2)
it is a flow control modifier.  You haven't made a good case for why
warn() can't be a function.

> > I also don't see what warnings have to do with assertions.  Assertions
> > are a mechanism to check for error conditions.  What happens if the
> > error is detected is of less importance -- it could raise an exception
> > (Python), issue a fatal error (C), or do nothing (in -O mode).
> 
> Warnings are issued when an error or dubious construct is detected.
> Assertions are "fatal warnings". You agree that it is appropriate for
> some "warnings" to kill the app in some circumstances. Isn't it just a
> hop-skip-and-a-jump to say that warnings and errors are just points on
> the spectrum:
> 
> Report Once
> Report Always
> Report and Die

Possibly -- and then 'assert' is a poor choice of name for the
feature.  Objection denied.

> Python trained me to think of function calls and object constructions as
> being basically the same thing -- and keyword arguments and variable
> names being basically the same thing. etc.
> 
> > With warnings I believe the issue is not so much the detection of the
> > condition (for which a regular 'if' statement does just fine) but the
> > reporting.  Again, this is motivated by the fact that I expect that
> > flexible filtering is essential for a successful warning mechanism.
> 
> I don't see why assertions need special syntax but warnings do not! I
> would have been happy for a "warn" keyword with assert-like syntax but I
> don't think I'll see that in my lifetime.

Indeed.  But binding arbitrary unrelated semantics to an existing
statement with a carefully chosen name is poor design too.  You might
as well propose print<<file as a new input statement: they're both I/O
related. :-)

> > This is just a matter of exposition, but when I first read your PEP I
> > had a hard time figuring out the purpose of the cls object.  It wasn't
> > until I got to the very end where I saw your example classes that I
> > realized what it is: it represents a specific warning condition or a
> > group of related warning conditions.
> 
> It sort of evolved into an exception-like mechanism in that the class is
> instantiated with arguments just as exception classes are.
> 
> > >         if action=="error":
> > >             *** existing assertion code ***
> > 
> > That's just
> > 
> >               raise AssertionError, message
> > 
> > Right?
> 
> Well I'd like the traceback to emanate from the caller's position not
> the warning handler's position. Python doesn't really have a way to say
> that simply. This may well be implemented in C so it might not matter.

OK.

> > Suggestion: have separate warning and error handlers, so that if I
> > want to override these branches of the if statement I don't have to
> > repeat the entire handler.
> 
> Good idea.
> 
> > I see no definition of sys.disabled_warnings.  Instead of
> > sys.disabled_warnings you meant sys.suppress, right?
> 
> Right.
> 
> > >               return "suppress"
> > >           for base in cls.__bases__:
> > >               rc = self.get_user_request(base)
> > >               if rc:
> > >                   return rc
> > >           else:
> > >               return None
> > 
> > This approach (searching for the class name or the name of one of its
> > base classes in a list) doesn't look very object-oriented.  It would
> > make more sense to store the desired return value as a class or
> > instance attribute.  The default warning action could be stored on the
> > base class.
> 
> The way I came to this odd structure is I wanted most subclasses to be
> just no-ops as many exceptions are:
> 
> class ActiveStateLintWarning(Warning): pass
> 
> class CodeLooksLikePerl(ActiveStateLintWarning): pass
> 
> class TooMuchOnOneLine(CodeLooksLikePerl): pass

The idea of using a class hierarchy to classify warnings is definitely
a good one.

> So what __init__ could I write that would look up -wTooMuchOnOneLine and
> then if it failed that, look up -wCodeLooksLikePerl and so forth?

Yeah, it's not clear what to do.  You would like the -w option to poke
a value in a class variable, but the problem there is that it doesn't
know in which module the class is defined.  (On the other hand, if
that problem could be solved, it would be the preferred solution --
since it would solve the problem of typos in the -w argument neatly.)

> It gets pretty mind-bending because you sort of want one method to call
> another and yet you want it to be the *same* inherited method (i.e.
> don't have to code it each time). E.g. you want to run the *same method*
> at each level of the hierarchy. So I just do that directly.
> 
> > >     -w XXX => sys.warnings.append("XXX")
> > >     -e XXX => sys.errors.append("XXX")
> > >     -no-w XXX => sys.suppress.append("XXX")
> > >     -wall => sys.default_warning_action => "warn"
> > >     -eall => sys.default_warning_action => "error"
> > >     -no-wall => sys.default_warning_action => "suppress"
> > 
> > Python doesn't support long options (I don't *like* long options so I
> > doubt that this is a good occasion to start lobbying for them :-).  We
> > can come up with different options though.
> 
> Are these long options? Or just overloaded behavior on -w, -e, -n .
> Think of "all" as the base class for warnings or something.

Yes: if you change -no-w into -n, they can all be short options.  Note
that no matter what syntax we choose, we'll always be deviants
compared to Perl and GCC: those require a -w or -W option to *enable*
warnings.  (I found some great quotes in the Perl man page:

    Whenever you get mysterious behavior, try the -w switch!!!  Whenever
    you don't get mysterious behavior, try using -w anyway.

    The -w switch produces some lovely diagnostics.

    Did we mention that you should definitely consider using the -w
    switch?

    Bugs
        The -w switch is not mandatory.
)

> > >     As per the code above, errors take precedence over warnings and
> > >     warnings over suppressions unless a particular assertion class
> > >     specifies otherwise.
> > 
> > I would use a different precedence scheme: a more specific filter
> > takes precedence over a more general filter.  So -eall -wdubious would
> > mean that "dubious" class warnings are warnings but all others are
> > errors, and -wall -edubious would mean the opposite.
> 
> Good idea. I think the code may work that way already. :)

Indeed it does. :-)  Then what did you mean by your remark about the
precedence?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Tue Nov  7 04:03:59 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:03:59 +1300 (NZDT)
Subject: [Python-Dev] Warnings PEP
In-Reply-To: <3A072A37.C2CAFED1@activestate.com>
Message-ID: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>

Paul Prescod <paulp@ActiveState.com>:

> Assertions are "fatal warnings".

No, the failure of an assertion *causes* a fatal warning.
Assertions themselves are tests, not warnings.

I agree with Guido -- "assert" is the wrong verb, "warn"
is the right verb.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fdrake@users.sourceforge.net  Tue Nov  7 04:05:34 2000
From: fdrake@users.sourceforge.net (Fred L. Drake)
Date: Mon, 6 Nov 2000 20:05:34 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011070405.UAA32052@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/


From py-dev@zadka.site.co.il  Tue Nov  7 12:24:22 2000
From: py-dev@zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 14:24:22 +0200
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com>
 of "Mon, 06 Nov 2000 21:01:40 +0100." <3A070E24.B07FA7D7@lemburg.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>  <3A070E24.B07FA7D7@lemburg.com>
Message-ID: <E13t7ne-0005wj-00@darjeeling.zadka.site.co.il>

[MAL]
> Take e.g. dictionaries: you could easily add a new dictionary
> type which uses case-insensitive string keys by extending the
> existing dictionary type.
> 
> The new type would reuse most of the slots of the original
> type and only replace the ones needed for lookup with the
> new logic for case-insensitivity.
> 
> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object. The
> underlying type objects would be different though (and also
> the type object address which is currently used to identify
> a builtin type).

This kind of thing will also help a numeric system like PEP-0228
be implemented more easily.
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From greg@cosc.canterbury.ac.nz  Tue Nov  7 04:19:46 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:19:46 +1300 (NZDT)
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A070E24.B07FA7D7@lemburg.com>
Message-ID: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal@lemburg.com>:

> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object.

Hang on a minute. What sort of assumptions is the
interpreter going to be making based on the fact that
the type id is PyDict_TypeID? Can it be sure that this
new case-insensitive dictionary doesn't break them
somehow?

In other words, what does this new type_id thing
actually *mean*?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From paul@prescod.net  Tue Nov  7 05:57:40 2000
From: paul@prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 21:57:40 -0800
Subject: [Python-Dev] Warnings PEP
References: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>
Message-ID: <3A0799D4.86FC8148@prescod.net>

Greg Ewing wrote:
> 
> Paul Prescod <paulp@ActiveState.com>:
> 
> > Assertions are "fatal warnings".
> 
> No, the failure of an assertion *causes* a fatal warning.
> Assertions themselves are tests, not warnings.

Fine, some assertions are tests for fatal errors and some assertions are
tests for non-fatal mistakes.
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov  7 06:37:01 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:37:01 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <87ofzs39lo.fsf@indy.progeny.com>
Message-ID: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>

On 6 Nov 2000, Eric Gillespie, Jr. wrote:

> There has been a problem in Debian's python-gdk-imlib package for
> quite some time: Transparent PNGs do not display properly (see
> attached example script). According to upstream
> (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> the proper solution is either to have Python use RTLD_GLOBAL in
> dlopen calls when loading extension modules,

Or, possible, using dlmodule to dlopen things ourselves rather then
using Python's import facilities for that.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov  7 06:41:56 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:41:56 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011070841110.13407-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> 
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

Oooops....I didn't mean to send it here (well, I did, then I change
my mind but forgot to tell my e-mail program about that)

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From paul@prescod.net  Tue Nov  7 06:54:14 2000
From: paul@prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 22:54:14 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
 <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A07A716.BDF9658B@prescod.net>

Guido van Rossum wrote:
> 
> Before I fall asleep let me write up my ideas about the warning
> framework.
> 
> Requirements:
> 
> - A C-level API that lets C code issue a warning with a single call
>   taking one or two arguments, e.g. Py_Warning(level, message).  (The
>   'level' argument is an example only; I'm not sure what if any we
>   need.)

I like the C-level API.

I think the "level" argument should be some sort of characterization,
not a number or enumeration. I think of it as being like an exception --
just another Python object.

Or it could be just a string. The main point is that filtering on a
number or enum is not flexible enough.

> - An equivalent Python level API, e.g. sys.warning(level, message).

I would prefer something easier to spell and with more of a central "you
should use this alot" feeling.

> Possible implementation:
> 
> - Each module can has a dictionary __warnings__ in its global
>   __dict__, which records the state of warnings.  It is created as an
>   emprt dict if it doesn't exist when it is needed.  The keys are
>   (message, linenumber) tuples (the module or file is implicit through
>   the use of the module's __dict__).  The value is None if no more
>   action is needed for this particular warning and location.  Some
>   other values may indicate the options "always print warning" (1?)
>   and "raise an exception" (-1?).

The problem with filtering based on line number is that it can't really
be done in a static manner because it is too fragile to code changes. In
my proposal every warning was assigned a "type" which would be the key
for filtering. A string would also be fine.

In general, I'm uncomfortable that I don't understand the requirements
enough. Are warnings something that the average Python programmer sees
rarely and then immediately goes in to fix the code so they don't see it
anymore (as compiler warnings are often handled)? Or do we expect most
Python programs to issue hundreds of warnings unless they are filtered.
Is filtering something you do constantly or as a partial workaround for
half-broken code that you can't fix right now?
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov  7 06:58:40 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:58:40 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <3A07A716.BDF9658B@prescod.net>
Message-ID: <Pine.GSO.4.10.10011070849580.13407-100000@sundial>

On Mon, 6 Nov 2000, Paul Prescod wrote:

> I like the C-level API.
> 
> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.

These are completely different things -- both should be there. An
exception doesn't need a level -- it's the highest level possible, saying
"get out of here, fast!". Warnings need both a level and characterization.
Having it a Python class is a nice touch, and keeps it consistent with
the way exceptions use classes for characterization.

> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

There are two main sources to copy from here:

gcc: You have -Wall, -Wadd-something, etc. Any warning you do see you
     either fix, or surround with a pragma so you don't see this.
     You also have -Werror to turn all warnings into errors.
Perl: -w gives runtime warnings for things any saner language would
      raise exceptions for. "dereferencing NULL", accessing non-existing
      elements in an array, etc. Warnings are serious bugs, and you must
      always use them.
Perl: "use strict" and friends: die because of some otherwise perfectly
      legal Perl if it's not declared properly.

I'd go for a more gcc-like approach: if you see a warning, you should
either 1. Silence it or 2. Fix it.
Silencing warnings is a serious issue: sometimes the programmer does
no better then the interpreter, and should have the ability to silence
any warning permanently -- otherwise he'll work with -silence_all, and
the benefit is lost.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From tim_one@email.msn.com  Tue Nov  7 08:38:35 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 7 Nov 2000 03:38:35 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBCHPAA.tim_one@email.msn.com>

[Guido]
> ...
> Note: it is open for debate whether the result of x/y for integer (or
> long integer) arguments should yield an integer (or long integer) in
> those cases where the result *is* representable as such (e.g. 4/2).
> It is possible that the numeric tower will render this problem moot --
> but that depends on what happens to Moshe's PEP 228, and that's a much
> longer story.

Note that for long ints i and j, i/j may (& easily so) be too large to
represent as a native double.  Rationals (of course) have no problem with
that.  It would certainly be curious if i*j didn't overflow but i/j did.

Just an observation -- I'm a fan of unintended consequences <wink>.




From mal@lemburg.com  Tue Nov  7 09:17:56 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 10:17:56 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>
Message-ID: <3A07C8C4.1F14985F@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com>:
> 
> > Then it sets the type id to PyDict_TypeID and Python will
> > use it as if it were an original dictionary object.
> 
> Hang on a minute. What sort of assumptions is the
> interpreter going to be making based on the fact that
> the type id is PyDict_TypeID? Can it be sure that this
> new case-insensitive dictionary doesn't break them
> somehow?
> 
> In other words, what does this new type_id thing
> actually *mean*?

For the interpreter it means that it can assume the type
interface to be binary compatible to the "original"
type, e.g. by setting the flag to say PyDict_TypeID
the type assures that all PyDict_*() APIs will work
on the type -- basically the same thing as PyDict_Check()
does now except that the type object needn't be the same
anymore.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fredrik@pythonware.com  Tue Nov  7 09:36:42 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 10:36:42 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
References: <200011070911.BAA13249@slayer.i.sourceforge.net>
Message-ID: <01e301c0489e$3e349540$0900a8c0@SPIFF>

Moshe Zadka wrote:
> Modified Files:
> pep-0216.txt 
> Log Message:
> Added structured text consensus.

when/where was this discussed?

</F>



From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov  7 09:49:54 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:49:54 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
In-Reply-To: <01e301c0489e$3e349540$0900a8c0@SPIFF>
Message-ID: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>

On Tue, 7 Nov 2000, Fredrik Lundh wrote:

> Moshe Zadka wrote:
> > Modified Files:
> > pep-0216.txt 
> > Log Message:
> > Added structured text consensus.
> 
> when/where was this discussed?

ummmm....doc-sig, where all things documentation-related are discussed?

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov  7 09:58:41 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:58:41 +0200 (IST)
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:
 python/nondist/peps pep-0216.txt,1.3,1.4)
In-Reply-To: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> ummmm....doc-sig, where all things documentation-related are discussed?

Just to clarify my answer: I'm happy to receive comments/complaints
by private e-mail, or you can share thm with all of doc-sig. I prefer
to be CCed on doc-sig e-mail, but if not, I'll read it in the doc-sig.

Python-Dev does not seem the place for documentation-related discussion,
since that's exactly what doc-sig is for. It's quite low-volume, so
please join, or browse the archives.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From fredrik@pythonware.com  Tue Nov  7 10:59:38 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 11:59:38 +0100
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:python/nondist/peps pep-0216.txt,1.3,1.4)
References: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>
Message-ID: <027f01c048a9$d324efa0$0900a8c0@SPIFF>

moshe wrote:
> Python-Dev does not seem the place for documentation-related discussion,
> since that's exactly what doc-sig is for. It's quite low-volume, so
> please join, or browse the archives.

oh, I thought I was subscribed.  guess I wasn't.

I'll check the archives.

</F>



From guido@python.org  Tue Nov  7 12:55:26 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 07:55:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Mon, 06 Nov 2000 22:54:14 PST."
 <3A07A716.BDF9658B@prescod.net>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
 <3A07A716.BDF9658B@prescod.net>
Message-ID: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>

> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.
> 
> Or it could be just a string. The main point is that filtering on a
> number or enum is not flexible enough.

OK, let's make this a class then.  Just keep exceptions out of it
-- this is a separate, disjoint set of classes.  Let's call this
"warning category".  There will be standard categories and user code
can add categories.

> > - An equivalent Python level API, e.g. sys.warning(level, message).
> 
> I would prefer something easier to spell and with more of a central "you
> should use this alot" feeling.

OK, let's make this a built-in: warning(category, message).

> > Possible implementation:
> > 
> > - Each module can has a dictionary __warnings__ in its global
> >   __dict__, which records the state of warnings.  It is created as an
> >   emprt dict if it doesn't exist when it is needed.  The keys are
> >   (message, linenumber) tuples (the module or file is implicit through
> >   the use of the module's __dict__).  The value is None if no more
> >   action is needed for this particular warning and location.  Some
> >   other values may indicate the options "always print warning" (1?)
> >   and "raise an exception" (-1?).
> 
> The problem with filtering based on line number is that it can't really
> be done in a static manner because it is too fragile to code changes. In
> my proposal every warning was assigned a "type" which would be the key
> for filtering. A string would also be fine.
> 
> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

All of the above.

If I'm a programmer maintaining a piece of code, of course I turn all
warnings into errors and fix them as they occur.

But if I'm using someone else's code that happens to generate
warnings, it's better to disable warnings in that code until its
author has released a fixed version.  I want to be able to be very
specific, so that turning off warnings in 3rd party code doesn't
disable them in my own code.  If the 3rd party code generates a single
warning during normal use at a single location (e.g. if there's one
unconverted integer divide somehwere) then it's best to turn it off
just at that location, so that when I feed it other data which may
trigger other warnings elsewhere I will still get the benefit of the
warnings -- which *may* mean there's something wrong with my data, not
with the 3rd party code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Tue Nov  7 13:17:02 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 14:17:02 +0100
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Tue, Nov 07, 2000 at 02:06:04PM +1300
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
Message-ID: <20001107141702.H27208@xs4all.nl>

On Tue, Nov 07, 2000 at 02:06:04PM +1300, Greg Ewing wrote:
> > It's a new keyword though, which has a much
> > higher threshold for acceptance than a new two-character operator
> > symbol.

> It could be non-reserved, since a div b is currently
> a syntax error.

Except for the fact our current parser can't handle the 'a div b' syntax
without making 'div' a reserved word, which also makes 'x.div', 'class div:'
and 'def div():' invalid syntax. It might be work around-able, but .... :P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From jim@interet.com  Tue Nov  7 13:28:16 2000
From: jim@interet.com (James C. Ahlstrom)
Date: Tue, 07 Nov 2000 08:28:16 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com>
Message-ID: <3A080370.30C36339@interet.com>

"M.-A. Lemburg" wrote:

> Now I get this error after working in interactive Python
> mode with zipfile:
> 
> Exception exceptions.AttributeError:
> "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

Reading the code, I don't see how this could have happened
unless __init__ has already raised an exception.
 
> I would like a method .copy(self, name, output) which
> reads the file name from the ZIP archive and writes it directly to
> the file-like object output. This should copy the file in chunks
> of say 64kB in order to reduce memory load.

This is only a few lines of Python, and I generally omit
any methods which are not absoultely necessary.  Does
anyone else think this should be added?

JimA


From guido@python.org  Tue Nov  7 13:33:46 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 08:33:46 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Tue, 07 Nov 2000 14:17:02 +0100."
 <20001107141702.H27208@xs4all.nl>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
 <20001107141702.H27208@xs4all.nl>
Message-ID: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>

> > > It's a new keyword though, which has a much
> > > higher threshold for acceptance than a new two-character operator
> > > symbol.
> 
> > It could be non-reserved, since a div b is currently
> > a syntax error.
> 
> Except for the fact our current parser can't handle the 'a div b' syntax
> without making 'div' a reserved word, which also makes 'x.div', 'class div:'
> and 'def div():' invalid syntax. It might be work around-able, but .... :P

No, we *could* use the 'import as' trick: define the syntax as

term: factor (('*'|'/'|'%'|NAME) factor)*

and add a check that NAME is "div" in the compiler.

But I don't know how comfy I am with a proliferation of hacks like
this -- and it's likely to cause more confusing error messages.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Tue Nov  7 14:46:15 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 15:46:15 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com> <3A080370.30C36339@interet.com>
Message-ID: <3A0815B7.BD0D9C4C@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> > Now I get this error after working in interactive Python
> > mode with zipfile:
> >
> > Exception exceptions.AttributeError:
> > "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored
> 
> Reading the code, I don't see how this could have happened
> unless __init__ has already raised an exception.

Probably has something to do with GC and cleaning up instances --
don't know. The error only shows up sometimes...

Instead of seeing the __del__ exception I now get the previous error
again:
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Nevermind... my setup must be broken in more way than I have
time to figure out now :-(
 
> > I would like a method .copy(self, name, output) which
> > reads the file name from the ZIP archive and writes it directly to
> > the file-like object output. This should copy the file in chunks
> > of say 64kB in order to reduce memory load.
> 
> This is only a few lines of Python, and I generally omit
> any methods which are not absoultely necessary.  Does
> anyone else think this should be added?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From pf@artcom-gmbh.de  Tue Nov  7 14:59:01 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Tue, 7 Nov 2000 15:59:01 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 7, 2000  8:33:46 am"
Message-ID: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum:
> No, we *could* use the 'import as' trick: define the syntax as
> 
> term: factor (('*'|'/'|'%'|NAME) factor)*
> 
> and add a check that NAME is "div" in the compiler.
> 
> But I don't know how comfy I am with a proliferation of hacks like
> this -- and it's likely to cause more confusing error messages.

In Modula-2 it is forbidden to declare a variable or procedure called
'DIV' or 'MOD', since both were reserved words in this language from
the very beginning.  But in Python the situation is different and
people might have used 'div' as an identifier.  So unless Guido is
able to fix this using the famous time machine ...sigh... for the
sake of backward compatibility using this "hack" seems to be the best
available choice.

I believe confusing error messages can be avoided and I see no 
"proliferation of hacks" in this two attempts to avoid defining
new keywords.  Keeping backward compatibility always had its price.
But it is often a price worth to pay.

Just my $0.02, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From gward@mems-exchange.org  Tue Nov  7 15:22:51 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Tue, 7 Nov 2000 10:22:51 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 07, 2000 at 07:55:26AM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com> <3A07A716.BDF9658B@prescod.net> <200011071255.HAA13355@cj20424-a.reston1.va.home.com>
Message-ID: <20001107102251.A15674@ludwig.cnri.reston.va.us>

On 07 November 2000, Guido van Rossum said:
> > Or it could be just a string. The main point is that filtering on a
> > number or enum is not flexible enough.
> 
> OK, let's make this a class then.  Just keep exceptions out of it
> -- this is a separate, disjoint set of classes.  Let's call this
> "warning category".  There will be standard categories and user code
> can add categories.

This sounds right -- I was going to suggest "warning class" instead of
"level", but "category" might be a better word.  My main rationale was
filtering: show me "integer divide" problems, but don't bother me with
"function argument not used".  (Hmm, those two sound more like specific
warnings rather than warning categories.  Probably the categories there
would be "arithmetic" and "dataflow".)

> > I would prefer something easier to spell and with more of a central "you
> > should use this alot" feeling.
> 
> OK, let's make this a built-in: warning(category, message).

Minor spelling nit: I would call it 'warn()' (or 'sys.warn()', or
'Py_Warn()', etc.) since that's a verb.

More importantly: if 'warn(ing)' is meant to be used mainly for
compiler-style warnings -- you're using this language or library feature
inappropriately -- then it should be left in sys.  But if it's meant to
also be used for printing some message to stderr (like Perl's 'warn'),
then there's a good case for making it a builtin.

Almost every Python script I write features

    def warn (msg):
        sys.stderr.write("warning: " + msg + "\n")

That might be a clue that something (albeit a tiny thing) is missing
from the language.  ;-)

        Greg


From fdrake@acm.org  Tue Nov  7 15:30:00 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 10:30:00 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
 <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
 <20001107141702.H27208@xs4all.nl>
 <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
Message-ID: <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > But I don't know how comfy I am with a proliferation of hacks like
 > this -- and it's likely to cause more confusing error messages.

  If "div" is it, I'd rather see it made a keyword and a warning
published to the community soon so that people have a chance to check
their code and either make it compatible with the change or scream
ahead of time.
  A tool to help them out wouldn't hurt, either, and that could be
written before any actual changes are made or even final decisions are
made -- it could search everything on sys.path and report on uses that
would be affected by each candidate change.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From guido@python.org  Tue Nov  7 16:14:21 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 11:14:21 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."
 <3A06BEF4.95B773BD@tismer.com>
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>
 <3A06BEF4.95B773BD@tismer.com>
Message-ID: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>

I've thought about it a little more, and drawn some pictures in my
head.

I still have to disagree with Christian when he says:

> Making Python completely coroutine aware, without
> tricking the C stack, is 90 percent of the problem.
> But after walking that far, there is no reason
> to leave the other 10 percent alone.

Without continuations, but with microthreads (uthreads) or coroutines,
each (Python) stack frame can simply be "paused" at a specific point
and continued later.  The semantics here are completely clear (except
perhaps for end cases such as unhandled exceptions and intervening C
stack frames).

With continuations, you have to decide how much state to save for a
future continuation.  It would seem easy enough: save all state kept
in the frame except for the local variables.  But now consider this:
the "evaluation stack" contained in each frame could easily be
replaced by a bunch of temporary variables, if we had a slightly
different instruction set (3-address opcodes instead of stack-based
opcodes).  Then would we save those temporary variables or not?  it
can make a difference!  Since the "save continuation" operation is a
function call, you can easily save a continuation while there are some
items on the value stack.  I believe the current implementation saves
these so they are restored when you jump to the continuation.  But if
the compiler were to use temporary variables instead of the evaluation
stack, they might not have been restored!

Here's another example.  Suppose you set up a for loop.  After three
iterations through the loop you save a continuation.  Then you finish
hree more iterations.  Then you return to the saved continuation.
Where does the loop continue: at 3 or at 6 iterations?  Try to answer
this without trying it.  My guess: it gets restarted at 3 iterations,
because the loop index is saved on the value stack.  If you rewrite
this using a while loop, however, it would get restarted at 6
iterations, because then your loop index is an unsaved local variable.
Ditto if you changed the bytecode compiler so for loops use an
anonymous local variable instead of an entry on the evaluation
stack.

This semantic imprecision is one of the reasons why I don't like the
concept of continuations.  (I've been told that the exact semantics of
continuations in Scheme differ greatly between Scheme implementations.)

Now let's look at Jython.  In Jython, we can simulate "paused frames"
well enough by using real Java threads.  However full continuations
would require modifications to the JVM -- which is unacceptable to a
language boasting "100% Pure Java".  Another reason against allowing
continuations.

So, all in all, I don't think of continuations as "the last 10% that
we might as well add to finish the job."  I see it as an ill-specified
hypergeneralization.

What *do* I want to see in a stackless PEP?

Not surprisingly, I'd like to see generators, coroutines, and
uthreads.  These all share a mechanism that pauses one frame and
resumes another.  I propose to make the concept of uthreads
fundamental -- this will simplify the emulation in Jython.


A strawman proposal:

The uthread module provides the new functionality at the lowest level.
Uthread objects represent microthreads.  An uthread has a chain of
stack frames linked by back pointers just like a regular thread.
Pause/resume operations are methods on uthread objects.  Pause/resume
operations do not address specific frames but specific uthreads;
within an uthread the normal call/return mechanisms can be used, and
only the top frame in the uthread's stack of call frames can be
paused/resumed (the ones below it are paused implicitly by the call to
the next frame, and resumed when that call returns).

- u = uthread.new(func) creates a new uthread object, u.  The new
  uthread is poised to call func() but doesn't execute yet.

- u = uthread.current() returns the uthread object for the current
  frame.

- u.yield() pauses the current uthread and resume the uthread u where
  it was paused.  The current uthread is resumed when some other
  uthread calls its yield() method.  Calling uthread.current().yield()
  is a no-op.

- When func() returns, the uthread that was executing it ceases to be
  runnable.  The uthread that most recently yielded to it is resumed,
  unless that is no longer runnable, in which case the uthread that
  most recently yielded to *it* is resumed, and so on until a runnable
  uthread is found or until no runnable uthreads are left, in which
  case the program terminates.  (XXX I need a proof here that this
  works.)

- When func() raises an unhandled exception, the exception gets
  propagated using the same rules as when it returns, and similarly
  its uthread ceases to be runnable.

- u.kill(exc) causes the yield() call that paused u to raise the
  exception exc.  (This can be caught in a try/except of course.)

- Calling u.yield() or u.kill() for a non-runnable uthread is an error
  and raises an exception.

I think this API should enough to implement Gordon's SelectDispatcher
code.  In general, it's easy to create a scheduler uthread that
schedules other uthreads.


Open issues:

- I'm not sure that I got the start conditions right.  Should func() be
  be allowed to run until its first yield() when uthread.new(func) is
  called?

- I'm not sure that the rules for returning and raising exceptions
  from func() are the right ones.

- Should it be possible to pass a value to another uthread by passing
  an argument to u.yield(), which then gets returned by the resumed
  yield() call in that uthread?

- How do uthreads interact with real threads?  Uthreads are explicitly
  scheduled through yield() calls; real threads use preemptive
  scheduling.  I suppose we could create a new "main" uthread for each
  real thread.  But what if we yield() to an uthread that's already
  executing in another thread?  How is that error detected?


Please help!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From cgw@fnal.gov  Tue Nov  7 16:30:16 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 10:30:16 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
 <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14856.11800.444274.851825@buffalo.fnal.gov>

 > Guido van Rossum:
 > > No, we *could* use the 'import as' trick: define the syntax as
 > > 
 > > term: factor (('*'|'/'|'%'|NAME) factor)*
 > > 
 > > and add a check that NAME is "div" in the compiler.
 > > 
 > > But I don't know how comfy I am with a proliferation of hacks like
 > > this -- and it's likely to cause more confusing error messages.

And what is the compelling reason for going through all this instead
of just using the '//' symbol?  Because it might be confused for a C++
comment?  This is a weak argument AFAIAC.  Python is not C++ and
everybody knows that.

I think that making "div" an infix operator would be setting a
horrible precedent.  Currently, all infix operators "look like"
operators, i.e. they are non-alphabetic characters, and things that
look like words are either functions or reserved words.

There's already a "divmod" builtin which is a function, not an infix
operator.  I think it would be rather inconsistent to write, on the
one hand:

    divmod(10, 2)

and on the other:

    10 div 2


Long before the creation of python-dev, this issue had been discussed
numerous times on c.l.py, and the '//' operator was suggested several
times, and I don't think anybody ever had a problem with it...


From gvwilson@nevex.com  Tue Nov  7 16:41:25 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Tue, 7 Nov 2000 11:41:25 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>

>  > Guido van Rossum:
>  > > No, we *could* use the 'import as' trick: define the syntax as
>  > > term: factor (('*'|'/'|'%'|NAME) factor)*
>  > > and add a check that NAME is "div" in the compiler.

> Charles G. Waldman:
> And what is the compelling reason for going through all this instead
> of just using the '//' symbol?  Because it might be confused for a C++
> comment?  This is a weak argument AFAIAC.  Python is not C++ and
> everybody knows that.
> Long before the creation of python-dev, this issue had been discussed
> numerous times on c.l.py, and the '//' operator was suggested several
> times, and I don't think anybody ever had a problem with it...

Greg Wilson:
As someone who teaches Python, I'm strongly opposed to using '//' in the
same language as '/', purely on readability grounds:

1. Every C/C++ book includes a warning about "=" vs. "==", because it's
   a common hard-to-spot error.

2. What mark would you give a student who had variables IO and I0 in the
   same module?

Greg

p.s. I was very disappointed to discover that Ruby uses both '..' and '...'
One means "up to but not including", the other means "up to and including".
It would be interesting to estimate the number of programmer-hours this will
cost... :-)



From thomas@xs4all.net  Tue Nov  7 17:26:21 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 18:26:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <200011071544.HAA31147@slayer.i.sourceforge.net>; from gward@users.sourceforge.net on Tue, Nov 07, 2000 at 07:44:24AM -0800
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
Message-ID: <20001107182621.I27208@xs4all.nl>

On Tue, Nov 07, 2000 at 07:44:24AM -0800, Greg Ward wrote:

>   - when compiling with GCC on any platform, add "-fPIC" to OPT
>     (without this, "$(CC) -shared" dies horribly)

Sorry for the late remark (I did see your earlier message) but after reading
the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
for non-essential stuff: warnings, debug-info and optimizations. Removing
things from OPT shouldn't break anything, and neither should adding options
that fit in the categories above. (Barring broken compilers, of course.)

Instead, the -fPIC option should be added to CFLAGS, I think. The Python
autoconf setup is slightly less versatile than most, though, since it's
doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
without editing configure(.in) :P If noone else does it before me, I'll see
about fixing at least the -fPIC thing later, when I find some time ;P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From paulp@ActiveState.com  Tue Nov  7 17:29:10 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 09:29:10 -0800
Subject: [Python-Dev] Integer division transition
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
 <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
 <20001107141702.H27208@xs4all.nl>
 <200011071333.IAA13497@cj20424-a.reston1.va.home.com> <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
Message-ID: <3A083BE6.4A502210@activestate.com>

"Fred L. Drake, Jr." wrote:
> 
> ...
>   A tool to help them out wouldn't hurt, either, and that could be
> written before any actual changes are made or even final decisions are
> made -- it could search everything on sys.path and report on uses that
> would be affected by each candidate change.

I think that the standard Python compiler is the appropriate tool for
this sort of thing. Anything that can be caught "statically" might as
well be implemented right in the compiler (at least from the user's
point of view) rather than in a separate "deprecation nanny."

 Paul Prescod


From nas@arctrix.com  Tue Nov  7 11:45:18 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Tue, 7 Nov 2000 03:45:18 -0800
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>; from gvwilson@nevex.com on Tue, Nov 07, 2000 at 11:41:25AM -0500
References: <14856.11800.444274.851825@buffalo.fnal.gov> <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <20001107034518.A12431@glacier.fnational.com>

On Tue, Nov 07, 2000 at 11:41:25AM -0500, Greg Wilson wrote:
> As someone who teaches Python, I'm strongly opposed to using '//' in the
> same language as '/', purely on readability grounds:

How do you feel about div(x, y)?

  Neil


From cgw@fnal.gov  Tue Nov  7 18:50:18 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 12:50:18 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
References: <14856.11800.444274.851825@buffalo.fnal.gov>
 <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <14856.20202.174151.647981@buffalo.fnal.gov>

Greg Wilson writes:
 > 
 > 2. What mark would you give a student who had variables IO and I0 in the
 >    same module?
 > 

I think this is a bit of a stretch - IO and I0 look almost identical
typographically (depending on the font) whereas // and / look pretty
different.  It would be a better analogy to say "What mark would you
give a student who used variables X and XX in the same program".  And,
I wouldn't have a problem with that.

How about URL's?  '/' and '//' have different meanings there and I
don't think people have a big problem with this.

The other point - "=" vs "==" - is a bit harder to answer.  Both of
these symbols are used in Python, but not in the same context.

All-the-good-symbols-are-already-taken-ly y'rs,
					//C




From gstein@lyra.org  Tue Nov  7 18:49:12 2000
From: gstein@lyra.org (Greg Stein)
Date: Tue, 7 Nov 2000 10:49:12 -0800
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 07, 2000 at 08:37:01AM +0200
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <20001107104912.Q14054@lyra.org>

On Tue, Nov 07, 2000 at 08:37:01AM +0200, Moshe Zadka wrote:
> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

There was quite a long conversation [on python-dev] a while back (geez, a
year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
were pros and cons for both directions, and I believe some compatibility
issues.

You may be able to find the conversation, then figure out why Python chose
its current mechanism. Heck... maybe it should change :-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From akuchlin@mems-exchange.org  Tue Nov  7 19:11:06 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 7 Nov 2000 14:11:06 -0500
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <20001107104912.Q14054@lyra.org>; from gstein@lyra.org on Tue, Nov 07, 2000 at 10:49:12AM -0800
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial> <20001107104912.Q14054@lyra.org>
Message-ID: <20001107141106.A10897@kronos.cnri.reston.va.us>

On Tue, Nov 07, 2000 at 10:49:12AM -0800, Greg Stein wrote:
>There was quite a long conversation [on python-dev] a while back (geez, a
>year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
>were pros and cons for both directions, and I believe some compatibility
>issues.

kronos Python>cvs log importdl.c 
  ...
revision 2.47
date: 1998/05/18 13:42:45;  author: guido;  state: Exp;  lines: +4 -6
Remove use of RTLD_GLOBAL.
----------------------------                                                    ...
revision 2.41
date: 1997/12/02 20:43:18;  author: guido;  state: Exp;  lines: +7 -3
Add the flag RTLD_GLOBAL to the dlopen() options.
This exports symbols defined by the loaded extension to other
extensions (loaded later).
 
(I'm not quite sure about this but suppose it can't hurt...)
----------------------------

Adding RTLD_GLOBAL in one version, removing it in the next: a new
Python tradition!

--amk



From est@hyperreal.org  Tue Nov  7 19:04:33 2000
From: est@hyperreal.org (est@hyperreal.org)
Date: Tue, 7 Nov 2000 11:04:33 -0800 (PST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 7, 2000 11:14:21 am"
Message-ID: <20001107190433.26578.qmail@hyperreal.org>

Guido van Rossum discourseth:
> 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest level.

I really like this as a primitive appropriate for Python's evolution.

> - When func() returns, the uthread that was executing it ceases to be
>   runnable.  The uthread that most recently yielded to it is resumed,
>   unless that is no longer runnable, in which case the uthread that
>   most recently yielded to *it* is resumed, and so on until a runnable
>   uthread is found or until no runnable uthreads are left, in which
>   case the program terminates.  (XXX I need a proof here that this
>   works.)

I'd like it added that when a uthread chains to its yielder it drops
(i.e., nulls and decrefs) the reference to that yielder.  I want
uthreads in some of the same applications in which i disable gc for
real-time purposes, and I don't want circular structures of unrunnable
uthreads leaking my memory.

> - Calling u.yield() or u.kill() for a non-runnable uthread is an error
>   and raises an exception.

A runnable() predicate might be nice.

> - I'm not sure that I got the start conditions right.  Should func() be
>   be allowed to run until its first yield() when uthread.new(func) is
>   called?

+1 for no on this.

> - I'm not sure that the rules for returning and raising exceptions
>   from func() are the right ones.

I'm particularly unsure about the exception propagation.  It could
always be disabled by a universal exception handler in the uthread,
but I'm not sure it's even worth the implementation effort.

> - Should it be possible to pass a value to another uthread by passing
>   an argument to u.yield(), which then gets returned by the resumed
>   yield() call in that uthread?

Certainly! :)

As written, the strawman seems to require that the dread intervening C
stack frames are handled transparently (since it doesn't say anything
about them).  This seems pretty important to me.  An instance method
may well know that it should yield, yet not know that it's being
called as a callback from a class/type that's just been moved to C.

OTOH..not handling this transparently would increase my market value
as a Python programmer.  Handling it right might get me some unpaid
work implementing some of the low-level details for Linux.  Hmm! :D

Eric



From paulp@ActiveState.com  Tue Nov  7 20:33:19 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 12:33:19 -0800
Subject: [Python-Dev] Integer division transition
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
 <m13tADF-000Dm8C@artcom0.artcom-gmbh.de> <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <3A08670F.A4703F32@activestate.com>

Charles G Waldman wrote:
> 
> ...
> 
> 
> I think that making "div" an infix operator would be setting a
> horrible precedent.  

I think it would be a good precedent because it is a cleaner upgrade
path to things like

matrixdiv, matrixmul, ...

 Paul Prescod


From tismer@tismer.com  Tue Nov  7 19:54:14 2000
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 07 Nov 2000 21:54:14 +0200
Subject: [Python-Dev] Re: uthread strawman
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>
 <3A06BEF4.95B773BD@tismer.com> <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <3A085DE6.28593202@tismer.com>

Just answering/clarifying a few bits,
since I can't change your opinion about
continuations, anyway.

Guido van Rossum wrote:
> 
> I've thought about it a little more, and drawn some pictures in my
> head.

I have to agree with Guido when he says:

> I still have to disagree with Christian when he says:
> 
> > Making Python completely coroutine aware, without
> > tricking the C stack, is 90 percent of the problem.
> > But after walking that far, there is no reason
> > to leave the other 10 percent alone.

... since I meant implementability. Of course there are
other reasons gainst continuations. I just did it since they
were in reach.

> Without continuations, but with microthreads (uthreads) or coroutines,
> each (Python) stack frame can simply be "paused" at a specific point
> and continued later.  The semantics here are completely clear (except
> perhaps for end cases such as unhandled exceptions and intervening C
> stack frames).

I agree. But also with continuations, the situation is identical,
as long as you don't try anything else where continuations
would be needed.
Note that they will not need to be created when the mentioned
structures are implemented well. We don't have to implement them,
but providing support for them in the interpreter framework
is simple. (that's the 10% issue).

> With continuations, you have to decide how much state to save for a
> future continuation.  It would seem easy enough: save all state kept
> in the frame except for the local variables.  But now consider this:
> the "evaluation stack" contained in each frame could easily be
> replaced by a bunch of temporary variables, if we had a slightly
> different instruction set (3-address opcodes instead of stack-based
> opcodes).  Then would we save those temporary variables or not?  it
> can make a difference!  Since the "save continuation" operation is a
> function call, you can easily save a continuation while there are some
> items on the value stack.  I believe the current implementation saves
> these so they are restored when you jump to the continuation.  But if
> the compiler were to use temporary variables instead of the evaluation
> stack, they might not have been restored!

I would consider these temporary variables registers which must
be preserved. They are immutable objects as part of the immutable
continuation, treated as values. Stack or registers, this is part
of an expression evaluation. Temporary results must conceptually
be read-only, whatever way I implement this.

> Here's another example.  Suppose you set up a for loop.  After three
> iterations through the loop you save a continuation.  Then you finish
> hree more iterations.  Then you return to the saved continuation.
> Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> this without trying it.  My guess: it gets restarted at 3 iterations,
> because the loop index is saved on the value stack.  If you rewrite
> this using a while loop, however, it would get restarted at 6
> iterations, because then your loop index is an unsaved local variable.
> Ditto if you changed the bytecode compiler so for loops use an
> anonymous local variable instead of an entry on the evaluation
> stack.

Wrong guess!
Exactly for that reason I changed the loop code to put a mutable
loopcounter object on the stack.
The loop works perfectly.

> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.  (I've been told that the exact semantics of
> continuations in Scheme differ greatly between Scheme implementations.)

In a sense, you have continuations already, also with the restriction
to gen/co/uthread structures. The only difference is to treat a
frame as exactly one continuation and to disallow to have more
than one at any time.
This saves the decision about the ambiguities you mentioned.

I agree that going to this point and not further for the
Python core is a good idea.
A PEP doesn't need to name continuations at all.

On the other hand, I don't see a reason why this hsould mean
that Python *must not* support them. What I'd like to keep
is the possibility to still write such an extension module.

Enabling this for educational purposes is a great value
that comes at a cheap price and no impact for the core.

> Now let's look at Jython.  In Jython, we can simulate "paused frames"
> well enough by using real Java threads.  However full continuations
> would require modifications to the JVM -- which is unacceptable to a
> language boasting "100% Pure Java".  Another reason against allowing
> continuations.

Not quite true, after I heard of a paper that shows how
to implement continuations in Java, using threads.
But I'll come back to that when I have the paper.

> So, all in all, I don't think of continuations as "the last 10% that
> we might as well add to finish the job."  I see it as an ill-specified
> hypergeneralization.

Can we agree to not support them without forbidding them?

...
> A strawman proposal:

Ok, this looks all very well to me. More on that later.
One question: Why do you want an explicit u.yield() ?
Uthreads are scheduled automatically now, like real
threads. Do you see a major drawback in supporting
this, maybe as an option? Or do you see automatic
scheduling as an extra construct on top with a special
"scheduler" uthread?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From jack@oratrix.nl  Tue Nov  7 22:57:53 2000
From: jack@oratrix.nl (Jack Jansen)
Date: Tue, 07 Nov 2000 23:57:53 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: Message by "M.-A. Lemburg" <mal@lemburg.com> ,
 Tue, 07 Nov 2000 10:17:56 +0100 , <3A07C8C4.1F14985F@lemburg.com>
Message-ID: <20001107225758.B77651301D9@oratrix.oratrix.nl>

> > In other words, what does this new type_id thing
> > actually *mean*?
> 
> For the interpreter it means that it can assume the type
> interface to be binary compatible to the "original"
> type, e.g. by setting the flag to say PyDict_TypeID
> the type assures that all PyDict_*() APIs will work
> on the type -- basically the same thing as PyDict_Check()
> does now except that the type object needn't be the same
> anymore.

I would be _very_ happy if this single type_id could somehow be
replaced by an array, or a bitset.

I have a lot of types in MacPython that are acceptable to the APIs of
other types, a sort of poor-mans-inheritance scheme. For instance, all 
operating system calls that accept a MacOS WindowPtr will also happily 
accept a DialogPtr. Major magic is needed to get this to work
reasonably in Python, and the Python user can still accidentally mess
up the refcounting scheme and free things s/he isn't aware of.

As the number of types in a given run of the interpreter appears to be 
limited (am I right here?) and type-identity-tests are valid within a
single interpreter run only (am I right here?) an API like
  typeindex = Py_TypeToTypeIndex(typeobject);
which would use a dictionary as storage for the mapping and generate
the index numbers on the fly would do the trick. Call it once during
module initalization and the
  Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
macro would be a oneliner to test the bit in the set.

A few hundred bits in the set would get us a long way, I guess.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From fdrake@acm.org  Tue Nov  7 23:06:25 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:06:25 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
 <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
 <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > I think that making "div" an infix operator would be setting a
 > horrible precedent.  Currently, all infix operators "look like"
 > operators, i.e. they are non-alphabetic characters, and things that
 > look like words are either functions or reserved words.

  Like "is", "in", "is not", and "not in"?

 > Long before the creation of python-dev, this issue had been discussed
 > numerous times on c.l.py, and the '//' operator was suggested several
 > times, and I don't think anybody ever had a problem with it...

  I don't have a strong preference for either // or div, but
definately want this to be an operator.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fdrake@acm.org  Tue Nov  7 23:17:10 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:17:10 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <3A083BE6.4A502210@activestate.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
 <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
 <20001107141702.H27208@xs4all.nl>
 <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
 <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
 <3A083BE6.4A502210@activestate.com>
Message-ID: <14856.36214.267784.220958@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I think that the standard Python compiler is the appropriate tool for
 > this sort of thing. Anything that can be caught "statically" might as
 > well be implemented right in the compiler (at least from the user's
 > point of view) rather than in a separate "deprecation nanny."

  For linting programs using the final specification, this fine.  I'm
thinking that a tool to read over people's sources and say "'div' is
used in 120 places out of 56K lines of code." would be helpful because
we could determine the extent of the effect of using "div" instead of
"//".  If there are a limited number of projects affected, it may be
entirely reasonable to find out that there aren't enough uses to worry
about, and it becomes acceptable to make it a keyword (like the other
textual infix operators).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From greg@cosc.canterbury.ac.nz  Wed Nov  8 00:42:40 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 13:42:40 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.20202.174151.647981@buffalo.fnal.gov>
Message-ID: <200011080042.NAA00284@s454.cosc.canterbury.ac.nz>

Charles G Waldman <cgw@fnal.gov>:

> The other point - "=" vs "==" - is a bit harder to answer.

I think the reason this causes so much trouble is that many
programming languages, not to mention mainstream mathematics,
use "=" to mean what C uses "==" for.

Other such pairs in C, e.g. "&" vs "&&" and "+" vs "++",
don't seem to cause anywhere near as much difficulty, so
I don't think the problem is one of visual confusion, but
of semantic confusion.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+




From guido@python.org  Wed Nov  8 01:34:49 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 20:34:49 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Tue, 07 Nov 2000 11:04:33 PST."
 <20001107190433.26578.qmail@hyperreal.org>
References: <20001107190433.26578.qmail@hyperreal.org>
Message-ID: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>

[Note: the stackless@starship.python.net list seems to have trouble
again.  BeOpen may have crashed the machine again. :-( ]

[Note: response to Christian at end.]

[Guido]
> > 
> > A strawman proposal:
> > 
> > The uthread module provides the new functionality at the lowest level.

Eric (Tiedemann, right?  There are two Erics here -- it would help if
you signed your full name :-) writes:

> I really like this as a primitive appropriate for Python's evolution.

Cool.  I think I spoke too soon when I called it uthreads -- these are
really more like coroutines.

I also forgot to mention that I am assuming with this strawman that we
can use an essentially stackless implementation of the PVM.  I hope
that it will be possible to make it a lot simpler than current
stackless though, by not doing continuations.  Freezing a frame in
place is a lot simpler than freezing for multiple uses, which requires
one to decide what to copy and what to share!

> > - When func() returns, the uthread that was executing it ceases to be
> >   runnable.  The uthread that most recently yielded to it is resumed,
> >   unless that is no longer runnable, in which case the uthread that
> >   most recently yielded to *it* is resumed, and so on until a runnable
> >   uthread is found or until no runnable uthreads are left, in which
> >   case the program terminates.  (XXX I need a proof here that this
> >   works.)
> 
> I'd like it added that when a uthread chains to its yielder it drops
> (i.e., nulls and decrefs) the reference to that yielder.  I want
> uthreads in some of the same applications in which i disable gc for
> real-time purposes, and I don't want circular structures of unrunnable
> uthreads leaking my memory.

Good point.

> > - Calling u.yield() or u.kill() for a non-runnable uthread is an error
> >   and raises an exception.
> 
> A runnable() predicate might be nice.

Yes.

> > - I'm not sure that I got the start conditions right.  Should func() be
> >   be allowed to run until its first yield() when uthread.new(func) is
> >   called?
> 
> +1 for no on this.

You're being unnecessarily cryptic.  "Yes for no"?  So you're for the
original proposal (which doesn't start func() at all until it is
yielded to for the first time).

> > - I'm not sure that the rules for returning and raising exceptions
> >   from func() are the right ones.
> 
> I'm particularly unsure about the exception propagation.  It could
> always be disabled by a universal exception handler in the uthread,
> but I'm not sure it's even worth the implementation effort.

Agreed.  We may have to experiment.

> > - Should it be possible to pass a value to another uthread by passing
> >   an argument to u.yield(), which then gets returned by the resumed
> >   yield() call in that uthread?
> 
> Certainly! :)

This affects the initial condition.  If u hasn't called func() yet,
and I call u.yield(42), where does the 42 go?  Does it call func(42)?
That may make it make it unnecessarily hard to get the end conditions
right for func().  Again, we'll have to write some sample code to see
how this turns out in practice.

> As written, the strawman seems to require that the dread intervening C
> stack frames are handled transparently (since it doesn't say anything
> about them).  This seems pretty important to me.  An instance method
> may well know that it should yield, yet not know that it's being
> called as a callback from a class/type that's just been moved to C.

Not sure what you meant by intervening.  I certainly intended Python
to Python calls to be handled without creating extra C stack frames.
When Python calls C which calls back into Python, this is considered
all part of the same uthread.

Where it gets tricky is when this spawns a new uthread, which also
calls C which calls Python.  Now the second uthread has a C stack
frame above the C stack frame that's part of the first uthread.  This
means that the second uthread must return from its C code before the
first uthread can return to its C code!  A little extra bookkeeping
will be necessary to check for this -- so that when it is attempted an
exception is raised, rather than a return attempted from the wrong
uthread back into C.  This is the same as for current stackless.  The
solution is simply that the application "shouldn't do that."

> OTOH..not handling this transparently would increase my market value
> as a Python programmer.  Handling it right might get me some unpaid
> work implementing some of the low-level details for Linux.  Hmm! :D

If you want do hack continuations in C, be my guest -- as long as you
stay 10,000 kilometers away from core Python. :-)

[Now replying to Christian:]

> Just answering/clarifying a few bits,
> since I can't change your opinion about
> continuations, anyway.

Right!

> Guido van Rossum wrote:
> > 
> > I've thought about it a little more, and drawn some pictures in my
> > head.
> 
> I have to agree with Guido when he says:
> 
> > I still have to disagree with Christian when he says:
> > 
> > > Making Python completely coroutine aware, without
> > > tricking the C stack, is 90 percent of the problem.
> > > But after walking that far, there is no reason
> > > to leave the other 10 percent alone.
> 
> ... since I meant implementability. Of course there are
> other reasons gainst continuations. I just did it since they
> were in reach.

Hm.  Having seen a few fragments of your implementation today (just a
very little bit, since we were having an all-day meeting) I feel that
there are a lot of extra hacks needed to make the reuse of
continuations necessary.  This shouldn't be needed in my version.

> > Without continuations, but with microthreads (uthreads) or coroutines,
> > each (Python) stack frame can simply be "paused" at a specific point
> > and continued later.  The semantics here are completely clear (except
> > perhaps for end cases such as unhandled exceptions and intervening C
> > stack frames).
> 
> I agree. But also with continuations, the situation is identical,
> as long as you don't try anything else where continuations
> would be needed.

But the complexity in the code still exists because a continuation
*could* be reused and you don't know if it will ever happen so you
must be prepared.

> Note that they will not need to be created when the mentioned
> structures are implemented well. We don't have to implement them,
> but providing support for them in the interpreter framework
> is simple. (that's the 10% issue).

Having seen your code (frankly, a mess!) I don't believe that it's
only 10% at all.

> > With continuations, you have to decide how much state to save for a
> > future continuation.  It would seem easy enough: save all state kept
> > in the frame except for the local variables.  But now consider this:
> > the "evaluation stack" contained in each frame could easily be
> > replaced by a bunch of temporary variables, if we had a slightly
> > different instruction set (3-address opcodes instead of stack-based
> > opcodes).  Then would we save those temporary variables or not?  it
> > can make a difference!  Since the "save continuation" operation is a
> > function call, you can easily save a continuation while there are some
> > items on the value stack.  I believe the current implementation saves
> > these so they are restored when you jump to the continuation.  But if
> > the compiler were to use temporary variables instead of the evaluation
> > stack, they might not have been restored!
> 
> I would consider these temporary variables registers which must
> be preserved. They are immutable objects as part of the immutable
> continuation, treated as values. Stack or registers, this is part
> of an expression evaluation. Temporary results must conceptually
> be read-only, whatever way I implement this.

I heard from Tim that he helped you get this right.  The fact that it
is so hard to know the requirements for a practical implementation
makes me very worried that continuations may have hidden bugs.

> > Here's another example.  Suppose you set up a for loop.  After three
> > iterations through the loop you save a continuation.  Then you finish
> > hree more iterations.  Then you return to the saved continuation.
> > Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> > this without trying it.  My guess: it gets restarted at 3 iterations,
> > because the loop index is saved on the value stack.  If you rewrite
> > this using a while loop, however, it would get restarted at 6
> > iterations, because then your loop index is an unsaved local variable.
> > Ditto if you changed the bytecode compiler so for loops use an
> > anonymous local variable instead of an entry on the evaluation
> > stack.
> 
> Wrong guess!
> Exactly for that reason I changed the loop code to put a mutable
> loopcounter object on the stack.
> The loop works perfectly.

Wow.  i'm impressed.  You must have borrowed my time machine. :-)

Still, I believe there was a time when the loop *didn't* work
perfectly yet. It is really hard to know what is needed.  Are you
*sure* that it now *always* does the right thing?  What if I save a
continuation in the middle of a shortcut Boolean expression (and/or
stuff)?  Or in cases like a<b<c?  (Here b gets saved on the stack to
avoid loading it again.)

> > This semantic imprecision is one of the reasons why I don't like the
> > concept of continuations.  (I've been told that the exact semantics of
> > continuations in Scheme differ greatly between Scheme implementations.)
> 
> In a sense, you have continuations already, also with the restriction
> to gen/co/uthread structures. The only difference is to treat a
> frame as exactly one continuation and to disallow to have more
> than one at any time.
> This saves the decision about the ambiguities you mentioned.

Yeah, right.  You can call pausable/resumable frames use-once
continuations if you want to.  And if that gives you the happy feeling
that I "support" continuations, fine.

> I agree that going to this point and not further for the
> Python core is a good idea.
> A PEP doesn't need to name continuations at all.

Good.

> On the other hand, I don't see a reason why this hsould mean
> that Python *must not* support them. What I'd like to keep
> is the possibility to still write such an extension module.
> 
> Enabling this for educational purposes is a great value
> that comes at a cheap price and no impact for the core.

I doubt it.  I'm not going to allow any compromises just to make it
easier to reuse continuations.  (Such as using a mutable counter in
the for-loop code.)

> > Now let's look at Jython.  In Jython, we can simulate "paused frames"
> > well enough by using real Java threads.  However full continuations
> > would require modifications to the JVM -- which is unacceptable to a
> > language boasting "100% Pure Java".  Another reason against allowing
> > continuations.
> 
> Not quite true, after I heard of a paper that shows how
> to implement continuations in Java, using threads.
> But I'll come back to that when I have the paper.

I've heard that his margin was too small to contain the proof.  I
expect that it will be a disappointment from a practical point of
view: perhaps he emulates the JVM in Java.

> > So, all in all, I don't think of continuations as "the last 10% that
> > we might as well add to finish the job."  I see it as an ill-specified
> > hypergeneralization.
> 
> Can we agree to not support them without forbidding them?

I won't forbid them, but I won't make compromises to the core PVM that
would make them easier to implement.  Your patch set would still be a
heck of a lot smaller of course.

> ...
> > A strawman proposal:
> 
> Ok, this looks all very well to me. More on that later.
> One question: Why do you want an explicit u.yield() ?
> Uthreads are scheduled automatically now, like real
> threads. Do you see a major drawback in supporting
> this, maybe as an option? Or do you see automatic
> scheduling as an extra construct on top with a special
> "scheduler" uthread?

See my response near the top to Eric about this.  I was thinking of a
lower-level concept, like coroutines.  I might consider automatic
scheduling of uthreads too.  But I've noticed that there are some ugly
hacks in that code, too. :-)

I've lived for years with a system (early Amoeba) that had threads
with only explicit scheduling: other threads would only run when you
were blocked for I/O or waiting for a semaphore.  It made for very
easy coding in some cases, since you didn't need to protect critical
sections with mutexes.  Unles, that is, you invoke stuff that might do
I/O (maybe for debugging :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Wed Nov  8 02:12:09 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 15:12:09 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>
Message-ID: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz>

Guido:

> I hope that it will be possible to make it a lot simpler than current
> stackless though, by not doing continuations.

My feeling is that this won't be the case. The fundamental
change of structure needed to make it stackless will be
much the same, as will the thought processes necessary
to understand how it works.

> Where it gets tricky is when this spawns a new uthread, which also
> calls C which calls Python... The solution is simply that the 
> application "shouldn't do that."

I worry that this is going to be a rather severe restriction.
For instance, it may make it impossible to spawn a uthread
from within a callback from a GUI framework. Since with many
GUI frameworks the entire application gets executed in
callbacks, you wouldn't be able to use uthreads at all with
such a framework.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Wed Nov  8 02:58:20 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 21:58:20 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 15:12:09 +1300."
 <200011080212.PAA00313@s454.cosc.canterbury.ac.nz>
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz>
Message-ID: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>

> Guido:
> 
> > I hope that it will be possible to make it a lot simpler than current
> > stackless though, by not doing continuations.

[Greg Ewing]
> My feeling is that this won't be the case. The fundamental
> change of structure needed to make it stackless will be
> much the same, as will the thought processes necessary
> to understand how it works.

I hope you are wrong but you may be right.  I'll have to have a good
look -- or someone else (not Christian!  With all due respect his code
is unreadable :-).

> > Where it gets tricky is when this spawns a new uthread, which also
> > calls C which calls Python... The solution is simply that the 
> > application "shouldn't do that."
> 
> I worry that this is going to be a rather severe restriction.
> For instance, it may make it impossible to spawn a uthread
> from within a callback from a GUI framework. Since with many
> GUI frameworks the entire application gets executed in
> callbacks, you wouldn't be able to use uthreads at all with
> such a framework.

But that's the same problem that current stackless has.  I take it
that you don't see the point of stackless then?  That's fine.  Maybe
this is not an application that could use uthreads.  They seem more
something for servers anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Wed Nov  8 03:18:10 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 16:18:10 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <200011080318.QAA00324@s454.cosc.canterbury.ac.nz>

Guido:

> I take it that you don't see the point of stackless then?

I have mixed feelings about it. I really like the idea of
uthreads, but I get the impression that Stackless as it
stands is only a partial implementation of the idea, with
no easy way in sight to evolve it into a full implementation.

> Maybe this is not an application that could use uthreads.

The point is that the user can be mixing Python and C stack
frames without even realising it. I was just giving one example
of how that can come about.

Saying "don't do that" isn't very helpful when "that" is
something which poeple often do quite unconsciously.

So, while I'd love to see uthreads as a core Python feature
one day, I think I'm in agreement with you that Stackless
isn't yet ready to be made into the standard Python
implementation.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pf@artcom-gmbh.de  Wed Nov  8 07:33:00 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 08:33:00 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com> from "Fred L. Drake, Jr." at "Nov 7, 2000  6: 6:25 pm"
Message-ID: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>

> Charles G Waldman writes:
>  > I think that making "div" an infix operator would be setting a
>  > horrible precedent.  Currently, all infix operators "look like"
>  > operators, i.e. they are non-alphabetic characters, and things that
>  > look like words are either functions or reserved words.
 
Fred L. Drake, Jr.:
>   Like "is", "in", "is not", and "not in"?

And not to forget "and", "or" which were also infix operators from
the very beginning.  So "div" is no precedent at all.

IMHO the term "horrible" applies to operator symbols composed out
of non-alphabetic characters, where the meaning of these operators 
is hard to guess.  

counter-example: Using "><" as a vector cross product operator might 
still make some sense.  

But what would be the meaning of all those other arbitrary combinations
like ".+", ".%", ".*", "//", "@.", "~*" or what else has been proposed
to extend Python in the numeric area?  As long as the meaning of such
an operator isn't obvious from basic math knowledge, I clearly prefer
keyword operators.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From mal@lemburg.com  Wed Nov  8 10:13:51 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 11:13:51 +0100
Subject: [Python-Dev] Starship down again ?!
References: <200011080620.WAA20659@slayer.i.sourceforge.net>
Message-ID: <3A09275F.CE4B01FA@lemburg.com>

> 
> +     Marc-André Lemburg's mx.Proxy package.  These Web pages appear to
> +     be unavailable at the moment.
> + 
> +         http://starship.python.net/crew/lemburg/
> + 

Looks like Starship is down again. Is this due to the move from
BeOpen to DC or has someone pulled the plug on that ADSL line ? ...

 9  sl-bb1-rly-0-0-0.sprintlink.net (144.232.14.6)  120 ms  117 ms  113 ms
10  beth1sr2-2-0-0.md.us.prserv.net (165.87.97.226)  114 ms  116 ms  114 ms
11  beth1br2-ge-6-0-0-0.md.us.prserv.net (165.87.29.182)  122 ms  121 ms  116 ms
12  sfra1br1-so-2-1-0-0.ca.us.prserv.net (165.87.233.42)  193 ms  192 ms  191 ms
13  sfra1sr3-ge-2-0-0-0.ca.us.prserv.net (165.87.33.121)  191 ms  189 ms  190 ms
14  165.87.161.13 (165.87.161.13)  191 ms  191 ms  189 ms
15  core4-g2-0.snfc21.pbi.net (209.232.130.77)  197 ms  192 ms  190 ms
16  rback26-fe2-0.snfc21.pbi.net (216.102.187.153)  212 ms  197 ms  197 ms
17  adsl-63-202-160-65.dsl.snfc21.pacbell.net (63.202.160.65)  206 ms  212 ms  204 ms
18  * * *
19  * * *

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From tismer@tismer.com  Wed Nov  8 11:23:46 2000
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 13:23:46 +0200
Subject: [Python-Dev] uthread strawman
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <3A0937C2.903D42@tismer.com>


Guido van Rossum wrote:
> 
> > Guido:
> >
> > > I hope that it will be possible to make it a lot simpler than current
> > > stackless though, by not doing continuations.
> 
> [Greg Ewing]
> > My feeling is that this won't be the case. The fundamental
> > change of structure needed to make it stackless will be
> > much the same, as will the thought processes necessary
> > to understand how it works.
> 
> I hope you are wrong but you may be right.  I'll have to have a good
> look -- or someone else (not Christian!  With all due respect his code
> is unreadable :-).

Are you talking of my changes to ceval.c or the continuationmodule?
I think it can't be the latter, since that does not matter at all
if we talk about Stackless. Stackless was written to make continuations
possible. It does not implement them.

My changes to ceval.c are written in the same style as your
original code, and it uses the same level of commenting as yours:
Nothing at all. :-)
With all due respect, I consider both versions equally unreadable,
unless one understands what the intent of the code is.

Until last October, I tried to keep everything as readable
and understandable as possible. Then it became clear that
this implementation would never make it into the core.
Then I gave up my efforts, and I also added a lot of
optimizations to the interpreter, by some systematic use
of macroes. Shurely his doesn't increase readability.

Forgetting about these optimizations, the code doesn't do
much more than the following:

eval_code2_setup
  is split off of the original eval_code2 function.
  It prepares a new frame for execution and puts it
  on top of the frame stack.

PyEval_Frame_Dispatch
  is a new function. It controls the execution of frames.
  Every initial or recursive interpreter call starts
  such a dispatcher. The nested dispatchers control the
  remaining "stackful" execution of Python.
  In its central loop, it runs the topmost frame of the
  frame stack, receives its return value and runs the
  next frame, until it sees the frame appear that
  spawned this dispatcher. Then it returns.

eval_code2_loop
  is the "real" part of the original eval_code2 function.
  It is not much different from the original. Major changes
  have been done to the entry code, the periodic checks
  in the loop, and the handling of function calls.

  The "big switch" has been simplified in the sense, that
  errors are no longer treated with various variables
  which have to be checked outside the switch.
  Instead, errors are directly mapped on a pseudo-opcode
  that allows to handle exceptions as just another case
  of the big switch.

  Every function call has got additional code that checks
  for the so-called unwind token, which tells us to leave
  this frame and to return to the scheduler.
  
  On entry to the frame, on every trip through the main loop,
  and after every function call, a callback f_callguard is
  checked for existence. If it exists, it is called, and
  if it returns -42, again the frame is left and we return
  to the scheduler.

  Entry into a frame has become a bit difficult, since we
  no longer know in advance whether a frame is expected
  to return a value or not. Due to uthread scheduling,
  switches occour between opcodes, and no values are
  transferred. When switching in the context of a function
  call, there *are* return values expected.
  This is all handled via some flags, in the frame entry
  code, line 948ff.

  Then, there are some very simple changes to the loop
  construct.

  Generally, more state variables are in the frames and
  kept up-to-date, like the instruction pointer.

  I'm omitting the extra code for uthread support here.

  Some functions were pulled out of the main loop, in order
  to make it smaller and easier to read. I would undo this
  today, since it makes comparison to the old version
  quite impossible, and it didn't yield more speed.

This is about all of it.
As you can see, there is no explicit support for co-anything
in the code. There are some generalisations to frame calling
and some callback hooks which actually do all co operations.

An implementation targeted for core integration would look
quite much different. It would provide more functionality
directly, without using callbacks.

A pure coroutine based implementation as you proposed
would not need the generalization of the frame parameter
passing, since switches can only occour in the context
of a function call.

Supporting auto-scheduled uthreads needs to distinguish
explicit and implicit switching, since implicit switching
occours between opcodes, not *in* opcodes.

The techniques for this can be written in quite a different
manner than I did.

Again, this code is not intended for inclusion in the core,
and not for drawing conclusions for the feasibility of
Stackless at all. The latter has been shown by applications
like the uthreads, and by its central use in the EVE game.
We need to judge the priciple, not the implementaiton.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From phil@river-bank.demon.co.uk  Wed Nov  8 12:26:54 2000
From: phil@river-bank.demon.co.uk (Phil Thompson)
Date: Wed, 08 Nov 2000 12:26:54 +0000
Subject: [Python-Dev] What to choose to replace Tkinter?
References: <049d01c0471f$d7899450$8119fea9@neil>
Message-ID: <3A09468E.9E767720@river-bank.demon.co.uk>

I've come to this discussion rather late...

Most modern GUI toolkits have (or will soon have) the widgets to compete
with Tk strengths. The difficult question with a Tkinter replacement is
the complete fragmentation of the GUI toolkit "market". I don't believe
that you can, today, identify a toolkit that you are sure is going to
have widespread support and the longevity needed (in 5 years time you
don't want to be in the position you are in today with Tk).

I see two alternatives...

- make the Tkinter replacement an abstraction layer between Python and
the *user's* choice of toolkit. The developer gets a consistent API, and
toolkits can be adopted and dropped as fashions change. This is the
approach taken by VeePee (http://www.thekompany.com/projects/vp/).

- don't bundle Tkinter with Python. At least you then make people think
a bit more about what they want from a toolkit and make an appropriate
choice - let Tkinter's replacement be found by natural selection. At the
very least let's have a more up-front presentation of the different
options, strengths/weaknesses etc on the web site. Cameron must be
getting bored of pointing people to his toolkit summary.

For the record, Qt has a good Canvas widget, Unicode support, user
selectable Windows/Mac/Unix look & feel etc, etc.

Phil


From guido@python.org  Wed Nov  8 13:20:02 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 08:20:02 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 13:23:46 +0200."
 <3A0937C2.903D42@tismer.com>
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
 <3A0937C2.903D42@tismer.com>
Message-ID: <200011081320.IAA21990@cj20424-a.reston1.va.home.com>

> Again, this code is not intended for inclusion in the core,
> and not for drawing conclusions for the feasibility of
> Stackless at all. The latter has been shown by applications
> like the uthreads, and by its central use in the EVE game.
> We need to judge the priciple, not the implementaiton.

Of course.  Thanks by the way for the clear explanation of what needs
to be done!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@mems-exchange.org  Wed Nov  8 14:02:49 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Wed, 8 Nov 2000 09:02:49 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001107182621.I27208@xs4all.nl>; from thomas@xs4all.net on Tue, Nov 07, 2000 at 06:26:21PM +0100
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>
Message-ID: <20001108090249.A28202@ludwig.cnri.reston.va.us>

On 07 November 2000, Thomas Wouters said:
> Sorry for the late remark (I did see your earlier message) but after reading
> the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
> for non-essential stuff: warnings, debug-info and optimizations. Removing
> things from OPT shouldn't break anything, and neither should adding options
> that fit in the categories above. (Barring broken compilers, of course.)
> 
> Instead, the -fPIC option should be added to CFLAGS, I think. The Python
> autoconf setup is slightly less versatile than most, though, since it's
> doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
> without editing configure(.in) :P If noone else does it before me, I'll see
> about fixing at least the -fPIC thing later, when I find some time ;P

Good point -- fixing CFLAGS instead of OPT sounds right to me.  I'm not
really sure on where to do this, though.

Ooh, I just noticed this in configure.in:

  # DG/UX requires some fancy ld contortions to produce a .so from an .a
  case $MACHDEP in
  dguxR4)
        LDLIBRARY='libpython$(VERSION).so'
        OPT="$OPT -pic"
        ;;

No prize for guessing that "-pic" on the DG/UX compiler has a similar
effect to GCC's -fPIC, and based on the comment this is required.  I'm
guessing this should be in CFLAGS as well.

Oh wait: CFLAGS is not exported from configure.in -- it's *only* defined
in the Makefile.  From Makefile.in:

  CFLAGS=		$(OPT) -I. $(DEFS)

IOW, it looks like OPT is used for all non-preprocessor compiler flags,
whether they're "frills" like optimization/debugging or not.

Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
right thing, but for the wrong reason.  Fixing it would require a little
more involved surgery on configure.in and the Makefiles.  And it would
also require reexamining every use of OPT in configure.in (not too hard,
"grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
so forth at config time or at make time.

I'm willing to spend some time on this; does anyone think it's a
pointless waste of time?

        Greg


From fdrake@acm.org  Wed Nov  8 14:04:32 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:04:32 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001108090249.A28202@ludwig.cnri.reston.va.us>
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
 <20001107182621.I27208@xs4all.nl>
 <20001108090249.A28202@ludwig.cnri.reston.va.us>
Message-ID: <14857.23920.646760.779849@cj42289-a.reston1.va.home.com>

Greg Ward writes:
 > I'm willing to spend some time on this; does anyone think it's a
 > pointless waste of time?

  I'm certainly willing to allocate some of your cycles to this.  ;)
I'll even help test it once you've checked it in.
  Seriously, I think you're right -- there needs to be a separation of
what's needed and optional stuff added from the make command line.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From guido@python.org  Wed Nov  8 14:21:20 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 09:21:20 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: Your message of "Wed, 08 Nov 2000 09:02:49 EST."
 <20001108090249.A28202@ludwig.cnri.reston.va.us>
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>
 <20001108090249.A28202@ludwig.cnri.reston.va.us>
Message-ID: <200011081421.JAA22160@cj20424-a.reston1.va.home.com>

> Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
> right thing, but for the wrong reason.  Fixing it would require a little
> more involved surgery on configure.in and the Makefiles.  And it would
> also require reexamining every use of OPT in configure.in (not too hard,
> "grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
> if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
> so forth at config time or at make time.
> 
> I'm willing to spend some time on this; does anyone think it's a
> pointless waste of time?

No, please fix it right!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From cgw@fnal.gov  Wed Nov  8 14:23:47 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Wed, 8 Nov 2000 08:23:47 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
 <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14857.25075.482728.276846@buffalo.fnal.gov>

 > > Charles G Waldman writes:
 > >  > I think that making "div" an infix operator would be setting a
 > >  > horrible precedent.  Currently, all infix operators "look like"
 > >  > operators, i.e. they are non-alphabetic characters, and things that
 > >  > look like words are either functions or reserved words.
 >  
 > Fred L. Drake, Jr.:
 > >   Like "is", "in", "is not", and "not in"?
 > 
 > Peter Funk writes:
 > And not to forget "and", "or" which were also infix operators from
 > the very beginning.  So "div" is no precedent at all.

OK, I stand corrected and feel suitably foolish.  However I still
think it's quite inconsistent to have divmod(a,b) but a div b.



From fdrake@acm.org  Wed Nov  8 14:38:51 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:38:51 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14857.25075.482728.276846@buffalo.fnal.gov>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
 <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
 <14857.25075.482728.276846@buffalo.fnal.gov>
Message-ID: <14857.25979.244131.879387@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > OK, I stand corrected and feel suitably foolish.  However I still
 > think it's quite inconsistent to have divmod(a,b) but a div b.

  I suspect div would be much more widely used than divmod(), which is
essentially a performance optimization when you need both results.
One reason *not* to make divmod() an operator, aside from issues of
legacy code, is that it really returns two results (never mind that it
returns exactly one tuple); I can't think of another operator that
conceptually returns two values.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From pf@artcom-gmbh.de  Wed Nov  8 15:32:22 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 16:32:22 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: <200011081517.HAA16919@slayer.i.sourceforge.net> from Guido van Rossum at "Nov 8, 2000  7:17:51 am"
Message-ID: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

this rather old module still contains string exceptions.
Since string exceptions are depreceated in favour of class based
exceptions wouldn't it be better to tweak those few lines into
class based exceptions now?

> Add 1994 Coroutine module by Tim Peters
[...]
> Killed = 'Coroutine.Killed'
> EarlyExit = 'Coroutine.EarlyExit'

Regards, Peter


From guido@python.org  Wed Nov  8 15:42:41 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 10:42:41 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: Your message of "Wed, 08 Nov 2000 16:32:22 +0100."
 <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de>
References: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <200011081542.KAA22738@cj20424-a.reston1.va.home.com>

> this rather old module still contains string exceptions.
> Since string exceptions are depreceated in favour of class based
> exceptions wouldn't it be better to tweak those few lines into
> class based exceptions now?
> 
> > Add 1994 Coroutine module by Tim Peters
> [...]
> > Killed = 'Coroutine.Killed'
> > EarlyExit = 'Coroutine.EarlyExit'

No.  This code is of historic interest only.  Don't touch it please!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pf@artcom-gmbh.de  Wed Nov  8 16:06:56 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 17:06:56 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: <200011081542.KAA22738@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 8, 2000 10:42:41 am"
Message-ID: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de>

I was nitpicking:
> > this rather old module still contains string exceptions.
> > Since string exceptions are depreceated in favour of class based
> > exceptions wouldn't it be better to tweak those few lines into
> > class based exceptions now?
> > 
> > > Add 1994 Coroutine module by Tim Peters
> > [...]
> > > Killed = 'Coroutine.Killed'
> > > EarlyExit = 'Coroutine.EarlyExit'

Guido van Rossum answered:
> No.  This code is of historic interest only.  Don't touch it please!

Hmmmm.... I always thought of the Demo directory as a repository for
example code, which may be used to teach Python programming to
beginners.  I know, that some pieces are a way out of date.  

But I think it would be a worthwile goal to update at least some of
those pieces step by step to reflect current Python coding habits.
The README in Demo says:

"""This directory contains various demonstrations of what you can do with
Python. [...]"""

If you want to turn the Demo directory into a museum of code snippets of
historic interest, at least the README should say so. ;-)

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From gmcm@hypernet.com  Wed Nov  8 17:09:48 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 8 Nov 2000 12:09:48 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com>
Message-ID: <3A09428C.12529.1044B070@localhost>

[Guido]
> Without continuations, but with microthreads (uthreads) or
> coroutines, each (Python) stack frame can simply be "paused" at a
> specific point and continued later.  The semantics here are
> completely clear (except perhaps for end cases such as unhandled
> exceptions and intervening C stack frames).

Exceptions require some thought, particularly because the "for" 
protocol uses an IndexError as a signal. In my own stuff I've found I 
need to catch all exceptions in the coroutine, primarily because I've 
always got resources to clean up, but clearly the implementation has 
to do the right thing when an exception crosses the boundary.
 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest
> level. Uthread objects represent microthreads.  An uthread has a
> chain of stack frames linked by back pointers just like a regular
> thread. Pause/resume operations are methods on uthread objects. 
> Pause/resume operations do not address specific frames but
> specific uthreads; within an uthread the normal call/return
> mechanisms can be used, and only the top frame in the uthread's
> stack of call frames can be paused/resumed (the ones below it are
> paused implicitly by the call to the next frame, and resumed when
> that call returns).

I'm not convinced (though I'm not asking you to convince me - I need to 
ponder some more) that this is the right approach. My worry is that to 
do coroutines, we end up with a bunch of machinery on top of 
uthreads, just like Tim's old coroutine stuff, or implementations of 
coroutines in Java.

My mental test case is using coroutines to solve the impedance 
mismatch problem. SelectDispatcher is a simple example (write 
"client" code that looks like it's using blocking sockets, but multiplex 
them behind the curtain). Using a "pull" parser as a "push" parser is 
another case, (that is, letting it think it's doing its own reads).

But what about using a "pull" lexer and a "pull" parser, but tricking 
them with coroutines so you can "push" text into them? Tim's 
implementation of the  Dahl & Hoare example (which I 
rewrote in mcmillan-inc.com/tutorial4.html) shows 
you *can* do this kind of thing on top of a thread 
primitive, but might it not be much better done on 
a different primitive?

Again, I'm not really asking for an answer, but I think this type of problem is not uncommon, and a wonderful use of coroutines; so I'm wondering if this is a good trade-off.

> - u.yield() pauses the current uthread and resume the uthread
> u where it was paused.  The current uthread is resumed when
> some other uthread calls its yield() method.  Calling
> uthread.current().yield() is a no-op. 

This doesn't seem like enough: sort of as though you designed a 
language in which "call" and "return" were spelled the same way. 
Certainly for coroutines and generators, people gravitate towards 
paired operations (eg. "suspend" and "resume"). Again, Tim's 
demonstrated you can do that on top of threads, but it sure seems to 
me like they should be primitives.

> I think this API should enough to implement Gordon's
> SelectDispatcher code.  In general, it's easy to create a
> scheduler uthread that schedules other uthreads.

Thank you :-).

> Open issues:
> 
> - I'm not sure that I got the start conditions right.  Should
> func() be
>   be allowed to run until its first yield() when
>   uthread.new(func) is called?

For coroutine stuff, that doesn't bother me. For uthreads, I'd think (like 
real threads) that creation and starting are different things.
 
> - I'm not sure that the rules for returning and raising
> exceptions
>   from func() are the right ones.
> 
> - Should it be possible to pass a value to another uthread by
> passing
>   an argument to u.yield(), which then gets returned by the
>   resumed yield() call in that uthread?

Argument (in and / or out) passing is a necessity for generators and 
coroutines. But, as mentioned above, I don't think a symmetrical 
"yield" is the right answer.
 
> - How do uthreads interact with real threads?  Uthreads are
> explicitly
>   scheduled through yield() calls; real threads use preemptive
>   scheduling.  I suppose we could create a new "main" uthread for
>   each real thread.  But what if we yield() to an uthread that's
>   already executing in another thread?  How is that error
>   detected?

I think I would (perhaps naively) expect that I could create a uthread in 
one (real) thread, and then pass it off to another (real) thread to 
execute.

Another post brings up GUIs and uthreads. I already expect that I'm 
going to have to dedicate a (real) thread to the GUI and ponder very 
carefully how that thread interacts with others. Of course, that's from 
lessons learned the hard way; but personally I'm not expecting 
uthreads / coroutines to make that any easier.

[About continuations: while I love the fact that Christian has made 
these available for playing, I have so far not found them productive. I 
wrote a simple minded backtracking parser using them, but found it no 
better than a coroutine based one. But I am interested in how a *real* 
pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good* 
idea, boss", please.]

- Gordon


From akuchlin@mems-exchange.org  Wed Nov  8 18:11:26 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 08 Nov 2000 13:11:26 -0500
Subject: [Python-Dev] Catalog-SIG created
Message-ID: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>

A mailing list for the Catalog SIG has been created, to discuss the
design and construction of a Vaults/CPAN/LSM-like index for Python.
Web pages for the SIG don't exist yet, but will be created soon.

The SIG's charter:

The Python Catalog SIG aims at producing a master index of Python
software and other resources.  It will begin by figuring out what the
requirements are, converging on a design for the data schema, and
producing an implementation.  ("Implementation" will almost certainly
include mean a set of CGI scripts for browsing the catalog, and may
also contain a standard library module for automatically fetching &
installing modules, if the SIG decides that's a worthwhile feature.)

--amk


From guido@python.org  Wed Nov  8 18:21:39 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 13:21:39 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: Your message of "Wed, 08 Nov 2000 17:06:56 +0100."
 <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de>
References: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <200011081821.NAA23572@cj20424-a.reston1.va.home.com>

> Hmmmm.... I always thought of the Demo directory as a repository for
> example code, which may be used to teach Python programming to
> beginners.  I know, that some pieces are a way out of date.  
> 
> But I think it would be a worthwile goal to update at least some of
> those pieces step by step to reflect current Python coding habits.
> The README in Demo says:
> 
> """This directory contains various demonstrations of what you can do with
> Python. [...]"""
> 
> If you want to turn the Demo directory into a museum of code snippets of
> historic interest, at least the README should say so. ;-)

If you want to make a proposal for reorganizing the Demo directory,
please do so.  There are more important problems with the Demo
directlry than the fact that some code still uses string exceptions.
Please don't start fixing the small nits without seeing the big
picture.

(That's all I have time for now.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tismer@tismer.com  Wed Nov  8 17:20:43 2000
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 19:20:43 +0200
Subject: [Python-Dev] uthread strawman
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com> <3A09428C.12529.1044B070@localhost>
Message-ID: <3A098B6B.522EA5E8@tismer.com>


Gordon McMillan wrote:

[snipped all the good stuff away for bigger brains than mine ]

> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

Yes, I saw Gordon making heavy use of naked continuations,
but actually they were not really what he needed. I believe
this since he made much use of co.update(), which moves
a continaution to the most current state of the frame.

In fact, what Gordon would need (and most probably most of
us as well) is just the handle to a frame, and the ability
to switch to it. In Gordon's case, these would probably
be "continuation" which are not frozen, but simply track
the frame as it is. I'm not absolutely shure, but quite.

I'm happy to toss continuations for core Python, if we
can find the right building blocks for coro/gen/uthreads.
I think Guido comes quite near this, already.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From fdrake@acm.org  Wed Nov  8 19:50:50 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 14:50:50 -0500 (EST)
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>
 <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
Message-ID: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>

Mark W. Alexander writes:
 > Is there another way to subscribe, or did I just jump the gun?

  You can get to the Mailman interface at:

	http://www.python.org/mailman/listinfo/catelog-sig/

  The Web pages aren't actually there yet; Andrew will get to this
when he can, I'm sure.  ;)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From mal@lemburg.com  Wed Nov  8 20:34:49 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 21:34:49 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <20001107225758.B77651301D9@oratrix.oratrix.nl>
Message-ID: <3A09B8E9.E247125F@lemburg.com>

Jack Jansen wrote:
> 
> > > In other words, what does this new type_id thing
> > > actually *mean*?
> >
> > For the interpreter it means that it can assume the type
> > interface to be binary compatible to the "original"
> > type, e.g. by setting the flag to say PyDict_TypeID
> > the type assures that all PyDict_*() APIs will work
> > on the type -- basically the same thing as PyDict_Check()
> > does now except that the type object needn't be the same
> > anymore.
> 
> I would be _very_ happy if this single type_id could somehow be
> replaced by an array, or a bitset.

I guess a bit array would be a possibility...

#define PyList_Check(obj) ((obj)->ob_type->\
                            capabilities[Py_ListType->cap_slot])
 
cap_slot could be set at type object creation time using
some Python slot id generator (a function which outputs integers
up to the maximum length of the capabilities array and raises
a Py_FatalError() if this limited is excceded).

> I have a lot of types in MacPython that are acceptable to the APIs of
> other types, a sort of poor-mans-inheritance scheme. For instance, all
> operating system calls that accept a MacOS WindowPtr will also happily
> accept a DialogPtr. Major magic is needed to get this to work
> reasonably in Python, and the Python user can still accidentally mess
> up the refcounting scheme and free things s/he isn't aware of.
> 
> As the number of types in a given run of the interpreter appears to be
> limited (am I right here?) and type-identity-tests are valid within a
> single interpreter run only (am I right here?) 

Right * 2

> an API like
>   typeindex = Py_TypeToTypeIndex(typeobject);
> which would use a dictionary as storage for the mapping and generate
> the index numbers on the fly would do the trick. Call it once during
> module initalization and the
>   Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
> macro would be a oneliner to test the bit in the set.
> 
> A few hundred bits in the set would get us a long way, I guess.

One thing I'm unsure about is whether changing cap_slot ids
between runs of the interpreter are a good idea. Also, I think that
the basic types should be given constant cat_slot ids to enhance
performance (the id generator could be made to start at say 10 
and the basic types be fixed in the range 0-9).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From trentm@ActiveState.com  Wed Nov  8 21:07:39 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 8 Nov 2000 13:07:39 -0800
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 08, 2000 at 02:50:50PM -0500
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us> <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net> <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>
Message-ID: <20001108130739.H27185@ActiveState.com>

On Wed, Nov 08, 2000 at 02:50:50PM -0500, Fred L. Drake, Jr. wrote:
> 	http://www.python.org/mailman/listinfo/catelog-sig/ 

.replace("catelog", "catalog")

:)


-- 
Trent Mick
TrentM@ActiveState.com


From jeremy@alum.mit.edu  Thu Nov  9 00:14:45 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:14:45 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
Message-ID: <14857.60533.766291.786182@bitdiddle.concentric.net>

--QTUWfOmTKZ
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit

I just uploaded a patch to sourceforge that revises the CALL_FUNCTION
implementation to use a bunch of functions and avoid the long block of
inline code in eval_code2.

The overall performance of the patch is about the same.  The current
patch causes a big slowdown for the use of keyword arguments, but is
otherwise as fast or faster then the old version.  The keyword
slowdown should be avoidable without too much effort.

I wrote a short benchmark that demonstrates the effect on many
variations of function calls.  The output lists the test time, and the
median, min, and max time of 10 executions.  (The benchmark script is
attached to this message.)

The vanilla CVS tree produces these results:

time_arg1 0.09 0.09 0.11
time_arg2 0.1 0.09 0.12
time_arg3 0.11 0.1 0.12
time_fact 0.12 0.1 0.14
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.1 0.08 0.11
time_callable 0.37 0.33 0.38
time_keyword 0.14 0.13 0.18
time_star 0.12 0.12 0.14
time_kw 0.25 0.24 0.27
time_kw2 0.69 0.66 0.73
time_starkw 0.24 0.23 0.26
time_init 0.64 0.63 0.68
total 3.18

The CVS tree with the CALL_FUNCTION patch applied produces these
results: 
time_arg1 0.09 0.09 0.1
time_arg2 0.1 0.09 0.1
time_arg3 0.11 0.09 0.13
time_fact 0.11 0.11 0.14
time_meth 0.09 0.09 0.1
time_umeth 0.1 0.1 0.11
time_builtin 0.08 0.07 0.09
time_callable 0.35 0.34 0.38
time_keyword 0.42 0.4 0.44      (*** big slowdown ***)
time_star 0.13 0.13 0.15
time_kw 0.25 0.23 0.29
time_kw2 0.66 0.61 0.79
time_starkw 0.24 0.22 0.27
time_init 0.66 0.6 0.72
total 3.39

Jeremy


--QTUWfOmTKZ
Content-Type: text/plain
Content-Disposition: inline;
	filename="callbench.py"
Content-Transfer-Encoding: 7bit

import time

MANY = 40000
AFEW = 2000

def time_arg1(iters=range(MANY)):
    def arg1(x):
        return x

    t0 = time.clock()
    for i in iters:
        arg1(i)
    t1 = time.clock()
    return t1 - t0

def time_arg2(iters=range(MANY)):
    def arg2(x, y):
        return y

    t0 = time.clock()
    for i in iters:
        arg2(i, i)
    t1 = time.clock()
    return t1 - t0

def time_arg3(iters=range(MANY)):
    def arg3(x, y, z):
        return z

    t0 = time.clock()
    for i in iters:
        arg3(i, i, i)
    t1 = time.clock()
    return t1 - t0

def fact(n):
    if n == 0:
        return 1L
    else:
        return n * fact(n - 1)

def time_fact(iters=range(AFEW)):
    t0 = time.clock()
    for i in iters:
        fact(10)
    t1 = time.clock()
    return t1 - t0

class Foo:
    def method(self, x):
        return x

def time_meth(iters=range(MANY)):
    inst = Foo()
    meth = inst.method
    t0 = time.clock()
    for i in iters:
        meth(i)
    t1 = time.clock()
    return t1 - t0

def time_umeth(iters=range(MANY)):
    inst = Foo()
    meth = Foo.method
    t0 = time.clock()
    for i in iters:
        meth(inst, i)
    t1 = time.clock()
    return t1 - t0

def time_builtin(iters=range(MANY)):
    l = []
    func = l.count
    t0 = time.clock()
    for i in iters:
        func(i)
    t1 = time.clock()
    return t1 - t0

class Bar:
    def __call__(self, x):
        return x

def time_callable(iters=range(MANY)):
    inst = Bar()
    t0 = time.clock()
    for i in iters:
        inst(i)
    t1 = time.clock()
    return t1 - t0

def time_keyword(iters=range(MANY)):
    def kwfunc(a=None, b=None):
        return a + b

    t0 = time.clock()
    for i in iters:
        kwfunc(a=i, b=i)
    t1 = time.clock()
    return t1 - t0

def time_star(iters=range(MANY)):
    def star(a, b, c):
        return b

    arg = 1, 2, 3
    t0 = time.clock()
    for i in iters:
        star(*arg)
    t1 = time.clock()
    return t1 - t0

def time_kw(iters=range(MANY)):
    def kw(a=0, b=0):
        return a * b

    dict = {'a': 1, 'b': -1}
    t0 = time.clock()
    for i in iters:
        kw(**dict)
    t1 = time.clock()
    return t1 - t0

def time_kw2(iters=range(MANY)):
    def kw(a=0, b=0, **d):
        return d.values()[a]

    d = {'a':1, 'c':2, 'd':3}
    t0 = time.clock()
    for i in iters:
        kw(**d)
    t1 = time.clock()
    return t1 - t0

def time_starkw(iters=range(MANY)):
    def func(a, b, c=None):
        return b

    t = (1,)
    d = {'c':1}
    t0 = time.clock()
    for i in iters:
        func(i, *t, **d)
    t1 = time.clock()
    return t1 - t0

class Test:
    def __init__(self, arg=None):
        pass

def time_init(iters=range(MANY)):
    constructor = Test
    t0 = time.clock()
    for i in iters:
        constructor(arg=i)
    t1 = time.clock()
    return t1 - t0
    
def median(x):
    x.sort()
    return x[len(x)/2]

sum = 0.0
for func in (
    time_arg1, time_arg2, time_arg3, time_fact, time_meth,
    time_umeth, time_builtin, time_callable, time_keyword,
    time_star, time_kw, time_kw2, time_starkw, time_init,
    ):  
    times = [func() for i in range(9)]
    med = median(times)
    print func.func_name, med, min(times), max(times)
    sum += med
print "total", sum
    

--QTUWfOmTKZ--


From jeremy@alum.mit.edu  Thu Nov  9 00:25:11 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:25:11 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <14857.60533.766291.786182@bitdiddle.concentric.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
 <14857.60533.766291.786182@bitdiddle.concentric.net>
Message-ID: <14857.61159.339908.704603@bitdiddle.concentric.net>

Looks like I jumped the gun with my last message.  A trivial change to
the logic prevented the keyword arguments slowdown.  I've revised the
SF patch.  The new numbers show that the patch is just a tad faster on
the benchmark.  (And the difference between gcc -O2 and -O3 makes a
big difference for this ceval-intensive benchmark.)

Jeremy

time_arg1 0.1 0.09 0.12
time_arg2 0.1 0.1 0.12
time_arg3 0.1 0.1 0.12
time_fact 0.12 0.11 0.13
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.09 0.08 0.1
time_callable 0.36 0.34 0.41
time_keyword 0.13 0.13 0.14
time_star 0.12 0.12 0.13
time_kw 0.23 0.22 0.24
time_kw2 0.64 0.61 0.66
time_starkw 0.24 0.23 0.26
time_init 0.64 0.64 0.69
total 3.08


From tim.one@home.com  Thu Nov  9 08:44:07 2000
From: tim.one@home.com (Tim Peters)
Date: Thu, 9 Nov 2000 03:44:07 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>

[Guido]

[examples of presumably fuzzy semantics in the presence of continuations]

> ...
> But now consider this:  the "evaluation stack" contained in each
> frame could easily be replaced by a bunch of temporary variables,
> if we had a slightly different instruction set ...
> Then would we save those temporary variables or not?

Yes, provided they remained anonymous (to the user) they're simply internal
implementation details, and continuations preserve all things "like that".
It's only visible name bindings that continuations let change across
resumptions.

> it can make a difference!

Indeed yes.

> ...
> But if the compiler were to use temporary variables instead of the
> evaluation stack, they might not have been restored!

The technical term for that would be "bug" <0.5 wink>.

Christian already covered the for-loop example.  A more interesting
variation is a for loop without a named "indexing vrbl":

    for x in sequence:
        etc

Scheme has nothing direct to say about this because Scheme has no loops.
Writing it as a recursive function instead leads to the same kind of result,
though.

> ...
> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.

It's clearer in Scheme because Scheme has fewer "primitive concepts" than
Python.

> (I've been told that the exact semantics of continuations in Scheme
> differ greatly between Scheme implementations.)

I believe you've been told that, but not by a clueful Scheme user!
Continuations are rigorously well-defined in Scheme.  What *isn't*
well-defined in Scheme is order of evaluation in many cases, so an
expression like

   (+ (f) (g))

can display wildly different behavior across implementations if one or both
of the functions {f, g}, directly or indirectly, creates or invokes a
continuation (say that g does:  it's not defined whether f has been invoked
by the time g is -- indeed, it's even OK to invoke f and g in parallel).

Note:  the Python eye sees something like that and leaps to the rule "OK,
chowderhead, so don't muck with continuations in the middle of
expressions!".  What that misses is that *everything* in Scheme is "an
expression".  Scheme implementations do "differ greatly" in *this* respect.
BTW, note that Scheme implementations can also display wildly different
behavior if f and g merely have side effects (i.e., this really has nothing
to do with continuations specifically:  they're just another form of
side-effect you can't always predict without knowing the order of evaluation
first).

[skipping the proposal because we talked about it instead]

[Gordon McMillan]
> ...
> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

I don't know of any comprehensible application of continuations that can't
be done without them.  The real appeal of continuations is in their
theoretical elegance (they're a single mechanism that can be used to build
all sorts of stuff, much as all of "if" and "while" and "for" can be built
out of compares and gotos -- continuations can be used to implement all of
calls, non-resumable and resumable exceptions, generators, coroutines,
non-deterministic evaluation, "thread like" stuff, ...).  In any specific
case, though, you want to live at the higher level, not at the raw
continuation level.

WRT a backtracking parser, note that this is what Icon *lives for*, and
generators alone suffice (and are indeed very pleasant) for natural
expression of that task.  Icon became what it is when Griswold decided to
open up the backtracking pattern-matching engine underlying SNOBOL4, and
make it the basis for all expression evaluation.  It took him 3 full
languages (SL5 and Rebus came before Icon) and 20 years to get this right.

Building a backtracking parser directly out of continuations sounds to me
mostly painful.  Building generators out of continuations *is* painful (I've
done it).  Curiously, the symmetry of coroutines appears to make building
them out of continuations easier (than building generators).

I'm not in love w/ continuations:  I *could* be if Guido got Continuation
Religion and wanted to redo exceptions and calls on top of continuations
too, but given that whatever happens here is destined (Christian will say
"doomed" <wink>) to co-exist with everything that's already here, the appeal
of continuations is minimal.  I've got no desire to play with novel new
control structures in Python (note that I don't consider generators-- or
even coroutines --to be "novel", not after they've been in multiple
languages for more than 30 years), and Python doesn't have the syntactic
flexibility that makes such experiments *pleasant* in Scheme anyway.  So
it's enough for me if Python supports the handful of new control-flow
gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
if we don't need continuations for those, fine by me.  BTW, I'd also like to
pickle a pure-Python computation in mid-stream, save it to disk, and resume
it later after a reboot (or on another machine!); we don't need
continuations for that either (although we do need much of what Stackless
does).

switching-from-redefining-truth-to-redefining-falsehood-ly y'rs  - tim



From tismer@tismer.com  Thu Nov  9 11:39:48 2000
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 13:39:48 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>
Message-ID: <3A0A8D04.3881E4FC@tismer.com>


Tim Peters wrote:
...

> Building a backtracking parser directly out of continuations sounds to me
> mostly painful.  Building generators out of continuations *is* painful (I've
> done it).  Curiously, the symmetry of coroutines appears to make building
> them out of continuations easier (than building generators).

Some things work very well, built with plain continuations.
See the attached ICON-style generator/backtrack framework
(going to post the 8 queens puzzle, soon).

> I'm not in love w/ continuations:  I *could* be if Guido got Continuation
> Religion and wanted to redo exceptions and calls on top of continuations
> too, but given that whatever happens here is destined (Christian will say
> "doomed" <wink>) to co-exist with everything that's already here, the appeal
> of continuations is minimal.  I've got no desire to play with novel new
> control structures in Python (note that I don't consider generators-- or
> even coroutines --to be "novel", not after they've been in multiple
> languages for more than 30 years), and Python doesn't have the syntactic
> flexibility that makes such experiments *pleasant* in Scheme anyway.

That's a very good point. Tricking Python to make continuations
useful is a pain in the a** and has led me to a quite weird API.
After the "sane" constructs are defined well, there is no much
reason to support continuations in the first place.

> So it's enough for me if Python supports the handful of new control-flow
> gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
> if we don't need continuations for those, fine by me.  BTW, I'd also like to
> pickle a pure-Python computation in mid-stream, save it to disk, and resume
> it later after a reboot (or on another machine!); we don't need
> continuations for that either (although we do need much of what Stackless
> does).

There is one application of continuations which I still consider
worthy. I'm shure that many people find it incredibly ugly.
Using continuations, I can build method-like functions
without classes and instances, which perform incredibly
fast. This cannot be done with simple one-shot continuations;
of course a class method would do the same, but slower:

<script labguage="python">
function expensive_to_init_and_cheap_to_call(*args):
    pass # initialize many local variables
    param = continuation.return_current() # ***
    return do_cheap_calculation(param)

# generate and initialize such a function    

fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

# now we are at ***

fastfunc(42)  # can call it many times, at high speed.
</script>

This is a function with expensive initialization and many
local variables involved. After initializing, the
continuation of *** is returned as a callable object.
All initialization is done, all locals are set, and
now we can pull out many results by repeatedly calling
this continuation.
This cannot be modelled as efficiently today with classes.

ciao - chris

p.s.: Here the simple ICON-like generator/backtrack framework.



---------------------------------------------------------------------

import continuation

class control:
    """ ICON style generators """
    
    def __init__(self):
        # the chain of alternatives is a stack of tuples
        self.more = None

    def suspend(self, value):
        """ return a value, but keep the caller for re-use """
        # push the caller on the alternatives stack
        self.more = (continuation.caller(), self.more)
        # simulate a return of the caller with the current value
        continuation.caller(2)(value)

    def fail(self):
        """ restart an alternative, if there is some.
        Otherwise do nothing """
        if self.more:
            back, self.more = self.more
            back()

    def clear(self):
        """ clear alternatives stack """
        self.more = None

    def asn(self, old, val):
        """ an undoable assignment. Usage:
        var = c.asn(var, value)
        Like the ICON operator "<-"
        """
        self.suspend(val)
        print "asn returning"
        return(old)

    def choice(self, *args):
        """ iterator over a fixed sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        for val in args[:-1]:
            self.suspend(val)
        return args[-1]

    # the above works only for sequences of known size.
    # The version below is better since it does not need to
    # know the size, but it has to do a look-ahead.
    
    def choice(self, *args):
        """ iterator over a general sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        
        # problem: how do we *return* the last element for any sequence?
        # solution: do a look-ahead by 1
        first = 1
        for nextval in args:
            if first:
                val = nextval
                first = 0
                continue
            self.suspend(val)
            val = nextval
        return val

    def to_by(self, a, b=None, c=None):
        """ a loop construct, modelled after the ICON
        for .. to .. by expression, but using xrange style
        """
        if c is None:
            if b is None:
                iterator = xrange(a)
            else:
                iterator = xrange(a,b)
        else:
            iterator = xrange(a,b,c)
        # problem: how do we *return* the last element?
        # solution: splitting the xrange is handy!
        for i in iterator[:-1]:
            self.suspend(i)
        return iterator[-1]

    # trying to get *slightly* more structure, here an
    # attempt to introduce something like "every".

---------------------------------------------------------------------

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From tismer@tismer.com  Thu Nov  9 13:04:00 2000
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:04:00 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com> <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <3A0AA0C0.EA9C137B@tismer.com>


Christian Tismer wrote:

I wrote:
> There is one application of continuations which I still consider
> worthy. I'm shure that many people find it incredibly ugly.
> Using continuations, I can build method-like functions
> without classes and instances, which perform incredibly
> fast. This cannot be done with simple one-shot continuations;
> of course a class method would do the same, but slower:
> 
> <script language="python">
> function expensive_to_init_and_cheap_to_call(*args):
>     pass # initialize many local variables
>     param = continuation.return_current() # ***
>     return do_cheap_calculation(param)
> 
> # generate and initialize such a function
> 
> fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

But this is again no argument for continuations, since
generators provide an equally performant solution:
(assuming a generator implementation, which passes the
generator object as first argument to itself. The
initialization part ends at self.start(), which
leaves the generator frame as is, and acts as an
entry point.)

function easily_done_with_generators(self, *args):
    pass # initialize many local variables
    param = self.start()
    while 1:
        result = do_cheap_calculation(param)
        param = self.suspend(result)

fastfunc = generator(easily_done_with_generators, manyargs)

fastfunc(42) # ...

----------------------

This implementation works without continuations and gives
equally fast generators.

Note the self.start() method: It is equivalent to the
continuation.return_current() function:
Leave the current frame as its own result, and provide
an entry point which accepts a value.

self.suspend() returns from the frame without ending it.
It returns a value, but preserves the entry point.

Does anybody know of a useful example where continuations
are really needed? The only remaining idea is at the moment
modelling of a case construct, but this reason is easily
removed by adding a switch statement to Python. :-)

cheers - chris   (stackless and continuationless)

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  9 14:07:14 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:07:14 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA0C0.EA9C137B@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>

On Thu, 9 Nov 2000, Christian Tismer wrote:

> Does anybody know of a useful example where continuations
> are really needed?

Well, it may be a bit unpythonic (for several reasons), but restartable
exceptions AFAICS cannot be implemented by generators or coroutines.
Restartable exceptions are not always appropriate, but tend to be
a pain to simulate when they are needed.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From tismer@tismer.com  Thu Nov  9 13:26:17 2000
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:26:17 +0200
Subject: [Python-Dev] uthread strawman
References: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>
Message-ID: <3A0AA5F9.67C782E2@tismer.com>


Moshe Zadka wrote:
> 
> On Thu, 9 Nov 2000, Christian Tismer wrote:
> 
> > Does anybody know of a useful example where continuations
> > are really needed?
> 
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

(sorry for posting to python-dev/null but Starship is still down)

How would restartable exceptions work? Like so?

try:     # here, a continuation is saved
    pass # some operations which may fail,
         # raising something that inherits from RestartableException

except RestartableException:
    pass # repair the condition
    sys.try_again()   # go back to the try statement

Probably not, since this would be doable by just an internal
jump operation.
But if the restartable exception were a callable object,
continuations might be necessary, since we now have a fork
of two concurrently existing execution paths in the frame:
We might continue with the exception handling but pass the
restartable to someone else, who tries to call it later.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov  9 14:38:30 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:38:30 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA5F9.67C782E2@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091627510.5020-100000@sundial>

[Christian Tismer]
> Does anybody know of a useful example where continuations
> are really needed?

[Moshe Zadka]
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

[Christian Tismer]
> How would restartable exceptions work? Like so?
> 
> try:     # here, a continuation is saved
>     pass # some operations which may fail,
>          # raising something that inherits from RestartableException
> 
> except RestartableException:
>     pass # repair the condition
>     sys.try_again()   # go back to the try statement

I imagined something like

try:
	raise RestartableException, continuation.current_continuation()
except RestartableException, ex:
	# correct the problem
	ex.saved_continuation()

IOW, just put the continuation in the exception object.
I don't think you can use generators/coroutines to simulate something
like this
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Thu Nov  9 15:30:03 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 10:30:03 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 16:38:30 +0200."
 <Pine.GSO.4.10.10011091627510.5020-100000@sundial>
References: <Pine.GSO.4.10.10011091627510.5020-100000@sundial>
Message-ID: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>

[Moshe]
> I imagined something like
> 
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
> 
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

You can make the exception handler into a separate coroutine -- then
no reusable continuations are needed.  The coroutine can repair the
exception and resume the other.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Thu Nov  9 16:02:31 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 9 Nov 2000 17:02:31 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
Message-ID: <00d801c04a66$7828bff0$0900a8c0@SPIFF>

> Remove AC_C_INLINE test from configure.in, since the only place the symbol
> occurs in the Python sources appears to be as text in comments.  We do not
> want to interfere with C++ keywords!

did you benchmark SRE before and after this change?

</F>



From fdrake@acm.org  Thu Nov  9 16:16:04 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Nov 2000 11:16:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: <00d801c04a66$7828bff0$0900a8c0@SPIFF>
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
 <00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <14858.52676.775791.32394@cj42289-a.reston1.va.home.com>

Fredrik Lundh writes:
 > did you benchmark SRE before and after this change?

  No, but I don't see how this change would affect SRE.  _sre looks
like it takes care of itself with regard to inlining; the only thing
you might lose is:

	#undef inline

  Am I misreading something?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fredrik@effbot.org  Thu Nov  9 20:00:59 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Thu, 9 Nov 2000 21:00:59 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <00c101c04a87$c8d497f0$3c6340d5@hagrid>

I wrote:


> > Remove AC_C_INLINE test from configure.in, since the only place the symbol
> > occurs in the Python sources appears to be as text in comments.  We do not
> > want to interfere with C++ keywords!
> 
> did you benchmark SRE before and after this change?

following up on myself: SRE uses inline only if USE_INLINE is enabled,
and it's appears to be disabled in the 2.0 code base, for all platforms
except Windows (don't recall doing this; maybe someone else thought
that 10-20% performance hit was good enough?).  and now that Fred
has removed portable inline support, I cannot switch it on again :-(

besides, do people really expect to be able to use one compiler when
running configure, and another to compile Python?  sounds like they're
asking for it...

</F>



From billtut@microsoft.com  Thu Nov  9 20:28:41 2000
From: billtut@microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 12:28:41 -0800
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com>

Christian Tismer wrote:
> Does anybody know of a useful example where continuations
> are really needed? 

Well, I don't know about needed, but let me explain a possible sever
architecture, and then see what new fangled control structure could help it
become more manageable. (irrespective if the perf #s of such an architecture
would actually be worth the architecture overhead in current CPython)

In a high performance server, any non-CPU intensive operation which blocks
the thread you're on reduces your scalability potential.

A fairly common way of writing multi-threaded servers is to have one client
per thread , whether this is either a thread pool, or just a simplistic
"Create thread, execute work, end thread" approach.

Threads are very expensive, and increases the context switch penalty your
server is inflicted with.

An alternative which reduces the context switch penalty dramatically is to
use a thread safe work item queue, and N threads where N is usually some
small multiple of the # of CPUs, and is < than the thread pool # of threads.
The solution to avoid these threads from blocking is an asynchronous state
machine. You asynchronously start the long-time operation, and on completion
of this operation insert the request state back in the thread safe queue.

You can further extend this idea to reduce the # of threads that you have so
that you have only 1 thread/CPU. Each thread being bound directly to the
CPU, and not allowed to run on other CPUs.
This tries to prevent the threads from CPU switching and ruining the CPU
cache, etc.
A work item must be able to be executed on any of these threads.

Another extension of this idea is to bundle these work items into separate
queues based on the state in the state machine.
The reason for doing this is trying to prevent unnecessary CPU cache
flushing.

The downside of this approach is that asynchronous state machines are a pain
to debug, maintain, understand, write, etc...
(BTW for the curious, the above architecture does allow you to achieve much
higher perf for certain tasks than other ways of handling the code, it's
been tested and used extensively in some internal C/C++ code (not mine))

The thought occurs to me that continuations would definitely help in this
situation. 
* You'd have more debugging state
* The code is organized around other needs besides boundaries between
blocking operations.

But it's not clear to me (mostly because I haven't applied a lot of thought
about it) if coroutines would suffice here.

Thoughts?

Bill



From loewis@informatik.hu-berlin.de  Thu Nov  9 21:11:00 2000
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Thu, 9 Nov 2000 22:11:00 +0100 (MET)
Subject: [Python-Dev] [Python-Help] Driver?????
Message-ID: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>

> My name is Kelli Collins, and I work for the Data Management Team at
> IBM.  I am currently working with a customer that is considering
> using DB2 as their database software, and has asked me to see if I
> can locate a driver for Python and DB2...... can you tell if one
> exists??????

Hi Kelli,

I believe there is actually more than one way to get a database module
for Python and DB2. Please have a look at the DB SIG pages, at

  http://www.python.org/topics/database/

You'll see a DB/2 module being maintained at

  ftp://people.linuxkorea.co.kr/pub/DB2/

In addition, since DB2 offers the CLI, and an ODBC driver, you can
also use the ODBC module. For that, you have two options again: on
Windows, you can use the ODBC module that is in PythonWin. On all
systems, you can use mxODBC (make sure the licensing is acceptable,
though).

In any case, all of these comply with the DB API, so application code
should port easily among these options. I guess the difference then is
ease-of-installation, performance, and access to advanced features;
since I don't use DB2 myself, I can't recommend a particular choice.

Good luck,
Martin


From guido@python.org  Thu Nov  9 21:19:21 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 16:19:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: Your message of "Thu, 09 Nov 2000 21:00:59 +0100."
 <00c101c04a87$c8d497f0$3c6340d5@hagrid>
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>
 <00c101c04a87$c8d497f0$3c6340d5@hagrid>
Message-ID: <200011092119.QAA32327@cj20424-a.reston1.va.home.com>

> following up on myself: SRE uses inline only if USE_INLINE is enabled,
> and it's appears to be disabled in the 2.0 code base, for all platforms
> except Windows (don't recall doing this; maybe someone else thought
> that 10-20% performance hit was good enough?).  and now that Fred
> has removed portable inline support, I cannot switch it on again :-(

Can you explain yourself better, Fredrik?  I don't think that
USE_INLINE was ever defined.  I'd gladly accept a patch to
configure.in that defines USE_INLINE if 'inline' seems accepted by the
compiler.

> besides, do people really expect to be able to use one compiler when
> running configure, and another to compile Python?  sounds like they're
> asking for it...

Yes!  C++ uses include the same headers that were used to compile
Python, and C++ definitely qualifies as a different compiler...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg@cosc.canterbury.ac.nz  Thu Nov  9 22:30:08 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Nov 2000 11:30:08 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <200011092230.LAA00785@s454.cosc.canterbury.ac.nz>

Christian Tismer <tismer@tismer.com>:

> Using continuations, I can build method-like functions
> without classes and instances

Couldn't you achieve the same thing with a closure, if 
Python acquires nested scopes?

My first attempt at implementing classes in Scheme
used continuations, until I realised that closures
were all that you needed.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Thu Nov  9 22:54:16 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 17:54:16 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 12:28:41 PST."
 <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com>
References: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com>
Message-ID: <200011092254.RAA04553@cj20424-a.reston1.va.home.com>

> Christian Tismer wrote:
> > Does anybody know of a useful example where continuations
> > are really needed? 

[Bill Tutt]
> Well, I don't know about needed, but let me explain a possible sever
> architecture, and then see what new fangled control structure could help it
> become more manageable. (irrespective if the perf #s of such an architecture
> would actually be worth the architecture overhead in current CPython)
[...]
> But it's not clear to me (mostly because I haven't applied a lot of thought
> about it) if coroutines would suffice here.

Indeed, this kind of server architecture is typically done with
coroutines -- it's not a good argument for the full power of
continuations. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal@lemburg.com  Thu Nov  9 23:30:20 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 10 Nov 2000 00:30:20 +0100
Subject: [Python-Dev] [Python-Help] Driver?????
References: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>
Message-ID: <3A0B338C.D1DD7CB2@lemburg.com>

Martin von Loewis wrote:
> 
> > My name is Kelli Collins, and I work for the Data Management Team at
> > IBM.  I am currently working with a customer that is considering
> > using DB2 as their database software, and has asked me to see if I
> > can locate a driver for Python and DB2...... can you tell if one
> > exists??????
> 

FYI, the next version of mxODBC will support DB/2 on all
platforms. It won't be freeware anymore, but in return you'll
get better support for Unicode and other ODBC 3.5 features.

The next round of mx.Modules will be published in the first
quarter of 2001. Until then you can use the existing mxODBC 1.1.1
which will work nicely on Windows and with some effort on Unix
for DB/2 too -- it'll give you a consistent interface to rely
on fo Windows and all supported Unix platforms.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From billtut@microsoft.com  Fri Nov 10 00:30:20 2000
From: billtut@microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 16:30:20 -0800
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>

Guido van Rossum [mailto:guido@python.org] wrote:
> > Christian Tismer wrote:
> > > Does anybody know of a useful example where continuations
> > > are really needed? 
> [Bill Tutt]
> > Well, I don't know about needed, but let me explain a possible sever
> > architecture, and then see what new fangled control structure could help
it
> > become more manageable. (irrespective if the perf #s of such an
architecture
> > would actually be worth the architecture overhead in current CPython)
> [...]
> > But it's not clear to me (mostly because I haven't applied a lot of
thought
> > about it) if coroutines would suffice here.

> Indeed, this kind of server architecture is typically done with
> coroutines -- it's not a good argument for the full power of
> continuations. :-)

Hrm, but can I get enough info from the coroutine support to
automatically determine which "work item queue" the paused routine
should go into after the "blocking" operation completes?
Additionally, the coroutine can't be stuck to only one system
thread. That would be bad.

If the above can't be done with coroutines, then I'm either stuck with
an async state machine, or something that might not scale as well as
it could. :(

(Ignoring the interesting discussion about how exactly you make this
architecture portable, since AFAIK only NT lets you bind threads to
CPUs)

I wish I had the time to play with stackless and hack up something
like I mentioned. The performance benefits to async state machine C++ code
that
use this idea are pretty impressive. CPU cache misses hurt. :(

Bill


From tismer@tismer.com  Fri Nov 10 11:39:24 2000
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 10 Nov 2000 13:39:24 +0200
Subject: [Python-Dev] uthread strawman
References: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>
Message-ID: <3A0BDE6C.1712F87E@tismer.com>


Bill Tutt wrote:
> 
> Guido van Rossum [mailto:guido@python.org] wrote:
> > > Christian Tismer wrote:
> > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > [Bill Tutt]

Bill on [server architecture, async state machine]

> > Indeed, this kind of server architecture is typically done with
> > coroutines -- it's not a good argument for the full power of
> > continuations. :-)
> 
> Hrm, but can I get enough info from the coroutine support to
> automatically determine which "work item queue" the paused routine
> should go into after the "blocking" operation completes?
> Additionally, the coroutine can't be stuck to only one system
> thread. That would be bad.

Coroutine vs. continuation isn't the point here, I think.
What you would likely need is gathering information about
the frames and variables in your co-whatever. This is quite
simple to implement, with either architecture.
My next version will not only allow to inspect the eval
stack of a frame, but also inspection and modification
of local variables; still this is no continuation feature.
We can simply inspect anything.

What makes the difference is callability:
Whenever we expose arbitrary frames as callable objects,
or we provide an unrestricted other way to execute them,
then exactly all the problems and consequences of
continuations pop up: we need to clone execution state,
since now it is possible to have more than one state for
a frame.
If we avoid this, we can still have all control and debugging
features with coroutines.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From billtut@microsoft.com  Fri Nov 10 19:49:40 2000
From: billtut@microsoft.com (Bill Tutt)
Date: Fri, 10 Nov 2000 11:49:40 -0800
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B7@red-msg-07.redmond.corp.microsoft.com>


From: 	Christian Tismer [mailto:tismer@tismer.com] 

> Bill Tutt wrote:
> > 
> > Guido van Rossum [mailto:guido@python.org] wrote:
> > > > Christian Tismer wrote:
> > > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > > [Bill Tutt]

> Bill on [server architecture, async state machine]

> > > Indeed, this kind of server architecture is typically done with
> > > coroutines -- it's not a good argument for the full power of
> > > continuations. :-)
> 
> > Hrm, but can I get enough info from the coroutine support to
> > automatically determine which "work item queue" the paused routine
> > should go into after the "blocking" operation completes?
> > Additionally, the coroutine can't be stuck to only one system
> > thread. That would be bad.

> Coroutine vs. continuation isn't the point here, I think.
> What you would likely need is gathering information about
> the frames and variables in your co-whatever. This is quite
> simple to implement, with either architecture.
> My next version will not only allow to inspect the eval
> stack of a frame, but also inspection and modification
> of local variables; still this is no continuation feature.
> We can simply inspect anything.

Coolness. 

> What makes the difference is callability:
> Whenever we expose arbitrary frames as callable objects,
> or we provide an unrestricted other way to execute them,
> then exactly all the problems and consequences of
> continuations pop up: we need to clone execution state,
> since now it is possible to have more than one state for
> a frame.
> If we avoid this, we can still have all control and debugging
> features with coroutines.

Ah, well this definitely would cause all sorts of confusing things.
I did mention I hadn't spent much time thinking about all of the
implementation details, I just know the requirements the architecture idea
has. :) As long as the co-whatevers can migrate to any system thread, I'm
one happy camper.

Thanks for helping clear stuff up folks,
Bill



From tim_one@email.msn.com  Sat Nov 11 09:47:10 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Sat, 11 Nov 2000 04:47:10 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEMOHPAA.tim_one@email.msn.com>

[Moshe]
> I imagined something like
>
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
>
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

[Guido]
> You can make the exception handler into a separate coroutine -- then
> no reusable continuations are needed.  The coroutine can repair the
> exception and resume the other.

Since I'm not a fan of resumable exceptions, I won't beat it to death.

Note that there are at least three meanings for "resumable exception" in
actual use (and I believe, e.g., that modern (ANSI) Smalltalk supports all
three):

1. Handler returns a value, and the failing operation acts as if that were
what it had computed from the start.  Very common in HW (e.g., a page fault
on a memory load acts this way; ditto 1e300**2 returning +Inf under 754
double arithmetic when the overflow signal is masked out).  Note that no
lexically enclosing "try block" is needed in this case (it is in the next
two, although they don't need "except" clauses attached to their lexically
enclosing "try").

2. Handler tells the entire originating "try block" to start over.

3. Handler resumes right after the originating try block.

In all of these, you have to search "up the call stack" for a handler, which
then wants to resume as if the call stack had not been unwound (albeit that
in #2 and #3, wants to resume at different places than where the exception
was raised!).  "Up the call stack" means within your current thread of
control, so it's hard to see how to make an independent coroutine out of
that on demand (the handler needs to work in its natural lexical context,
incl. modifying values of *its* scope's locals that will be visible again
when the deeper routine with the exception eventually does normal returns
back to it).

The details are excruciating -- so it's a good thing nobody in their right
mind wants any of these <wink>.

relievedly y'rs  - tim




From mal@lemburg.com  Sat Nov 11 09:52:27 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 10:52:27 +0100
Subject: [Python-Dev] Help wanted...
Message-ID: <3A0D16DB.A8724B9B@lemburg.com>

Hi everybody,

I just wanted to inform you that I won't be able to work on the
patches needed to fix the bugs assigned to me for the next month.
I currently have to finish a huge project with a deadline in mid-
December which has high priority (anyone ever write a B2B
eCommerce order solution in one month ?) ... I hope Python
get's me there in time.

Anyway, if any of the bugs currently assigned to me need a
fast fix, I'd suggest someone else taking a look at them, since
I won't be able to do much about these until around Christmas.
If you need help about some Unicode detail, just drop me line.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sat Nov 11 14:05:38 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 15:05:38 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <3A0D5232.CF6A884@lemburg.com>

I am looking for a way to do DNS resolution with a timeout
on the lookup. The easy way to do these lookups using the
socket module often result in processes hanging for several
seconds (up to minutes on Windows), so I figured emulating the
standard DNS lookup (including the /etc/hosts and /etc/resolv.conf
files) would be a good idea.

I know of the Demo/dns/, but I'm unsure whether it can be
tweaked to allow timeouts.

Are there any other such implementations out there ?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From thomas@xs4all.net  Sat Nov 11 14:23:58 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 11 Nov 2000 15:23:58 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: <3A0D5232.CF6A884@lemburg.com>; from mal@lemburg.com on Sat, Nov 11, 2000 at 03:05:38PM +0100
References: <3A0D5232.CF6A884@lemburg.com>
Message-ID: <20001111152358.O27208@xs4all.nl>

On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:

> I am looking for a way to do DNS resolution with a timeout on the lookup.
> [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> allow timeouts.

> Are there any other such implementations out there ?

Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
timeouts, but it does do async lookups ;) He posted this a couple of days
ago, to the python list:

---------------------------------------------------------------------------
From: Anthony Baxter <anthony@interlink.com.au>
To: support@internetdiscovery.com (Mike Clarkson)
Cc: python-list@python.org
Subject: Re: Async DNS? 

The python DNS library I released a while ago (based on Guido's code)
did this. I just had a poke around and realised that it's not visible
anywhere on the net - I just dropped a copy into

http://www.zope.org/Members/anthony/software/dns.tar.gz
(sorry DC guys for abusing the zope.org site - to make it fair, I'll
whip up a DNS zope product at some point :)

I haven't looked at it in an age or two, but it used to work fine. :)

Anthony

>>> Mike Clarkson wrote
> Does anyone have asynchronous DNS lookup for Python?
> 
> Presumably it could be done in Python using sockets, but someone
> must have done a C implementation.
> 
> Many thanks in advance,
> 
> Mike.
> -- 
> http://www.python.org/mailman/listinfo/python-list
> 

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.
---------------------------------------------------------------------------

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Sat Nov 11 18:15:52 2000
From: guido@python.org (Guido van Rossum)
Date: Sat, 11 Nov 2000 13:15:52 -0500
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: Your message of "Sat, 11 Nov 2000 15:05:38 +0100."
 <3A0D5232.CF6A884@lemburg.com>
References: <3A0D5232.CF6A884@lemburg.com>
Message-ID: <200011111815.NAA12217@cj20424-a.reston1.va.home.com>

> I know of the Demo/dns/, but I'm unsure whether it can be
> tweaked to allow timeouts.

Hm, typically the protocol used is UDP, and it's easy to set timeouts
there: just use select on the socket.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@loewis.home.cs.tu-berlin.de  Sat Nov 11 23:58:12 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 12 Nov 2000 00:58:12 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>

> Are there any other such implementations out there ?

How about http://dustman.net/andy/python/adns-python?

Regards,
Martin

P.S. Is this really on-topic for python-dev?


From moshez@zadka.site.co.il  Sun Nov 12 20:03:00 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Sun, 12 Nov 2000 22:03:00 +0200
Subject: [Python-Dev] Weak References
Message-ID: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>

For those of you who do not read c.l.py

From: shindich@my-deja.com
Newsgroups: comp.lang.python
Subject: Weak References
Date: Sun, 12 Nov 2000 11:07:32 GMT

Ever wanted to have weak references in Python?
Pre-alpha release of PyWeakReference project is now available on Source
Forge (http://sourceforge.net/projects/pyweakreference/)
Documentation is coming soon... (some docs are available in the Doc
subdirectory of the zip file)


Sent via Deja.com http://www.deja.com/
Before you buy.
- -- 
http://www.python.org/mailman/listinfo/python-list



------- End of Forwarded Message

-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From mal@lemburg.com  Sun Nov 12 12:55:42 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:55:42 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>
Message-ID: <3A0E934E.8BC64ED3@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Are there any other such implementations out there ?
> 
> How about http://dustman.net/andy/python/adns-python?

Thanks for the hint.
 
> Regards,
> Martin
> 
> P.S. Is this really on-topic for python-dev?

Not really... but I thought since this question often arises
when using sockets, it might be a good idea to have something
like dnslib in the standard lib together with some support
for doing DNS lookups with predefined timeouts.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sun Nov 12 12:58:44 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:58:44 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <200011111815.NAA12217@cj20424-a.reston1.va.home.com>
Message-ID: <3A0E9404.68BD77D0@lemburg.com>

Guido van Rossum wrote:
> 
> > I know of the Demo/dns/, but I'm unsure whether it can be
> > tweaked to allow timeouts.
> 
> Hm, typically the protocol used is UDP, and it's easy to set timeouts
> there: just use select on the socket.

Hmm, yes that would be a possibility. I'll see if I can come
up with some code which implements this.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sun Nov 12 13:00:05 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:00:05 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <20001111152358.O27208@xs4all.nl>
Message-ID: <3A0E9455.83AE479E@lemburg.com>

Thomas Wouters wrote:
> 
> On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:
> 
> > I am looking for a way to do DNS resolution with a timeout on the lookup.
> > [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> > allow timeouts.
> 
> > Are there any other such implementations out there ?
> 
> Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
> timeouts, but it does do async lookups ;) He posted this a couple of days
> ago, to the python list:

I did know of that lib -- but it seems a bit over the top for the
simple lookups I hav in mind.
 
> ---------------------------------------------------------------------------
> From: Anthony Baxter <anthony@interlink.com.au>
> To: support@internetdiscovery.com (Mike Clarkson)
> Cc: python-list@python.org
> Subject: Re: Async DNS?
> 
> The python DNS library I released a while ago (based on Guido's code)
> did this. I just had a poke around and realised that it's not visible
> anywhere on the net - I just dropped a copy into
> 
> http://www.zope.org/Members/anthony/software/dns.tar.gz
> (sorry DC guys for abusing the zope.org site - to make it fair, I'll
> whip up a DNS zope product at some point :)
> 
> I haven't looked at it in an age or two, but it used to work fine. :)
> 
> Anthony
> 
> >>> Mike Clarkson wrote
> > Does anyone have asynchronous DNS lookup for Python?
> >
> > Presumably it could be done in Python using sockets, but someone
> > must have done a C implementation.
> >
> > Many thanks in advance,
> >
> > Mike.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sun Nov 12 13:17:33 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:17:33 +0100
Subject: [Python-Dev] Weak References
References: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <3A0E986D.43B170BA@lemburg.com>

Moshe Zadka wrote:
> 
> For those of you who do not read c.l.py
> 
> From: shindich@my-deja.com
> Newsgroups: comp.lang.python
> Subject: Weak References
> Date: Sun, 12 Nov 2000 11:07:32 GMT
> 
> Ever wanted to have weak references in Python?
> Pre-alpha release of PyWeakReference project is now available on Source
> Forge (http://sourceforge.net/projects/pyweakreference/)
> Documentation is coming soon... (some docs are available in the Doc
> subdirectory of the zip file)

Since more and more of these implementations are popping up
like mushrooms ;-) ... could you add some discussion about the
different methods of adding weak refs to Python to the PEP 205 ?

--
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Sun Nov 12 13:55:51 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Sun, 12 Nov 2000 15:55:51 +0200 (IST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011121554400.1528-100000@sundial>

On Sun, 12 Nov 2000, M.-A. Lemburg wrote:

> Since more and more of these implementations are popping up
> like mushrooms ;-) ... could you add some discussion about the
> different methods of adding weak refs to Python to the PEP 205 ?

Well, it's one of the few PEPs which are not yet mine <wink>:
I suppose Fred will do that pretty soon, though.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Mon Nov 13 12:54:43 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 07:54:43 -0500
Subject: [Python-Dev] Nested functions and class scope
In-Reply-To: Your message of "Mon, 13 Nov 2000 10:24:51 +1100."
 <3A0F26C3.7CD823BF@ozemail.com.au>
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>
 <3A0F26C3.7CD823BF@ozemail.com.au>
Message-ID: <200011131254.HAA22379@cj20424-a.reston1.va.home.com>

John Max Skaller wrote:
> Guido van Rossum wrote:
> ===============>>>>
> > Other issues that need sorting out:
> > 
> > x = 2
> > class C:
> >   x = 1
> >   C = 'some string'
> >   def a(self):
> >      print x
> >   def b(self):
> >      global x
> >      x = 3
> > 
> > class D(C):
> >   C = 'some string'
> >   def a(self):
> >      C.a(self)
> >      print C
> > 
> > o = C()
> > o.a()
> > o.b()
> > o.a()
> > 
> > o = D()
> > o.a()
> > 
> > What would the output look like under your proposal ?
> 
> This is a good point!  If we considered the class as a nested scope
> here, I think it might break too much code, plus it would allow a new
> coding style where you could reference class variables without a self
> or <classname> prefix.  I don't like that prospect, so I'm in favor
> for ruling this out.
> <<<===================
> 
> You're missing the point. you can refer to class variables without
> a prefix right now:
> 
> class X:
> 	x = 1
> 	y = x
> 
> 
> It would be madness NOT to support lexically scoped nested classes
> as well as functions. But note that these scopes only exist
> when a class is executed, that is, when the class is defined.
> This has NO impact on finding class variables from an instance.
> 
> output of above from Vyper:
> --------------------------------------------------------------
> Viperi 2.0.1
> Copyright Maxtal P/L, John Skaller, Australia, 1999
> >>>x = 2
> ...class C:
> ...  x = 1
> ...  C = 'some string'
> ...  def a(self):
> ...     print x
> ...  def b(self):
> ...     global x
> ...     x = 3
> ...
> >>>class D(C):
> ...  C = 'some string'
> ...  def a(self):
> ...     C.a(self)
> ...     print C
> ...
> >>>o = C()
> ...o.a()
> ...o.b()
> ...o.a()
> ...
> 1 
> 3 
> >>>o = D()
> ...o.a()
> ...
> Error Near line 4
> 
> Uncaught Python Exception at top level!!
>   .. Kind: Instance of AttributeError
>   .. Attributes:
>       args --> Cannot find attribute of "some string" "a"
> 
> Traceback (innermost first)
> File: <string> Line: 4
> <cannot get line from file>
> File: <string> Line: 2
> <cannot get line from file>
> 
> -- 
> John (Max) Skaller, mailto:skaller@maxtal.com.au
> 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
> checkout Vyper http://Vyper.sourceforge.net
> download Interscript http://Interscript.sourceforge.net

Looks like you are missing the point.  Of course I want to keep the
class as a local scope *inside the class def*.  But I need it out of
the way when executing methods.  The example code should print

2			# a() references global x, not C.x
3			# global x got set to three by b()
2			# C refers to global class C
<class C ...>		# ditto

I agree that this is surprising.  But I believe it would break too
much code if we implemented your (simpler) scheme.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Nov 13 14:09:46 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 13 Nov 2000 09:09:46 -0500 (EST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
References: <3A0E986D.43B170BA@lemburg.com>
 <Pine.GSO.4.10.10011121554400.1528-100000@sundial>
 <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <14863.63018.678116.930743@cj42289-a.reston1.va.home.com>

M.-A. Lemburg writes:
 > Since more and more of these implementations are popping up
 > like mushrooms ;-) ... could you add some discussion about the
 > different methods of adding weak refs to Python to the PEP 205 ?

Moshe Zadka writes:
 > Well, it's one of the few PEPs which are not yet mine <wink>:
 > I suppose Fred will do that pretty soon, though.

  I plan to do this sometime this week.  I have a bunch of text that
isn't already checked in, so this isn't the time to open up editorial
control just yet.
  The notice Moshe forwarded didn't slip past me, but I've been a
little under the weather the past few days.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From nas@arctrix.com  Mon Nov 13 08:36:53 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Mon, 13 Nov 2000 00:36:53 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001113003653.A13421@glacier.fnational.com>

I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil


From mal@lemburg.com  Mon Nov 13 15:32:04 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 13 Nov 2000 16:32:04 +0100
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
References: <20001113003653.A13421@glacier.fnational.com>
Message-ID: <3A100974.969045C0@lemburg.com>

Neil Schemenauer wrote:
> 
> I making this available now in the hope that it will generate
> some dicussion:
> 
>     http://arctrix.com/nas/python/coerce-2.diff
> 
> The patch is based on Marc-Andre Lemburg's coercion patch for
> 1.5.1 downloaded from the starship:
> 
>     http://starship.python.net/~lemburg/CoercionProposal.html
> 
> All credit goes to him.  The bugs are probably introduced by me.
> What the patch does:
> 
>   - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
>     set then the tp_as_number methods are treated differently.
>     Coercion is not done before passing arguments to the method.
>     New style methods do their own coercion or return the
>     NotImplemented singleton.  See Marc's page for more details.

Very nice !

I wanted to suggest the usage of a type flag
instead of the slot magic in the original patch too -- your time
machine was faster ;-)
 
>   - Decide on handling of nb_coerce slot (eg. should it be called
>     on new style numbers).

Probably not, since all coercion is now handled by the slots
themselves.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From trentm@ActiveState.com  Mon Nov 13 21:16:26 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:16:26 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011131948.LAA04697@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Mon, Nov 13, 2000 at 11:48:25AM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net>
Message-ID: <20001113131626.B4553@ActiveState.com>

On Mon, Nov 13, 2000 at 11:48:25AM -0800, Guido van Rossum wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv4689
> 
> Modified Files:
> 	fileobject.c 
> Log Message:
> Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> hack for TELL64()...  Sounds like there's something else going on
> really.  Does anybody have a clue I can buy?

Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
when it should not really be. Essentially, these platforms lie when they say
they support largefiles, at least according to the configure test for
HAVE_LARGEFILE_SUPPORT>

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From guido@python.org  Mon Nov 13 21:38:01 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 16:38:01 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:16:26 PST."
 <20001113131626.B4553@ActiveState.com>
References: <200011131948.LAA04697@slayer.i.sourceforge.net>
 <20001113131626.B4553@ActiveState.com>
Message-ID: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>

> > Log Message:
> > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > hack for TELL64()...  Sounds like there's something else going on
> > really.  Does anybody have a clue I can buy?
> 
> Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> when it should not really be. Essentially, these platforms lie when they say
> they support largefiles, at least according to the configure test for
> HAVE_LARGEFILE_SUPPORT>

Mh...  Thanks.  And how are we testing for large file support?
Perhaps the test should be improved rather than hacks piled on top of
each other?  What's wrong with the current test?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From trentm@ActiveState.com  Mon Nov 13 21:59:05 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:59:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 04:38:01PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>
Message-ID: <20001113135905.B7508@ActiveState.com>

On Mon, Nov 13, 2000 at 04:38:01PM -0500, Guido van Rossum wrote:
> > > Log Message:
> > > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > > hack for TELL64()...  Sounds like there's something else going on
> > > really.  Does anybody have a clue I can buy?
> > 
> > Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> > when it should not really be. Essentially, these platforms lie when they say
> > they support largefiles, at least according to the configure test for
> > HAVE_LARGEFILE_SUPPORT>
> 
> Mh...  Thanks.  And how are we testing for large file support?
> Perhaps the test should be improved rather than hacks piled on top of
> each other?  What's wrong with the current test?
> 

I agree: the test should improve. Here it is:

AC_MSG_CHECKING(whether to enable large file support)
if test "$have_long_long" = yes -a \
    "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
    "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
  AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
  AC_MSG_RESULT(yes)
else
  AC_MSG_RESULT(no)
fi


BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
(i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
think). In windows is is _tell64, I think. ...anyway there are a lot of ways
to spell it.

I don't know right away how to translate that into an appropriate configure
test. I.e. how do you test that the platform has an ftell/fseek-like function
that uses an index variable whose sizeof() is at least 8. 

Note that, IIRC, windows had some funny case where the test above would have
failed but that was okay because there were lower level I/O functions that
used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
details. 



Trent


-- 
Trent Mick
TrentM@ActiveState.com


From akuchlin@mems-exchange.org  Mon Nov 13 22:13:58 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 13 Nov 2000 17:13:58 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113135905.B7508@ActiveState.com>; from trentm@ActiveState.com on Mon, Nov 13, 2000 at 01:59:05PM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com>
Message-ID: <20001113171358.A7407@kronos.cnri.reston.va.us>

On Mon, Nov 13, 2000 at 01:59:05PM -0800, Trent Mick wrote:
>I don't know right away how to translate that into an appropriate configure
>test. I.e. how do you test that the platform has an ftell/fseek-like function
>that uses an index variable whose sizeof() is at least 8. 

If you look at the very latest autoconf alpha release, autoconf 2.49a,
at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
macro that may be what's needed.  Perhaps we can back-port this macro
to the current autoconf release.

--amk


From thomas@xs4all.net  Mon Nov 13 22:20:20 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:20:20 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113171358.A7407@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 13, 2000 at 05:13:58PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <20001113171358.A7407@kronos.cnri.reston.va.us>
Message-ID: <20001113232020.R27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:13:58PM -0500, Andrew Kuchling wrote:

> If you look at the very latest autoconf alpha release, autoconf 2.49a,
> at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
> macro that may be what's needed.  Perhaps we can back-port this macro
> to the current autoconf release.

Aahh, hmm. I'll look at what this macro does tomorrow, unless someone else
beats me to it. I've been busy, between gobs of work, trying to find out how
*other* applications determine large file support... that's suprisingly
tricky in some cases -- I still haven't figured out how zsh does it.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Nov 13 22:36:06 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:36:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:59:05 PST."
 <20001113135905.B7508@ActiveState.com>
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>
 <20001113135905.B7508@ActiveState.com>
Message-ID: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>

> > Mh...  Thanks.  And how are we testing for large file support?
> > Perhaps the test should be improved rather than hacks piled on top of
> > each other?  What's wrong with the current test?
> > 
> 
> I agree: the test should improve. Here it is:
> 
> AC_MSG_CHECKING(whether to enable large file support)
> if test "$have_long_long" = yes -a \
>     "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
>     "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
>   AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
>   AC_MSG_RESULT(yes)
> else
>   AC_MSG_RESULT(no)
> fi
> 
> 
> BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
> (i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
> posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
> think). In windows is is _tell64, I think. ...anyway there are a lot of ways
> to spell it.
> 
> I don't know right away how to translate that into an appropriate configure
> test. I.e. how do you test that the platform has an ftell/fseek-like function
> that uses an index variable whose sizeof() is at least 8. 
> 
> Note that, IIRC, windows had some funny case where the test above would have
> failed but that was okay because there were lower level I/O functions that
> used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
> details. 

After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
means that the low-level system calls (lseek(), stat() etc.)  support
large files, through an off_t type that is at least 8 bytes (assumed
to be equivalent with a long long in some places, given the use of
PyLong_FromLongLong() and PyLong_AsLongLong()).

But the problems occur in fileobject.c, where we're dealing with the
stdio library.  Not all stdio libraries seem to support long files in
the same way, and they use a different typedef, fpos_t, which may be
larger or smaller in size than off_t.

Aha!  It looks like the problem is that under a variety of
circumstances TELL64(fd) is required; but these have in common that
HAVE_LARGEFILE_SUPPORT is defined, and in all cases except MS_WIN64 it
is defined as equal to lseek((fd),0,SEEK_CUR).

So wouldn't the problem be solved if we changed the #ifdefs so that
that is the default definition, instead of only defining that for
specific platforms?  As more and more platforms develop kernel support
for 64-bit offsets but their stdio libraries lag behind (or use APIs
we don't know about, or assume that fsetpos() is sufficient), we'll
run into this more and more often.

Here's the current logic for defining TELL64() in fileobject.c:

#if defined(MS_WIN64)
#define TELL64 _telli64
#elif defined(__NetBSD__) || ... /* etc. */
/* NOTE: this is only used on older
   NetBSD prior to f*o() funcions */
#define TELL64(fd) lseek((fd),0,SEEK_CUR)
#endif

There's exactly one place where TELL64() is subsequently used, which
is inside

#elif defined(HAVE_LARGEFILE_SUPPORT) && SIZEOF_FPOS_T >= 8
#else

Any objections???

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Mon Nov 13 22:41:21 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:41:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:36:06PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>
Message-ID: <20001113234121.S27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:

> After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> means that the low-level system calls (lseek(), stat() etc.)  support
> large files, through an off_t type that is at least 8 bytes (assumed
> to be equivalent with a long long in some places, given the use of
> PyLong_FromLongLong() and PyLong_AsLongLong()).

> But the problems occur in fileobject.c, where we're dealing with the
> stdio library.  Not all stdio libraries seem to support long files in
> the same way, and they use a different typedef, fpos_t, which may be
> larger or smaller in size than off_t.

This isn't the problem. The problem is that we assume that because off_t is
8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
bytes, but none of the available fseek/ftell variations take an off_t as
argument ;P The TELL64() workaround works around that problem, but still
doesn't enable large file support, because there isn't any such support in
BSDI.

(Trust me... we've had logfiles and IP-traffic databases truncated because
of that... 2Gb is the limit, currently, on BSDI.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Nov 13 22:52:46 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:52:46 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 23:41:21 +0100."
 <20001113234121.S27208@xs4all.nl>
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>
 <20001113234121.S27208@xs4all.nl>
Message-ID: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>

> On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> 
> > After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> > means that the low-level system calls (lseek(), stat() etc.)  support
> > large files, through an off_t type that is at least 8 bytes (assumed
> > to be equivalent with a long long in some places, given the use of
> > PyLong_FromLongLong() and PyLong_AsLongLong()).
> 
> > But the problems occur in fileobject.c, where we're dealing with the
> > stdio library.  Not all stdio libraries seem to support long files in
> > the same way, and they use a different typedef, fpos_t, which may be
> > larger or smaller in size than off_t.
> 
> This isn't the problem. The problem is that we assume that because off_t is
> 8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
> bytes, but none of the available fseek/ftell variations take an off_t as
> argument ;P The TELL64() workaround works around that problem, but still
> doesn't enable large file support, because there isn't any such support in
> BSDI.
> 
> (Trust me... we've had logfiles and IP-traffic databases truncated because
> of that... 2Gb is the limit, currently, on BSDI.)

Sure, but the #ifdef isn't really about how well the kernel supports
large files -- it is about what code you must use.  There's one set of
places that uses off_t, and those are guided just fine by
HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
larger than 2Gb!

But there's a *different* set of test that must be used to determine
what to do for the stdio calls.  Note that on all platforms so far
where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
and Mac OSX), there were only compilation problems in fileobject.c,
and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).

What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
doesn't actually support files larger than 2Gb?  I suppose you get
some sort of I/O error when you try to write such files.  Since they
can't be created, you can't run into trouble when trying to read
them! :-)

Or am I still missing something?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From trentm@ActiveState.com  Mon Nov 13 23:07:05 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 15:07:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:52:46PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com> <20001113234121.S27208@xs4all.nl> <200011132252.RAA25873@cj20424-a.reston1.va.home.com>
Message-ID: <20001113150705.A12976@ActiveState.com>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> Sure, but the #ifdef isn't really about how well the kernel supports
> large files -- it is about what code you must use.  There's one set of
> places that uses off_t, and those are guided just fine by
> HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
> larger than 2Gb!
> 
> But there's a *different* set of test that must be used to determine
> what to do for the stdio calls.  Note that on all platforms so far
> where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
> and Mac OSX), there were only compilation problems in fileobject.c,
> and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).
> 
> What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
> doesn't actually support files larger than 2Gb?  I suppose you get
> some sort of I/O error when you try to write such files.  Since they
> can't be created, you can't run into trouble when trying to read
> them! :-)
> 
> Or am I still missing something?

I don't *think* so. Though having HAVE_LARGEFILE_SUUPORT still be defined for
a platform where this is not true still nags.

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From jeremy@alum.mit.edu  Tue Nov 14 04:16:01 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Mon, 13 Nov 2000 23:16:01 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
Message-ID: <14864.48257.324726.976892@bitdiddle.concentric.net>

I just saw your note about whether IDLE was packaged with the Python
RPMs.  It isn't, nor is anything in the Tools directory.  I did the
simplest thing possible with the RPM and ran "make install".  The RPM
installs whatever is installed as a result of that.

We've discussed installation of other tools once or twice, but I've
never decided what exactly to do.  The compiler package, for example,
comes with a distutils setup.py script that will do the job.  I think
the best way to move forward is to write setup.py scripts for all the
other tools that ought to be installed.

I don't know how to package up idle cleanly, though.  I need to decide
what to install and where to install it.  The Tools/idle directory is
nominally a package, but it doesn't seem like that will help much for
installing it.  If we want to call the executable script idle.py, it
will clash with the idle package name.  Any advice here?  It doesn't
seem like there is any obvious way to "install" IDLE; it looks like it
is intended to be run out of the source directory.

If anyone else owns (or otherwise feels responsible for) a package in
the Tools directory that ought to be included with the RPMs, please
write a setup script for it.

Jeremy


From moshez@zadka.site.co.il  Tue Nov 14 13:19:11 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Tue, 14 Nov 2000 15:19:11 +0200
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Message from Jeremy Hylton <jeremy@alum.mit.edu>
 of "Mon, 13 Nov 2000 23:16:01 EST." <14864.48257.324726.976892@bitdiddle.concentric.net>
References: <14864.48257.324726.976892@bitdiddle.concentric.net>
Message-ID: <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>

[Jeremy Hylton]
> We've discussed installation of other tools once or twice, but I've
> never decided what exactly to do.  The compiler package, for example,
> comes with a distutils setup.py script that will do the job.  I think
> the best way to move forward is to write setup.py scripts for all the
> other tools that ought to be installed.

That does seem reasonable. It seems the most non-intrusive, since we
get out of the way of proffessional packages, while still giving them
something to start from.

> I don't know how to package up idle cleanly, though.  I need to decide
> what to install and where to install it.  The Tools/idle directory is
> nominally a package, but it doesn't seem like that will help much for
> installing it.  If we want to call the executable script idle.py, it
> will clash with the idle package name.  Any advice here?  It doesn't
> seem like there is any obvious way to "install" IDLE; it looks like it
> is intended to be run out of the source directory.

I don't think we want to call the executable script idle.py. Each platform
will have it's own problems with it: for UNIXish stuff, call it "idle".
For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
should *not* be in the same directory as the idle package. In the UNIX
case it should be in $prefix/bin, and in the windows case, it should be
where python.exe is.
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From tim.one@home.com  Tue Nov 14 06:06:01 2000
From: tim.one@home.com (Tim Peters)
Date: Tue, 14 Nov 2000 01:06:01 -0500
Subject: [Python-Dev] RE: This Project Has Not Released Any Files
In-Reply-To: <200010191731.TAA01114@loewis.home.cs.tu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECAIAAA.tim.one@home.com>

[Tim, on 19 October]
> From BeOpen.com's POV, so long as they were paying major bills, they
> would rather have download traffic tickle their ad banners than SF's
> ad banners.

[Martin v. Loewis]
> Even though this should have been clear to me all the time, stating it
> explicitly triggers alarms for me.
> ...
> Of course, having ftp mirrors would mean that many downloads do not
> tickle anybody's ad banners - which would probably be in the interest
> of other users as well, just not in the interest of BeOpen. So I'm
> curious how this conflict of interest is resolved...

I hope it's clear now how it got resolved:  at the time I wrote that, the
PythonLabs guys were officially but secretly unemployed.  The issue of
distributing Python was one bone of contention at The End, but not the
mongoose that pecked the python's eyes out.

or-if-you're-not-american-it-wasn't-the-straw-that-tickled-the-
    camel-to-death-ly y'rs  - tim



From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov 14 09:20:01 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 14 Nov 2000 11:20:01 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: <200011132021.MAA09003@slayer.i.sourceforge.net>
Message-ID: <Pine.GSO.4.10.10011141118470.16939-100000@sundial>

On Mon, 13 Nov 2000, Guido van Rossum wrote:

> +     - Killing a thread from another thread.  Or maybe sending a
> +       signal.  Or maybe raising an asynchronous exception.
> + 
> +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470

In general, killing threads in considered a bad idea. It is usually much
better to design the application properly, and being aware that a thread
doing nothing is not that big a deal.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Tue Nov 14 12:06:48 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 07:06:48 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:20:01 +0200."
 <Pine.GSO.4.10.10011141118470.16939-100000@sundial>
References: <Pine.GSO.4.10.10011141118470.16939-100000@sundial>
Message-ID: <200011141206.HAA28215@cj20424-a.reston1.va.home.com>

> On Mon, 13 Nov 2000, Guido van Rossum wrote:
> 
> > +     - Killing a thread from another thread.  Or maybe sending a
> > +       signal.  Or maybe raising an asynchronous exception.
> > + 
> > +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470
> 
> In general, killing threads in considered a bad idea. It is usually much
> better to design the application properly, and being aware that a thread
> doing nothing is not that big a deal.

Sorry.  I agree 99% (see the referenced thread).  But this keeps
coming up as a feature request, so I figured that there should be
*something* we could offer -- if only some support library calls to
make implementing this common pattern smooth.

For that 1%, I believe that raising an asynchronous exception takes away
*most* of the objections against killing threads -- it gives the
thread control over recovery, since exceptions are to be expected
anyway.

Also note that in a long-running server, losing a thread every once in
a while means leaking a lot of memory!  So (in that case) the
application design cannot afford to just "park" unproductive threads
forever -- it must contain a robust way of recovering their resources.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skaller@ozemail.com.au  Tue Nov 14 13:25:27 2000
From: skaller@ozemail.com.au (John Max Skaller)
Date: Wed, 15 Nov 2000 00:25:27 +1100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>
 <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>
Message-ID: <3A113D47.ABF1D625@ozemail.com.au>

Guido van Rossum wrote:

> Looks like you are missing the point.  Of course I want to keep the
> class as a local scope *inside the class def*.  

	Ah, ok: seem's I was missing your intent.

> But I need it out of the way when executing methods.  

	I don't think this is the right idea. Consider:

	class X:
		x = 'x'
		def f(self): print x
	a = X()
	a.f() # should print 'x'

and then

	def g(self): print x
	class X:
		f = g
	a = X()
	a.f() # should fail

and then:

	def maker():
		x = 'x'
		def f(self): print x
		return f
	class X:
		f = maker()
	a = X()
	a.f() # should work

The point is that I think there is not really much choice
in the matter: when you lookup an unqualified name in a function,
you must look in the function locals, then in the scope of the
object in which the function is defined, and then in the scope
in which that is defined .. and so ontil you reach the bottom
of the stack (which presumably is 'builtins').

	There's no real issue of keeping the class of the object
of a bound method 'out of the way' when executing the function
of the bound method. The scope of that class should be searched
if, and only if, the function happens to have been defined
inside that class. Note there is no such thing as a method
in python: executing a bound (or unbound) method results
in executing a plain old function.

	You _could_ exclude class scopes from the search stack
associated with a function, and just search locals, then
enclosing functions, then the enclosing module and finally
builtins. But it would be highly inconsistent, and not what
people used to lexical scoping would expect. It will be hard
enough to explain the _consistent_ rule that only local
bindings can be changed by assignments -- unless a global
directive changes that. But that rule IS consistent (and the
global directive much less so!)

	I think there is general agreement that lexically
scoping functions inside functions will not break much code.
Do you have an example of real application code where searching a class
in
which a function is defined will break code?

-- 
John (Max) Skaller, mailto:skaller@maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
checkout Vyper http://Vyper.sourceforge.net
download Interscript http://Interscript.sourceforge.net


From guido@python.org  Tue Nov 14 13:29:17 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 08:29:17 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 15:19:11 +0200."
 <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>
References: <14864.48257.324726.976892@bitdiddle.concentric.net>
 <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>
Message-ID: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>

> [Jeremy Hylton]
> > We've discussed installation of other tools once or twice, but I've
> > never decided what exactly to do.  The compiler package, for example,
> > comes with a distutils setup.py script that will do the job.  I think
> > the best way to move forward is to write setup.py scripts for all the
> > other tools that ought to be installed.
> 
> That does seem reasonable. It seems the most non-intrusive, since we
> get out of the way of proffessional packages, while still giving them
> something to start from.

Agreed.

> > I don't know how to package up idle cleanly, though.  I need to decide
> > what to install and where to install it.  The Tools/idle directory is
> > nominally a package, but it doesn't seem like that will help much for
> > installing it.  If we want to call the executable script idle.py, it
> > will clash with the idle package name.  Any advice here?  It doesn't
> > seem like there is any obvious way to "install" IDLE; it looks like it
> > is intended to be run out of the source directory.
> 
> I don't think we want to call the executable script idle.py. Each platform
> will have it's own problems with it: for UNIXish stuff, call it "idle".
> For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
> should *not* be in the same directory as the idle package. In the UNIX
> case it should be in $prefix/bin, and in the windows case, it should be
> where python.exe is.

For Windows, the IDLE problem is solved: it's installed in directory
<topdir>/Tools/idle/ and there's a link to it in the start menu.  It's
not directly accessible from the command line -- but on Windows,
nobody cares about the command line.

Maybe we can extend this idea to Unix.  Concrete proposal: install
IDLE as <prefix>/lib/python<version>/site-packages/Tools/idle/, and
place a symlink or 2-line bootstrap script in <prefix>/bin/idle.

The introduction of the extra directory level "Tools" implies that
idle is not importable as a package from regular Python.  This is how
I want it -- otherwise I could/would have made IDLE a package in the
Lib/lib-tk directory instead of a directory in Tools.

I'd gladly accept a contribution of a setup.py file that does this for
me.  (I looked at the distutils manual for a bit but decided that I'd
have to read it cover-to-cover to figure out whether what I want is
even possible...)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Nov 14 14:00:39 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 09:00:39 -0500
Subject: [Python-Dev] Re: Nested functions and class scope
In-Reply-To: Your message of "Wed, 15 Nov 2000 00:25:27 +1100."
 <3A113D47.ABF1D625@ozemail.com.au>
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com> <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>
 <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <200011141400.JAA28465@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> 
> > Looks like you are missing the point.  Of course I want to keep the
> > class as a local scope *inside the class def*.  

John Max Skaller replied:

> 	Ah, ok: seem's I was missing your intent.
> 
> > But I need it out of the way when executing methods.  
> 
> 	I don't think this is the right idea. Consider:
> 
> 	class X:
> 		x = 'x'
> 		def f(self): print x
> 	a = X()
> 	a.f() # should print 'x'

We understand each other perfectly.  I just disagree with you on what
this example should do!  I believe that in the sake of backwards
compatibility *as well as* not "giving in" to Java and C++ conventions
that I believe are a bad idea for Python, the reference to 'x' from
inside 'f' should not find the class variable 'x' but only a
module-global 'x'.

Backwards compatibility: currently this program prints '1':

    x = 1
    class C:
	x = 2
	def f(self): print x
    C().f()

I believe it should not be broken; I expect that this is, for better
or for worse, a popular idiom, especially when you look at methods
rather than varibles.  I believe it might be common to find this:

    from lower_level_module import some_call
    class C:
	def some_call(self):
	    "OO wrapper around some_call()"
	    return some_call(self.data)

Your rule would break this, because the reference to some_call()
inside C.some_call() would be changed to make a (senseless, recursive)
reference to itself instead of to the globally imported some_call()
function.

Java/C++ expectations: in Java/C++, the rules are different.  Inside a
method, not just names defined in the current class are in scope, but
also all names defined in its base classes.  Look at this:

    # Module a.py
    class B:
        def f(self): pass

    # Module b.py
    import a
    class C(a.B):
	def g(self): pass
        def h(self):
	    g(self)
            f(self)

Under your rules, the reference to g() in h() would work, but the
reference to f() would fail.  In similarly structured Java or C++
code, h() could reference both g() and f() without a naming
disambiguation.

Python requires that you specify the class when you reference a class
attribute (method or class variable).  I don't want to lose this rule,
it is important (to me) for readability.

[...more examples elided...]
> The point is that I think there is not really much choice
> in the matter: when you lookup an unqualified name in a function,
> you must look in the function locals, then in the scope of the
> object in which the function is defined, and then in the scope
> in which that is defined .. and so ontil you reach the bottom
> of the stack (which presumably is 'builtins').

I claim different, and above I have explained why I think this is
necessary.  I'd agree if you said "the scope of the *function* in
which it is defined."

> 	There's no real issue of keeping the class of the object
> of a bound method 'out of the way' when executing the function
> of the bound method. The scope of that class should be searched
> if, and only if, the function happens to have been defined
> inside that class. Note there is no such thing as a method
> in python: executing a bound (or unbound) method results
> in executing a plain old function.

If the code generator keeps track of the distinction between class and
function scopes, my rules can be implemented easily without
introducing a distinction between functions and methods.

> 	You _could_ exclude class scopes from the search stack
> associated with a function, and just search locals, then
> enclosing functions, then the enclosing module and finally
> builtins. But it would be highly inconsistent, and not what
> people used to lexical scoping would expect.


Yet, this is what I want.  It's impossible to design things so that
people coming from very different backgrounds will never encounter a
surprise.  Since Python is primarily an OO language, and there's ten
years of established Python rules, I prefer to give the OO people and
existing Python users an easier time (note that the OO people at least
can count on *consistently* not seeing the class scope in the
"methods") and tell the lexical-scoping people about the exceptions.

> It will be hard
> enough to explain the _consistent_ rule that only local
> bindings can be changed by assignments -- unless a global
> directive changes that. But that rule IS consistent (and the
> global directive much less so!)

> 	I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't found one yet, but I've seen plenty of examples where a
class defines a method with the same name as a global function
(including lots in the standard library -- it really is a common
idiom).  I think this comes too close to risk breakage.

But even if we find that this doesn't break anything, I still think
it's unwise to allow unqualified access to class attributes from
within methods.  People would discover that in this example:

    class C:
	def m1(self):
	    pass
	def m2a(self):
	    m1(self)
	def m2b(self):
	    self.m1()

m2a() and m2b() would be equivalent, i.e. m1(self) and self.m1() are
equivalent.  That's evil, because it opens up a meaningless choice
between alternative (and the m1(self) notation is inferior, because it
doesn't look in base classes).

But I'm beginning to repeat myself, so I think I've said all that can
be said about this.  Sorry the scope rules in Vyper are wrong.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Tue Nov 14 14:09:37 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2000 15:09:37 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>              <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <013701c04e44$871496b0$0900a8c0@SPIFF>

John Max Skaller wrote:
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't been following this thread closely, but are we talking
about a scoping model where a class can override any built-in
function or method by defining a method (or a class attribute)
with the same name?

like in urllib.py, for example:

class URLopener:
    ...
    def open(self, fullurl, data=None):
        fullurl = unwrap(fullurl)
        if self.tempcache and self.tempcache.has_key(fullurl):
            filename, headers = self.tempcache[fullurl]
            fp = open(filename, 'rb')
            return addinfourl(fp, headers, fullurl)
    ...

this won't fly.

</F>



From mal@lemburg.com  Tue Nov 14 14:23:50 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 14 Nov 2000 15:23:50 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>
 <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <3A114AF6.EC584A17@lemburg.com>

John Max Skaller wrote:
> 
>         I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

class Object:

   type = 'typename'

   def __init__(self, obj):

       if type(obj) == types.IntType:
           ...

Basically all builtins which are in common use like open(), id()
type(), etc. would no longer be accessible.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From klm@digicool.com  Tue Nov 14 15:53:47 2000
From: klm@digicool.com (Ken Manheimer)
Date: Tue, 14 Nov 2000 10:53:47 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

There's a seminal steele and sussman paper, _The Art of the Interpreter,
or the Modularity Complex (Parts Zero, One, and Two)_, which, among other
things, introduces closures and describes why dynamic scoping is evil.  
(The paper sketches out an elementary, lexically scoped lisp interpreter
which i think is the precursor to scheme.)  It's been over 20 years since
i read it, but i recall something known as the funarg problem, where
dynamic scoping + recursion (maybe, + functions as first class objects)
inherently breaks certain parameter passing arrangements.

If you're interested, see:

  ftp://publications.ai.mit.edu/ai-publications/0-499/AIM-453.ps

(I'm working at home today, so can't even glance at it to refresh my
memory, and oughtn't at the moment, anyway.)

In any case, dynamic scoping breaks locality of reference for just about
any definition of "locality".

With lexical scoping, you can trace through the text of the program to
track down implicit references.  Even with acquisition, the containment
connections are made explicitly (though it seems magic with web
programming, because the zope publisher applies .__of__() for you), and
can be nailed down by introspection on the objects (.aq_base, .aq_parent,
.aq_inner...).

With dynamic scoping, variable references go up the stack to the context
of *any* intervening call frame, with no way for the programmer to know
where it's being resolved.  In practice, even without the funarg breakage,
it doesn't scale up - it's too promiscuous a state-communication
mechanism.  You can work around it - a lot of emacs lisp code does so -
but it's a terrible burden, not a benefit.

Ken



From esr@golux.thyrsus.com  Tue Nov 14 16:59:24 2000
From: esr@golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 11:59:24 -0500
Subject: [Python-Dev] Making popen2 actually work
Message-ID: <200011141659.LAA01557@golux.thyrsus.com>

OK, so Guido told me at one point that 2.0 was supposed to have
included changes to the way buffering of I/O to subprocesses is
handled.  I've installed 2.0, and I need to know how to make the
following program work:

import os, sys

(child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
print "* Child process spawned"

while 1:
    line = raw_input(">");
    child_stdin.write(line); child_stdin.flush()
    print "* Line written to child process"
    transformed = child_stdout.readline(); child_stdout.flush()
    print "* Line read back from child process."
    sys.stdout.write(transformed)

If anybody can tell me what the right magic is here, I'll write up 
a patch for the library docs explaining it and including the corrected
example.  This has to be a FAQ.
--
						>>esr>>


From esr@golux.thyrsus.com  Tue Nov 14 17:08:28 2000
From: esr@golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 12:08:28 -0500
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
Message-ID: <200011141708.MAA01643@golux.thyrsus.com>

I installed from the Beopen-Python-2.0 RPM at Source Forge.  Now:

golux:~/WWW/fetchmail$ fetchmailconf
Traceback (most recent call last):
  File "./fetchmailconf", line 9, in ?
    from Tkinter import *
  File "/usr/local/lib/python2.0/lib-tk/Tkinter.py", line 35, in ?
    import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter

This is a serious problem.  Is there a non-broken RPM out there?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The right of self-defense is the first law of nature: in most
governments it has been the study of rulers to confine this right
within the narrowest limits possible.  Wherever standing armies
are kept up, and when the right of the people to keep and bear
arms is, under any color or pretext whatsoever, prohibited,
liberty, if not already annihilated, is on the brink of
destruction." 
	-- Henry St. George Tucker (in Blackstone's Commentaries)


From jeremy@alum.mit.edu  Tue Nov 14 17:38:09 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 12:38:09 -0500 (EST)
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
In-Reply-To: <200011141708.MAA01643@golux.thyrsus.com>
References: <200011141708.MAA01643@golux.thyrsus.com>
Message-ID: <14865.30849.276890.227441@bitdiddle.concentric.net>

The RPM isn't "broken."  It just doesn't include _tkinter.  You'll
need to get the separate Tkinter RPM for that.

The separation is intentional, because different Linux distriubtions
have different Tk versions installed.  Rather than have the main RPM
depend on a single Tk version, they are packaged separately.  You can
also use the distutils package to install _tkinter.so.  It will work
with any version of Tk, while the RPMs will only work with the version
of Tk installed on my machine.

Jeremy


From guido@python.org  Wed Nov 15 03:48:14 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 22:48:14 -0500
Subject: [Python-Dev] Making popen2 actually work
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:59:24 EST."
 <200011141659.LAA01557@golux.thyrsus.com>
References: <200011141659.LAA01557@golux.thyrsus.com>
Message-ID: <200011150348.WAA05535@cj20424-a.reston1.va.home.com>

> OK, so Guido told me at one point that 2.0 was supposed to have
> included changes to the way buffering of I/O to subprocesses is
> handled.  I've installed 2.0, and I need to know how to make the
> following program work:
> 
> import os, sys
> 
> (child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
> print "* Child process spawned"
> 
> while 1:
>     line = raw_input(">");
>     child_stdin.write(line); child_stdin.flush()
>     print "* Line written to child process"
>     transformed = child_stdout.readline(); child_stdout.flush()
>     print "* Line read back from child process."
>     sys.stdout.write(transformed)
> 
> If anybody can tell me what the right magic is here, I'll write up 
> a patch for the library docs explaining it and including the corrected
> example.  This has to be a FAQ.

First, raw_input() strips the newline, so you should change the second
line of the loop body to

     child_stdin.write(line+'\n'); child_stdin.flush()

Second, with "cat" instead of "/usr/bin/rev", this works fine for me,
so your logic is fine.

The trick seems to be to get the subprocess not to buffer its output
(or input).  There's nothing that Python can do for you here!

Unfortunately, most programs use stdio for all their console I/O, and
stdio automatically buffers its output when connected to a pipe... :-(

If you absolutely, positively have to do this with a program that
won't unbuffer its output, your only resort would be to use pseudo
ttys -- but that's quite a different story... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Wed Nov 15 02:13:36 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 21:13:36 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>
References: <14864.48257.324726.976892@bitdiddle.concentric.net>
 <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>
 <200011141329.IAA28404@cj20424-a.reston1.va.home.com>
Message-ID: <14865.61776.50840.609922@bitdiddle.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  >> > I don't know how to package up idle cleanly, though.  I need to
  >> > decide what to install and where to install it.  The Tools/idle
  >> > directory is nominally a package, but it doesn't seem like that
  >> > will help much for installing it.  If we want to call the
  >> > executable script idle.py, it will clash with the idle package
  >> > name.  Any advice here?  It doesn't seem like there is any
  >> > obvious way to "install" IDLE; it looks like it is intended to
  >> > be run out of the source directory.
  >>

  GvR> For Windows, the IDLE problem is solved: it's installed in
  GvR> directory <topdir>/Tools/idle/ and there's a link to it in the
  GvR> start menu.  It's not directly accessible from the command line
  GvR> -- but on Windows, nobody cares about the command line.

  GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
  GvR> install IDLE as
  GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
  GvR> place a symlink or 2-line bootstrap script in
  GvR> <prefix>/bin/idle.

How about
<prefix>/lib/python<version>/Tools/idle
instead.  I suggest the change to support an until-now-implicit policy
that anything in site-packages actually be a package.  If things in
site-packages aren't supposed to be packages, it ought to be called
something like site-tools <0.2 wink>.

  GvR> The introduction of the extra directory level "Tools" implies
  GvR> that idle is not importable as a package from regular Python.
  GvR> This is how I want it -- otherwise I could/would have made IDLE
  GvR> a package in the Lib/lib-tk directory instead of a directory in
  GvR> Tools.

I think the extra Tools directory still implies this, even if it isn't
in site-packages.

  GvR> I'd gladly accept a contribution of a setup.py file that does
  GvR> this for me.  (I looked at the distutils manual for a bit but
  GvR> decided that I'd have to read it cover-to-cover to figure out
  GvR> whether what I want is even possible...)

Is there a reason why IDLE has this peculiar installation process?
Might there be a better way to do it now that we have distutils.  If
there is, we could put it in PEP 42.

Jeremy


From guido@python.org  Wed Nov 15 04:10:06 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 23:10:06 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 21:13:36 EST."
 <14865.61776.50840.609922@bitdiddle.concentric.net>
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com>
 <14865.61776.50840.609922@bitdiddle.concentric.net>
Message-ID: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>

>   GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
>   GvR> install IDLE as
>   GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
>   GvR> place a symlink or 2-line bootstrap script in
>   GvR> <prefix>/bin/idle.
> 
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Fine.

>   GvR> The introduction of the extra directory level "Tools" implies
>   GvR> that idle is not importable as a package from regular Python.
>   GvR> This is how I want it -- otherwise I could/would have made IDLE
>   GvR> a package in the Lib/lib-tk directory instead of a directory in
>   GvR> Tools.
> 
> I think the extra Tools directory still implies this, even if it isn't
> in site-packages.

Sure.

>   GvR> I'd gladly accept a contribution of a setup.py file that does
>   GvR> this for me.  (I looked at the distutils manual for a bit but
>   GvR> decided that I'd have to read it cover-to-cover to figure out
>   GvR> whether what I want is even possible...)
> 
> Is there a reason why IDLE has this peculiar installation process?
> Might there be a better way to do it now that we have distutils.  If
> there is, we could put it in PEP 42.

IDLE doesn't have a peculiar installation process, but it may not be
what distutils is currently good at.  IDLE is an application that
needs its files installed somewhere but not on the module search path
for other apps.  I don't know how to do that!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From klm@digicool.com  Wed Nov 15 05:05:10 2000
From: klm@digicool.com (Ken Manheimer)
Date: Wed, 15 Nov 2000 00:05:10 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>
Message-ID: <Pine.LNX.4.21.0011142355310.8955-100000@serenade.digicool.com>

On Tue, 14 Nov 2000, Ken Manheimer wrote:

> In any case, dynamic scoping breaks locality of reference for just about
> any definition of "locality".

(Huh - i realized that "locality of reference" isn't the right phrase, not
sure what is.  I was trying to refer to something along the lines of
modularity - basically, the virtue of lexical scoping is that you can,
eventually, trace the connections by reading the program text.  Dynamic
scoping is not so well behaved - any routine that calls your routine may
contain a variable whose value you're expecting to get from elsewhere.  
It's too promiscuous...)

Ken



From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov 15 07:11:29 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 15 Nov 2000 09:11:29 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011150904510.16783-100000@sundial>

On Tue, 14 Nov 2000, Guido van Rossum wrote:

> > How about
> > <prefix>/lib/python<version>/Tools/idle
> > instead.  I suggest the change to support an until-now-implicit policy
> > that anything in site-packages actually be a package.  If things in
> > site-packages aren't supposed to be packages, it ought to be called
> > something like site-tools <0.2 wink>.
> 
> Fine.

Sorry, just noticed a problem with that:

Say you put idle in .../Tools/idle
And compiler in .../Tools/compiler

Then the fact that some script does

sys.path.insert(0, ".../Tools") means that it would get all the tools
as packages to import, instead of only those it needs (say, for
idle-callers, idle). That means a lot of backwards compat. issues.
Why not have each tool be in things like ..../idle-support/idle/
And .../compiler/compiler-support/compiler, etc.?

Actually, once we do that, why bring the Python path into that at all?
"idle" is a program. The fact it is written in Python is incidental --
it might as well have been written in C, for all most users care. The
FHS policy would seem to indicate that the support files would need
to be in things like /usr/lib and /usr/local/lib
Say, the default installation would put it in /usr/local/lib/idle-support/
and would add a line to sys.path.insert(), and put "idle" (the script)
under autoconf control too.

sorry-for-challenging-the-consensus-again-ly y'rs, Z.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From pf@artcom-gmbh.de  Wed Nov 15 07:32:59 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Wed, 15 Nov 2000 08:32:59 +0100 (MET)
Subject: Using telnetlib instead of 'popen2' (was: Re: [Python-Dev] Making popen2 actually work)
In-Reply-To: <200011150348.WAA05535@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 14, 2000 10:48:14 pm"
Message-ID: <m13vx3z-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum wrote:
> The trick seems to be to get the subprocess not to buffer its output
> (or input).  There's nothing that Python can do for you here!
> 
> Unfortunately, most programs use stdio for all their console I/O, and
> stdio automatically buffers its output when connected to a pipe... :-(
> 
> If you absolutely, positively have to do this with a program that
> won't unbuffer its output, your only resort would be to use pseudo
> ttys -- but that's quite a different story... :-(

Yes.  But it should be noted here, that sometimes you can avoid the rather
complicated and platform depending handling of pseudo ttys, if you use
the 'telnetlib' module from the Python standard library.  The fine lib
documentation contains a good example ready for cut'n'paste.  Disadvantage:
This only works, if the telnet service is not disabled due to security
reasons.

Regards, Peter


From gward@mems-exchange.org  Wed Nov 15 18:11:30 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Wed, 15 Nov 2000 13:11:30 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <14865.61776.50840.609922@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 14, 2000 at 09:13:36PM -0500
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>
Message-ID: <20001115131130.A22029@ludwig.cnri.reston.va.us>

On 14 November 2000, Jeremy Hylton said:
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Actually, lots of non-packagized modules get installed to site-packages
-- they just wind up right in site-packages, rather than in a
subdirectory of it.

I've always thought this directory was misnamed -- should have been
"site-lib" or something.  Oh well.

For the record, the Distutils don't currently support the "application"
model of installation that Guido wants.  It's on the (unofficial,
notional, blue-sky) to-do list.

        Greg


From nas@arctrix.com  Wed Nov 15 17:35:59 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Wed, 15 Nov 2000 09:35:59 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001115093559.A19286@glacier.fnational.com>

[The python-dev list server seems to have gotten hungry and eaten my
original message.  I'm trying again.]


I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil



From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov 16 07:06:14 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 16 Nov 2000 09:06:14 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <20001115131130.A22029@ludwig.cnri.reston.va.us>
Message-ID: <Pine.GSO.4.10.10011160905260.24460-100000@sundial>

On Wed, 15 Nov 2000, Greg Ward wrote:

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.
> 
> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

On a similar note, there's a discussion going on about this in 
debian-python@lists.debian.org (there are archives on the Debian
site)

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From guido@python.org  Thu Nov 16 16:26:16 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 11:26:16 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Wed, 15 Nov 2000 13:11:30 EST."
 <20001115131130.A22029@ludwig.cnri.reston.va.us>
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>
 <20001115131130.A22029@ludwig.cnri.reston.va.us>
Message-ID: <200011161626.LAA10243@cj20424-a.reston1.va.home.com>

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.

The name doesn't matter that much.  I think this is a fine use of
site-packages.

> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

At least this admission saves me wasting time trying to force
distutils into submssion. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Thu Nov 16 19:44:59 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 14:44:59 -0500
Subject: [Python-Dev] curses demos
Message-ID: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us>

I'd like to add a Demo/curses directory to the CVS tree.  If anyone
has reasonably small programs that use the curses module, please let
me know.

Right now I have two programs: a Life game that I wrote, and Alexei
Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
manager.  I also want to write scripts to demonstrate reading the
mouse, and writing a threaded curses program.

--amk


From guido@python.org  Thu Nov 16 20:03:47 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 15:03:47 -0500
Subject: [Python-Dev] curses demos
In-Reply-To: Your message of "Thu, 16 Nov 2000 14:44:59 EST."
 <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us>
References: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us>
Message-ID: <200011162003.PAA14322@cj20424-a.reston1.va.home.com>

> I'd like to add a Demo/curses directory to the CVS tree.  If anyone
> has reasonably small programs that use the curses module, please let
> me know.
> 
> Right now I have two programs: a Life game that I wrote, and Alexei
> Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
> manager.  I also want to write scripts to demonstrate reading the
> mouse, and writing a threaded curses program.

Cool.  Go for it!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Nov 17 00:42:56 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 19:42:56 -0500
Subject: [Python-Dev] coroutines and microthreads
Message-ID: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>

(I would cc this to the Stackless list as well, except that starship
mail is still down it seems.  I *am* posting to the python-coro list,
which seems to be appropriate here.  Will Ware, are you listening?)

Coroutines!
-----------

Inspired by Gordon McMillan's Stackless tutorial
(http://www.mcmillan-inc.com/stackless.html), I've been thinking more
about a coroutine API for Python.

I proposed one before (see the thread "uthread strawman" in
python-dev).  I've been thinking a bit more and am now revising my
proposal a bit.

I'll first give an example.  Tim Peters gave a standard example to
return the fringe of a list (checked in as Demo/threads/fcmp.py).

Using my proposed API this example could be rewritten a bit
cleaner, as follows:

    from coro import coroutine, suspend, EarlyExit  # Provisional module name

    def fringe(L):
	for x in L:
	    if type(x) is type(L):
		fringe(x)
	    else:
		suspend(x)

    def printinorder(L):
	c = coroutine(f, L)
	try:
	    while 1:
		print c(),
	except EarlyExit:
	    pass
	print

    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
    printinorder(x) # prints "0 1 2 3 4 5 6"

This is about as simple as it gets.  It supports both generators (like
the example here) and coroutines (like Tim's Demo/threads/squasher.py
example).

Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
propose a function current() which returns the current coroutine
object.  There should also be a way to kill a coroutine (or at least
to send an exception).  When a coroutine falls through at its end,
*some* other coroutine needs to be resumed.  

I believe this can be implemented with a much simplified stackless
approach, that doesn't cater towards continuations (but still borrows
a lot of wisdom from Christian's Stackless).  It can also be
implemented using threads, which should give some hope for getting the
same API supported in JPython, making it more attractive.  I am hoping
to create an implementation on top of Stackless, just to experiment
with the idiom.

Microthreads?
-------------

I'm much less clear about microthreads (uthreads).  Last time, I
thought this was a fancy name for coroutines.  It isn't!  Microthreads
are "almost real" threads, with round-robin scheduling.  What makes
them attractive to some is the fact that they don't use operating
system resources: there's no OS-level stack or kernel process table
entry associated with them.  This also accounts for their biggest
weakness: when a microthread is suspended for I/O, *all* microthreads
are suspended.  In limited contexts, such as a network server, this
can be solved by an approach similar to that in Gordon's
SelectDispatcher.  (It would be a titanic effort to do this for every
potentially blocking I/O operation, and it probably wouldn't work well
with C extensions.)

I'm not sure what applications require the round-robin scheduling
property of uthreads -- certainly Gordon's application would seem to
be doing just fine without it (I'm not sure if it is being scheduled
round-robin or not).

Proper round-robin scheduling for uthreads requires explicit switching
code in the Python interpreter.  Stackless provides this, at the same
place where in regular Python the global interpreter lock is released
and re-acquired to give other threads a chance to run.  Is this
needed?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gmcm@hypernet.com  Fri Nov 17 02:29:52 2000
From: gmcm@hypernet.com (Gordon McMillan)
Date: Thu, 16 Nov 2000 21:29:52 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <3A1451D0.12627.17914E75@localhost>

[Guido]
> I'll first give an example.  Tim Peters gave a standard example to
> return the fringe of a list (checked in as Demo/threads/fcmp.py).
> 
> Using my proposed API this example could be rewritten a bit
> cleaner, as follows:
> 
>     from coro import coroutine, suspend, EarlyExit  # Provisional module name
> 
>     def fringe(L):
> 	for x in L:
> 	    if type(x) is type(L):
> 		fringe(x)
> 	    else:
> 		suspend(x)
> 
>     def printinorder(L):
> 	c = coroutine(f, L)
> 	try:
> 	    while 1:
> 		print c(),
> 	except EarlyExit:
> 	    pass
> 	print
> 
>     x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>     printinorder(x) # prints "0 1 2 3 4 5 6"
> 
> This is about as simple as it gets.  It supports both generators (like
> the example here) and coroutines (like Tim's Demo/threads/squasher.py
> example).

This looks very nice and clean. It looks sufficient for the type 
of thing I'm doing (and planning to do), but is it really a full 
coroutine API? That is, doesn't the fact that you always 
suspend to the guy who just activated you make this a 
generator API? (OTOH, if I want A and B to talk to each other 
as "coroutines", is it sufficient to make them both "generators" 
and then glue them together with another routine that just 
swaps results?)
 
> Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> propose a function current() which returns the current coroutine
> object.  There should also be a way to kill a coroutine (or at least
> to send an exception).  When a coroutine falls through at its end,
> *some* other coroutine needs to be resumed.  

Random thoughts:
 - I've found it handy that Christian's stuff lets you grab a 
coroutine as well as return one (that is, either side can 
instigate it). Not sure if that's necessary.
 - What do you mean by "kill a coroutine"? You can't interrupt 
one, so isn't it sufficient that when it goes out of scope it gets 
GC'd somehow?
 - It appears from your example that falling off the end 
automatically raises an EarlyExit. I think I see more 
arguments for that than against it :-).
 
> Microthreads?
> -------------
> 
> I'm much less clear about microthreads (uthreads).  Last time, I
> thought this was a fancy name for coroutines.  It isn't!  Microthreads
> are "almost real" threads, with round-robin scheduling.  What makes
> them attractive to some is the fact that they don't use operating
> system resources: there's no OS-level stack or kernel process table
> entry associated with them.  This also accounts for their biggest
> weakness: when a microthread is suspended for I/O, *all* microthreads
> are suspended.  In limited contexts, such as a network server, this
> can be solved by an approach similar to that in Gordon's
> SelectDispatcher.  (It would be a titanic effort to do this for every
> potentially blocking I/O operation, and it probably wouldn't work well
> with C extensions.)

Using the raw Win32 API, I think you could come pretty close. 
I've been wondering if it's possible to do something that would 
get Cameron to quit raving about Tcl's event loop ;-).
 
> I'm not sure what applications require the round-robin scheduling
> property of uthreads -- certainly Gordon's application would seem to
> be doing just fine without it (I'm not sure if it is being scheduled
> round-robin or not).

Each coroutine (or whatever they are) runs until it calls one of 
the SelectDispatcher methods that suspends it.  The socket 
methods suspend it until select says the socket is ready; 
yield suspends it till the next time round the select loop 
(which has a timeout). So piggish routines are encouraged to 
yield once in a while.
 
(While I'm doing things the other way 'round, I believe I could 
use your API without changing SelectDispatcher's API at all.)

> Proper round-robin scheduling for uthreads requires explicit switching
> code in the Python interpreter.  Stackless provides this, at the same
> place where in regular Python the global interpreter lock is released
> and re-acquired to give other threads a chance to run.  Is this
> needed?

While I'm not really a uthread user, I think they would give you 
an unqualified "yes". The advantage to explicitly yeilding is 
that (with proper thought) you don't need nasty things like 
locks; the disadvantage (as demonstrated by a particular OS 
with a rabidly fanatical following) is that one jerk can ruin it for 
everybody.

- Gordon


From guido@python.org  Fri Nov 17 03:15:06 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 22:15:06 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: Your message of "Thu, 16 Nov 2000 21:29:52 EST."
 <3A1451D0.12627.17914E75@localhost>
References: <3A1451D0.12627.17914E75@localhost>
Message-ID: <200011170315.WAA21528@cj20424-a.reston1.va.home.com>

> > This is about as simple as it gets.  It supports both generators (like
> > the example here) and coroutines (like Tim's Demo/threads/squasher.py
> > example).
> 
> This looks very nice and clean. It looks sufficient for the type 
> of thing I'm doing (and planning to do), but is it really a full 
> coroutine API? That is, doesn't the fact that you always 
> suspend to the guy who just activated you make this a 
> generator API? (OTOH, if I want A and B to talk to each other 
> as "coroutines", is it sufficient to make them both "generators" 
> and then glue them together with another routine that just 
> swaps results?)

Yes, it is a full coroutine API -- you don't have to use suspend() at
all.  Appended is my version of squasher.py, an almost mechanical
translation from Tim's version (co.tran(cofoobar, arg) => cofoobar(arg)).
That's built out of a bunch of real coroutines with only explicit
transfers!

> > Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> > propose a function current() which returns the current coroutine
> > object.  There should also be a way to kill a coroutine (or at least
> > to send an exception).  When a coroutine falls through at its end,
> > *some* other coroutine needs to be resumed.  
> 
> Random thoughts:
>  - I've found it handy that Christian's stuff lets you grab a 
> coroutine as well as return one (that is, either side can 
> instigate it). Not sure if that's necessary.

Sorry, not sure what this means.

>  - What do you mean by "kill a coroutine"? You can't interrupt 
> one, so isn't it sufficient that when it goes out of scope it gets 
> GC'd somehow?

That's one solution.  You can't interrupt threads either, and most
people are happy with that (although there's a request for thread
interrupts in PEP 42).

>  - It appears from your example that falling off the end 
> automatically raises an EarlyExit. I think I see more 
> arguments for that than against it :-).

Yes, this follows Tim's lead.  What would be an argument against it?

> > Microthreads?
> > -------------
> > 
> > I'm much less clear about microthreads (uthreads).  Last time, I
> > thought this was a fancy name for coroutines.  It isn't!  Microthreads
> > are "almost real" threads, with round-robin scheduling.  What makes
> > them attractive to some is the fact that they don't use operating
> > system resources: there's no OS-level stack or kernel process table
> > entry associated with them.  This also accounts for their biggest
> > weakness: when a microthread is suspended for I/O, *all* microthreads
> > are suspended.  In limited contexts, such as a network server, this
> > can be solved by an approach similar to that in Gordon's
> > SelectDispatcher.  (It would be a titanic effort to do this for every
> > potentially blocking I/O operation, and it probably wouldn't work well
> > with C extensions.)
> 
> Using the raw Win32 API, I think you could come pretty close. 

You can do it on Unix too, by trapping open() to always set files in
nonblocking mode.  It's just a lot of work because there are so many
calls to trap (cf. your dispatcher, which just traps read, write,
accept, connect).  Long ago, there was a user-mode thread package on
Solaris that did this; it's still immortalized in Python/thread_lwp.h
although I doubt that it still works.  GNU Pth uses the same approach
for user-mode threads.  Hm...  This makes me wonder...  Maybe instead
of uthreads we could promote Python linked with Pth?

> I've been wondering if it's possible to do something that would 
> get Cameron to quit raving about Tcl's event loop ;-).

Cameron is now lobbying for a separation between Tcl and Tk.  I guess
that will mean that the event loop will have to be moved back to Tk,
where it started many years ago. :-)

> > I'm not sure what applications require the round-robin scheduling
> > property of uthreads -- certainly Gordon's application would seem to
> > be doing just fine without it (I'm not sure if it is being scheduled
> > round-robin or not).
> 
> Each coroutine (or whatever they are) runs until it calls one of 
> the SelectDispatcher methods that suspends it.  The socket 
> methods suspend it until select says the socket is ready; 
> yield suspends it till the next time round the select loop 
> (which has a timeout). So piggish routines are encouraged to 
> yield once in a while.

That's what I figured.

> (While I'm doing things the other way 'round, I believe I could 
> use your API without changing SelectDispatcher's API at all.)

I think so too.  Maybe that'll be my next exercise.  I've already done
8 queens with coroutines (slower than recursion, alas -- should be
faster when coroutines are implemented in C though because you save on
number of function calls).  I've also made tho working implementations of
my proposed coro API: one using threads, one using Stackless and the
continuation module.  (Found a few quirks in the examples in your
tutorial.  Interested in details?)

> > Proper round-robin scheduling for uthreads requires explicit switching
> > code in the Python interpreter.  Stackless provides this, at the same
> > place where in regular Python the global interpreter lock is released
> > and re-acquired to give other threads a chance to run.  Is this
> > needed?
> 
> While I'm not really a uthread user, I think they would give you 
> an unqualified "yes". The advantage to explicitly yeilding is 
> that (with proper thought) you don't need nasty things like 
> locks; the disadvantage (as demonstrated by a particular OS 
> with a rabidly fanatical following) is that one jerk can ruin it for 
> everybody.

Let me guess -- you don't like the Mac much, do you? :-)

And yes, I noticed that the first thing the uthread API does is define
a bunch of synchronization primitives. :-(  I think it wouldn't be
hard to implement the round-robin scheduling, but it's yet more added
complexity...

BTW, how's PEP 219 coming along? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

# Coroutine example:  general coroutine transfers
# (After Tim Peters)
#
# The program is a variation of a Simula 67 program due to Dahl & Hoare,
# (Dahl/Dijkstra/Hoare, Structured Programming; Academic Press, 1972)
# who in turn credit the original example to Conway.
#
# We have a number of input lines, terminated by a 0 byte.  The problem
# is to squash them together into output lines containing 72 characters
# each.  A semicolon must be added between input lines.  Runs of blanks
# and tabs in input lines must be squashed into single blanks.
# Occurrences of "**" in input lines must be replaced by "^".
#
# Here's a test case:

test = """\
   d    =   sqrt(b**2  -  4*a*c)
twoa    =   2*a
   L    =   -b/twoa
   R    =   d/twoa
  A1    =   L + R
  A2    =   L - R\0
"""

# The program should print:

# d = sqrt(b^2 - 4*a*c);twoa = 2*a; L = -b/twoa; R = d/twoa; A1 = L + R;
#A2 = L - R
#done

# getline: delivers the next input line to its invoker
# disassembler: grabs input lines from getline, and delivers them one
#    character at a time to squasher, also inserting a semicolon into
#    the stream between lines
# squasher:  grabs characters from disassembler and passes them on to
#    assembler, first replacing "**" with "^" and squashing runs of
#    whitespace
# assembler: grabs characters from squasher and packs them into lines
#    with 72 character each, delivering each such line to putline;
#    when it sees a null byte, passes the last line to putline and
#    then kills all the coroutines
# putline: grabs lines from assembler, and just prints them

from coro import coroutine, suspend, main, EarlyExit

def getline(text):
    for line in string.splitfields(text, '\n'):
        codisassembler(line)

def disassembler():
    while 1:
        card = cogetline()
        for i in range(len(card)):
            cosquasher(card[i])
        cosquasher(';')

def squasher():
    while 1:
        ch = codisassembler()
        if ch == '*':
            ch2 = codisassembler()
            if ch2 == '*':
                ch = '^'
            else:
                coassembler(ch)
                ch = ch2
        if ch in ' \t':
            while 1:
                ch2 = codisassembler()
                if ch2 not in ' \t':
                    break
            coassembler(' ')
            ch = ch2
        coassembler(ch)

def assembler():
    line = ''
    while 1:
        ch = cosquasher()
        if ch == '\0':
            break
        if len(line) == 72:
            coputline(line)
            line = ''
        line = line + ch
    line = line + ' ' * (72 - len(line))
    coputline(line)

def putline():
    try:
        while 1:
            line = coassembler()
            print line
    except EarlyExit:
        main() # Hack -- resume main coroutine to exit

import string
cogetline = coroutine(getline, test)
coputline = coroutine(putline)
coassembler = coroutine(assembler)
codisassembler = coroutine(disassembler)
cosquasher = coroutine(squasher)

coputline()
print "done"

raw_input("Exit?")

# end of example


From akuchlin@mems-exchange.org  Fri Nov 17 04:17:15 2000
From: akuchlin@mems-exchange.org (A.M. Kuchling)
Date: Thu, 16 Nov 2000 23:17:15 -0500
Subject: [Python-Dev] Python/Distutil draft PEP
Message-ID: <200011170417.XAA00509@207-172-146-153.s153.tnt3.ann.va.dialup.rcn.com>

PEP: XXX
Title: Using Distutils to Build Python
Version: $Revision:  $
Author: akuchlin@mems-exchange.org (A.M. Kuchling)
Status: Draft
Type: Standards
Created: 16-Nov-2000

Introduction

    The Modules/Setup mechanism has some flaws:

    * People have to remember to uncomment bits of Modules/Setup in
      order to get all the possible modules.

    * Moving Setup to a new version of Python is tedious; new modules
      have been added, so you can't just copy the older
      version, but have to reconcile the two versions.

    * Users have to figure out where the needed libraries, such as
      zlib, are installed.


Proposal

    Use the Distutils to build the modules that come with Python.

    The changes can be broken up into several pieces:

    1. The Distutils needs some Python modules to be able to build
       modules.  Currently I believe the minimal list is posix, _sre,
       and string.

       These modules will have to be built before the Distutils can be
       used, so they'll simply be hardwired into Modules/Makefile and 
       be automatically built.  

    2. A top-level setup.py script will be written that checks the
       libraries installed on the system and compiles 
       as many modules as possible.       

    3. Modules/Setup will be kept and settings in it will override
       setup.py's usual behaviour, so you can disable a module known
       to be buggy, or specify particular compilation or linker flags.
       However, in the common case where setup.py works correctly, 
       everything in Setup will remain commented out.  The other
       Setup.*  become unnecessary, since nothing will be generating 
       Setup automatically.


Unresolved Issues
 
    Do we need to make it possible to disable the 3 hard-wired modules
    without manually hacking the Makefiles?  If yes, perhaps a
    configure switch is sufficient.

    The Distutils always compile modules as shared libraries.  How do
    we support compiling them statically into the resulting Python
    binary?
   
    makesetup and the other contents of $(LIBPL)/config need to be
    preserved for compatibility with existing modules; for how many
    versions do we need to keep them around?


Copyright

    This document has been placed in the public domain.


From moshez@zadka.site.co.il  Fri Nov 17 14:04:53 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 16:04:53 +0200
Subject: [Python-Dev] PEP-228 (fw)
Message-ID: <E13wm8M-00085z-00@darjeeling.zadka.site.co.il>

I'm forwarding this message about PEP-228, but I haven't fully
understood it. In any case, since PEP-228 is apparently not
a 2.1 candidate, I've told Michael that his remarks will be considered,
but they are not relevant yet.

------- Forwarded Message

From: Michael McLay <mclay@nist.gov>
Date: Fri, 17 Nov 2000 04:20:45 -0500 (EST)
To: pep@zadka.site.co.il
Subject: PEP:228

I've dropped off the Python mailing lists for the last couple years
because the volume is too high and I don't have time to contribute to
the discussion.  I do read the summaries and PEP:228 was mentioned in
reference to the 2.1 release.  I'd like to see this problem solve in
the 2.1 release and I have a few thoughts on the matter that may be of
use to this task.  Is there a mailing list specific to PEP:228 or
would I have to join python-dev to participate in the discussion?
Please feel free to forward my message to the python-dev list or use
my comments in PEP:228.

Python's use in the XML community is growing and broad-based XML
support in Python may be key to widespread adoption of Python.  There
is some support for XML Schema, the XML Schema Validation software is
written in Python, but there is no direct support in the core Python
XML library.  One problem area for an XML Schema implementation is
related to the numerical types in Python.  The ability to implement
XML Schema types could be improved if the requirements are considered
when creating the PEP:228 implementation.

And from the Python language perspective a review of the numerical
model used in XML Schema may be useful to the discussion of the
revised numerical model for Python.  The authors of the XML Schema
specification put a good deal of effort into defining a modern,
portable numerical model and I think it might be a good fit with the
Python object model.  I might also be possible that the XML Schema
type and entity definition mechanism would be helpful in the
discussion of defining type interfaces for Python.

A table of simple types built into XML Schema are defined in:

   http://www.w3.org/TR/xmlschema-0/#simpleTypesTable

These are defined more fully in "XML Schema Part 2: DataTypes" at:

    http://www.w3.org/TR/2000/CR-xmlschema-2-20001024/

This includes a discussion of the difference between built-inprimitive
and built-inderived datatypes.

The derived types are defined in the XML Schema at: 

    http://www.w3.org/2000/10/datatypes.xsd

In this document, for instance, an integer is derived from a decimal
type in XML Schema. 

 <simpleType name="integer">
   <restriction base="decimal">
      <scale value="0" fixed="true"/>
   </restriction>
 </simpleType>

The use of XML Schema in conjunction with Python would be very helpful
to a soon to be large domain of applications.  I am working with a
group that is currently defining 20 standards using XML Schema and
this is probably just the tip of the iceburg.  I need to be able to
define constrained numerical types as defined in XML Schma.  In Python
this might look like:

  size = ConstrainedDecimal(initial=2, minInclusive=2, 
				       maxInclusive=3000, scale=0)

  for i in range(3000-2):
    size += 1

  print size

The  ConstrainedDecimal would define a restriction range on the value
of "size".  





------- End of Forwarded Message

-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From just@letterror.com  Fri Nov 17 08:55:48 2000
From: just@letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 09:55:48 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <l03102803b63a9e816862@[193.78.237.158]>

At 7:42 PM -0500 16-11-2000, Guido van Rossum wrote:
[ snipp ]
>Using my proposed API this example could be rewritten a bit
>cleaner, as follows:
>
>    from coro import coroutine, suspend, EarlyExit  # Provisional module name
>
>    def fringe(L):
>	for x in L:
>	    if type(x) is type(L):
>		fringe(x)
>	    else:
>		suspend(x)
>
>    def printinorder(L):
>	c = coroutine(f, L)
>	try:
>	    while 1:
>		print c(),
>	except EarlyExit:
>	    pass
>	print
>
>    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>    printinorder(x) # prints "0 1 2 3 4 5 6"
>
>This is about as simple as it gets.  It supports both generators (like
>the example here) and coroutines (like Tim's Demo/threads/squasher.py
>example).

Neat! (It's _very_ close to what I proposed myself, so how can I not like
it? ;-)

One question: I assume values can also be passed when explicitly resuming a
coroutine. However, the _first_ time a coroutine is "resumed" (started,
really), the stored init arguments get passed to the function. What to do
with a value passed to the resume call? Two possible solutions:
1) don't not allow passing a value the first time
2) add the value(s) to the init arguments

As code:

  def fribble(initarg):
     ...

  c = coroutine(fribble, initarg)
  c(x)  # <-- where does x go? or is it not allowed?

>Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
>propose a function current() which returns the current coroutine
>object.

Starts looking more and more like what I proposed myself...

>There should also be a way to kill a coroutine (or at least
>to send an exception).  When a coroutine falls through at its end,
>*some* other coroutine needs to be resumed.

Here's what I did in my own strawman:
- resume the guy that resumed me, if any (making it equivalent to suspend())
- if we were resumed by a suspend() call (which in my proposal means we
don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
resume the "main" coroutine, which is the (non-coro) code that started the
first coroutine.

Maybe coroutines should have a kill() method, which would post the
EarlyExit exception. Maybe even a postException() method (like uthreads
have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

>I believe this can be implemented with a much simplified stackless
>approach, that doesn't cater towards continuations (but still borrows
>a lot of wisdom from Christian's Stackless).  It can also be
>implemented using threads, which should give some hope for getting the
>same API supported in JPython, making it more attractive.  I am hoping
>to create an implementation on top of Stackless, just to experiment
>with the idiom.

Here's my own code, based on stackless 1.1:
   http://www.petr.com/just/corojust.tar.gz
or
   http://www.petr.com/just/corojust/
for the individual files.

Apart from the initialisation it's pretty much what you propose.

Just




From guido@python.org  Fri Nov 17 12:51:09 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 07:51:09 -0500
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."
 <l03102803b63a9e816862@[193.78.237.158]>
References: <l03102803b63a9e816862@[193.78.237.158]>
Message-ID: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>

(A typically brotherly dialog. :-)

[Guido]
> >This is about as simple as it gets.  It supports both generators (like
> >the example here) and coroutines (like Tim's Demo/threads/squasher.py
> >example).

[Just]
> Neat! (It's _very_ close to what I proposed myself, so how can I not like
> it? ;-)

I looked at what you did, and probably a dozen other examples.

> One question: I assume values can also be passed when explicitly resuming a
> coroutine. However, the _first_ time a coroutine is "resumed" (started,
> really), the stored init arguments get passed to the function. What to do
> with a value passed to the resume call? Two possible solutions:
> 1) don't not allow passing a value the first time
> 2) add the value(s) to the init arguments
> 
> As code:
> 
>   def fribble(initarg):
>      ...
> 
>   c = coroutine(fribble, initarg)
>   c(x)  # <-- where does x go? or is it not allowed?

According to Tim, it's a standard issue that every coroutine API has
to deal with.  I lay awake worrying about this for a long time.  I
read the thread about this in the python-coro list.  I even started
thinking about a very different API, that would disinguish between
get() and put() operations.  But there was nothing that satisfied all
constraints.  My current solution: for the initial transfer, *no
argument is allowed*.  (Except None. :-)  I've looked at several
examples (8 queens, squasher, fringe) and they all work fine under
this assumption.

> >Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> >propose a function current() which returns the current coroutine
> >object.
> 
> Starts looking more and more like what I proposed myself...

Great minds and all that... :-)

I've been thinking about names a bit more, and 'main' is a truly lousy
name.  I now propose a function initial() to get at the main
coroutine.  (A function out of symmetry with current(), but also
because in an environment that also supports OS-level threads, there
is a different main coroutine per OS thread.)

> >There should also be a way to kill a coroutine (or at least
> >to send an exception).  When a coroutine falls through at its end,
> >*some* other coroutine needs to be resumed.
> 
> Here's what I did in my own strawman:
> - resume the guy that resumed me, if any (making it equivalent to suspend())

But there could be a cycle here!  What if A resumed B resumed C
resumed A?

> - if we were resumed by a suspend() call (which in my proposal means we
> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
> resume the "main" coroutine, which is the (non-coro) code that started the
> first coroutine.

But the main coroutine may not be expecting this -- e.g. in the
fringe() example, the main coroutine is expecting the next fringe
item or an EarlyExit exception.

> Maybe coroutines should have a kill() method, which would post the
> EarlyExit exception. Maybe even a postException() method (like uthreads
> have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

Indeed.  This needs a bit more thinking (which so far I've
successfully managed to avoid :-).

> >I believe this can be implemented with a much simplified stackless
> >approach, that doesn't cater towards continuations (but still borrows
> >a lot of wisdom from Christian's Stackless).  It can also be
> >implemented using threads, which should give some hope for getting the
> >same API supported in JPython, making it more attractive.  I am hoping
> >to create an implementation on top of Stackless, just to experiment
> >with the idiom.
> 
> Here's my own code, based on stackless 1.1:
>    http://www.petr.com/just/corojust.tar.gz
> or
>    http://www.petr.com/just/corojust/
> for the individual files.
> 
> Apart from the initialisation it's pretty much what you propose.

I saw that, and even peeked a bit at it.  It seems you must have
struggled a bit with the initialization sequence in the squasher
example, gmcm_coro.py (which BTW is due to Tim Peters in its current
-- Python -- form).  That's alright -- coroutine initialization *is* a
problem.

I've found that finalization is a problem, too: I'm not at all sure
that my version of that example correctly frees all the coroutines
when it is done, since several of them are still suspended (waiting
for more input) when the decision to exit is made.  Also, I had to
invent the 'main' API in order to make it work.  Tim's original
(Demo/threads/squasher.py in the CVS tree) has a kill operation that
makes this easier.  His kill() kills the entire *group* of related
coroutines -- a concept that we don't have and probably don't need: it
just makes using the API more complicated, and Tim has said he'd do
away with it if he had to do it again.

BTW, I've got an almost-working implementation of the coro API using
Stackless continuations!  Problem: I don't know how to raise an
exception for a continuation -- which I need to implement EarlyExit.
Can anyone help?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Fri Nov 17 13:44:51 2000
From: just@letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 14:44:51 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."
 <l03102803b63a9e816862@[193.78.237.158]>
 <l03102803b63a9e816862@[193.78.237.158]>
Message-ID: <l03102807b63ae45cd9f0@[193.78.237.158]>

At 7:51 AM -0500 17-11-2000, Guido van Rossum wrote:
[ initialisation ]
>My current solution: for the initial transfer, *no
>argument is allowed*.

In retrospect, that indeed seems the best solution. I wonder why we (as in
the python-coro@egroups clan) didn't think of that before... Glad to have
you here ;-)


[ "main" coro ]
>I've been thinking about names a bit more, and 'main' is a truly lousy
>name.  I now propose a function initial() to get at the main
>coroutine.  (A function out of symmetry with current(), but also
>because in an environment that also supports OS-level threads, there
>is a different main coroutine per OS thread.)

Fine with me.

>> Here's what I did in my own strawman:
>> - resume the guy that resumed me, if any (making it equivalent to suspend())
>
>But there could be a cycle here!  What if A resumed B resumed C
>resumed A?

Here's what I did: if A does suspend(), it resumes C, but at the same time
A _loses_ it's "resumer". So if it does do a suspend() again, it'll resume
the "initial" coro.

>> - if we were resumed by a suspend() call (which in my proposal means we
>> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
>> resume the "main" coroutine, which is the (non-coro) code that started the
>> first coroutine.
>
>But the main coroutine may not be expecting this -- e.g. in the
>fringe() example, the main coroutine is expecting the next fringe
>item or an EarlyExit exception.

Sure, but you need to be able to go _somewhere_, and this seems more useful
than, say, terminating the app ;-)

[ my own implementiation ]
>> Apart from the initialisation it's pretty much what you propose.
>
>I saw that, and even peeked a bit at it.  It seems you must have
>struggled a bit with the initialization sequence in the squasher
>example, gmcm_coro.py (which BTW is due to Tim Peters in its current
>-- Python -- form).  That's alright -- coroutine initialization *is* a
>problem.

Still, the first-time-no-arguments-allowed rule solves it nicely.

>I've found that finalization is a problem, too: I'm not at all sure
>that my version of that example correctly frees all the coroutines
>when it is done, since several of them are still suspended (waiting
>for more input) when the decision to exit is made.

Are you talking of freeing all data associated with the coroutine, or about
properly finalizing it (so that any finally clauses get executed)? If the
former: don't even try, it's a nightmare with continuations. If the latter:
good question ;-).

>BTW, I've got an almost-working implementation of the coro API using
>Stackless continuations!  Problem: I don't know how to raise an
>exception for a continuation -- which I need to implement EarlyExit.
>Can anyone help?

def hellraiser():
    continuation.return_current()
    raise <whatever>

k = hellraiser()
k.caller = your_co
k()

Just




From tismer@tismer.com  Fri Nov 17 13:47:53 2000
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 15:47:53 +0200
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."             <l03102803b63a9e816862@[193.78.237.158]> <l03102803b63a9e816862@[193.78.237.158]> <l03102807b63ae45cd9f0@[193.78.237.158]>
Message-ID: <3A153709.A56845AC@tismer.com>


Just van Rossum wrote:
...
> >BTW, I've got an almost-working implementation of the coro API using
> >Stackless continuations!  Problem: I don't know how to raise an
> >exception for a continuation -- which I need to implement EarlyExit.
> >Can anyone help?
> 
> def hellraiser():
>     continuation.return_current()
>     raise <whatever>
> 
> k = hellraiser()
> k.caller = your_co
> k()
> 
> Just

Hey, there is it again, the little trick. I love it :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From thomas.heller@ion-tof.com  Fri Nov 17 16:21:50 2000
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 17:21:50 +0100
Subject: [Python-Dev] imputil
Message-ID: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>

imputil, which is now an official part of
python, changes the semantics of sys.path
as soon as importers are installed.

On the other hand, a quick and dirty search
finds at least these files in the standard library
where it is assumed that sys.path is a list of strings:

linecache.py, profile.py, pdb.py, pyclbr.py

Should imputil be fixed to install the ImportManager
in a different way (not installing the importers
into sys.path), or should the library be fixed?

Thomas




From jeremy@alum.mit.edu  Fri Nov 17 04:31:23 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 16 Nov 2000 23:31:23 -0500 (EST)
Subject: [Python-Dev] screwup on the patches & bugs notifications
Message-ID: <14868.46235.22940.869048@bitdiddle.concentric.net>

I hurried to get notifications of outstanding patches and bugs last
night and botched both of them.  Sorry about that.  If you can a
message about patches, you can disregard it entirely.  It was made from
an old snapshot of the SF database.  The bug mail contains current
bugs, but the formatting of the mail headers was off.  I've fixed that
for the future.

Jeremy


From guido@python.org  Fri Nov 17 16:35:09 2000
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 11:35:09 -0500
Subject: [Python-Dev] imputil
In-Reply-To: Your message of "Fri, 17 Nov 2000 17:21:50 +0100."
 <038701c050b2$7dd00740$e000a8c0@thomasnotebook>
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>
Message-ID: <200011171635.LAA26120@cj20424-a.reston1.va.home.com>

> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

Sorry to burst your bubble.  Imputil is still experimental.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Nov 17 16:45:58 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 17:45:58 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>
Message-ID: <3A1560C6.B872D948@lemburg.com>

Thomas Heller wrote:
> 
> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

My understanding was that Importers need to provide
a __str__ method which is then used... haven't looked
at imputil.py in ages though, so I can't really comment.

Perhaps imputil.py should leave sys.path alone (or maybe just
wipe it from unneeded entries) and use a new sys.importers
object for the import management ?!

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fdrake@acm.org  Fri Nov 17 16:58:47 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 17 Nov 2000 11:58:47 -0500 (EST)
Subject: [Python-Dev] forwarded message from nimoy
Message-ID: <14869.25543.871007.185471@cj42289-a.reston1.va.home.com>

--JgZtNmo+G5
Content-Type: text/plain; charset=us-ascii
Content-Description: message body and .signature
Content-Transfer-Encoding: 7bit


  It sounds like people are happy with Python.  ;)  Just thought I'd
pass this bit of praise araound, since it seems to apply to everyone's
efforts.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations


--JgZtNmo+G5
Content-Type: message/rfc822
Content-Description: forwarded message
Content-Transfer-Encoding: 7bit

Return-Path: <jtnimoy@ucla.edu>
Received: from h14.mail.home.com ([24.0.95.48]) by mail.rdc1.md.home.com
          (InterMail vM.4.01.03.00 201-229-121) with ESMTP
          id <20001117053626.LAIP10139.mail.rdc1.md.home.com@h14.mail.home.com>
          for <fdrake01@mail.reston1.va.home.com>;
          Thu, 16 Nov 2000 21:36:26 -0800
Received: from mx6-w.mail.home.com (mx6-w.mail.home.com [24.0.95.71])
	by h14.mail.home.com (8.9.3/8.9.0) with ESMTP id VAA12246
	for <fdrake01@home.com>; Thu, 16 Nov 2000 21:36:26 -0800 (PST)
Received: from mail.acm.org (mail.acm.org [199.222.69.4])
	by mx6-w.mail.home.com (8.9.1/8.9.1) with ESMTP id VAA00618
	for <fdrake01@home.com>; Thu, 16 Nov 2000 21:36:25 -0800 (PST)
Received: from dinsdale.python.org (dinsdale.cnri.reston.va.us [132.151.1.21])
	by mail.acm.org (8.9.3/8.9.3) with ESMTP id AAA35788
	for <fdrake@acm.org>; Fri, 17 Nov 2000 00:36:12 -0500
Received: by dinsdale.python.org (Postfix)
	id B86811D27C; Fri, 17 Nov 2000 00:36:20 -0500 (EST)
Delivered-To: python-docs@python.org
Received: from panther.noc.ucla.edu (panther.noc.ucla.edu [169.232.10.21])
	by dinsdale.python.org (Postfix) with ESMTP id 165371D0B1
	for <python-docs@python.org>; Fri, 17 Nov 2000 00:36:20 -0500 (EST)
Received: from nimoy (lsanca1-ar2-013-122.dsl.gtei.net [4.33.13.122])
	by panther.noc.ucla.edu (8.9.1a/8.9.1) with SMTP id VAA10956
	for <python-docs@python.org>; Thu, 16 Nov 2000 21:36:19 -0800 (PST)
Message-ID: <015a01c05056$f0b24b80$7a0d2104@dsl.vz.genuity.com>
Reply-To: "nimoy" <jtnimoy@ucla.edu>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="Windows-1252"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2919.6600
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600
From: "nimoy" <jtnimoy@ucla.edu>
To: <python-docs@python.org>
Subject: feedback
Date: Thu, 16 Nov 2000 21:26:26 -0800

Hi Python people,

I am a 4th year undergraduate design | media arts student at UCLA in Los
Angeles. My professor showed my Python while I was visiting the MIT Media
laboratory and then later I used it control the CGI for a communication site
for the department. I just want to say thank you and tell you that the site
is doing fine by itself. No one has touched the server in over a year!

Thanks

-Josh Nimoy

Here is the site:
http://forum.design.ucla.edu



--JgZtNmo+G5--



From thomas.heller@ion-tof.com  Fri Nov 17 17:24:17 2000
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 18:24:17 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook> <3A1560C6.B872D948@lemburg.com>
Message-ID: <04a501c050bb$372df8c0$e000a8c0@thomasnotebook>

> Thomas Heller wrote:
> > 
> > imputil, which is now an official part of
> > python, changes the semantics of sys.path
> > as soon as importers are installed.
> > 
> > On the other hand, a quick and dirty search
> > finds at least these files in the standard library
> > where it is assumed that sys.path is a list of strings:
> > 
> > linecache.py, profile.py, pdb.py, pyclbr.py
> > 
> > Should imputil be fixed to install the ImportManager
> > in a different way (not installing the importers
> > into sys.path), or should the library be fixed?
[MAL]
> 
> My understanding was that Importers need to provide
> a __str__ method which is then used... haven't looked
> at imputil.py in ages though, so I can't really comment.
> 
No importer currently does define __str__, neither is it used
in the standard libary. The pattern is (in linecache):

        for dirname in sys.path:
            fullname = os.path.join(dirname, basename)

> Perhaps imputil.py should leave sys.path alone (or maybe just
> wipe it from unneeded entries) and use a new sys.importers
> object for the import management ?!
This sounds better.

Thomas




From mal@lemburg.com  Fri Nov 17 17:13:42 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 18:13:42 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
Message-ID: <3A156746.D2B59356@lemburg.com>

Hi,

sorry to bother you again with this, but the current state
of Python's community elements is in such a bad shape, that
I'm getting more and more frustrated...

First, starship is taken off the net due to a mailing problem.
After a few days starship resurfaces: but apparently using
some really old backup which didn't even contain all files
-- at least the .ssh dirs seem to be lost causing RSA logins
using SSH to fail (don't know about normal logins -- my old
passwords don't work anymore either, but that could have been
caused by the harddisk crash earlier this year).

As consequence, I moved all my Python Pages to a new site
which should provide more stability. Next, I tried to get this
message across to the comp.lang.python.* crowd: I posted
messages to the python.org mailing lists which should have
been gatewayed to the newsgroups -- nothing happened. The
messages seem to be stuck somewhere half-way across the 
Atlantic.

The result: even though I tried to do some good, by moving my
site, noone will notice and I still get complaints about
my extensions not working with 2.0 -- the announcement about
the new versions *did* make it to comp.lang.python and the
announcement list: but it still had the starship URL on it,
so now people find the announcement and there's no way
to point them to the new URL (I can't login to starship to
post a notice there...).

Help someone, please.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From akuchlin@mems-exchange.org  Fri Nov 17 17:45:07 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 12:45:07 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <3A156746.D2B59356@lemburg.com>; from mal@lemburg.com on Fri, Nov 17, 2000 at 06:13:42PM +0100
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <20001117124507.G25735@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 06:13:42PM +0100, M.-A. Lemburg wrote:
>As consequence, I moved all my Python Pages to a new site
>which should provide more stability. Next, I tried to get this

Frankly I think this is the right move, and my pages are now at amk.ca
for pretty much the same reason.  

>message across to the comp.lang.python.* crowd: I posted
>messages to the python.org mailing lists which should have
>been gatewayed to the newsgroups -- nothing happened. The

News/mail gatewaying seems fairly seriously broken at the moment; I've
noticed many followups to posts that I've never seen.  I read c.l.p on
news.cnri.reston.va.us, presumably the same machine used for NNTP
posting by python.org's Mailman, which would imply that Mailman is at
fault.  

--amk


From tismer@tismer.com  Fri Nov 17 16:59:57 2000
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 18:59:57 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A15640D.C2412AB1@tismer.com>

Marc,

"M.-A. Lemburg" wrote:

[justified complaints about Starship]

I'd like to apologize for this mess, but I'd be happier
if I were guilty. Instead, I'm unable to do anything.
Since the Starship has begun to move
again, it got out of my control. I still don't have root
access to the new machine via ssh. This might be by chance,
at least I hope so!
This situation is absolutely what I do not want.
If things don't improve quickly, I might consider
to move the domain to a better server. Moving to
BeOpen should have been a good thing. Finally it
wasn't, but nobody could predict this. Then Starship
went into some Zombie state and was unaccessible
when the DC site was built. Meanwhile the old ship
re-appeared under a different IP, and Jim Tittsler
offered to do a transfer of the current data, but he
also seems to have no super user for the new site.

BTW, you might have ssh access via port 10022, at least
for me this works just fine:

ssh -v -l pirx -p 10022 starship.python.net

Marc, you are the first Starship user! If you leave,
I consider the project dead. Please don't.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From barry@digicool.com  Fri Nov 17 17:59:36 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 12:59:36 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
 <20001117124507.G25735@kronos.cnri.reston.va.us>
Message-ID: <14869.29192.697233.746198@anthem.concentric.net>

>>>>> "AK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

    AK> News/mail gatewaying seems fairly seriously broken at the
    AK> moment; I've noticed many followups to posts that I've never
    AK> seen.  I read c.l.p on news.cnri.reston.va.us, presumably the
    AK> same machine used for NNTP posting by python.org's Mailman,
    AK> which would imply that Mailman is at fault.

Not necessarily.

First, there's certainly nothing in the Mailman log files indicating
any problems writing to nntp.  Second, Mailman on python.org actually
gates messages to ournews.cnri.reston.va.us which is an internal
machine on the CNRI network, and should be one hop downstream from
news.cnri.reston.va.us (a.k.a. news.alterdial.uu.net).

My prediction is that the news is piling up on ournews and there are
problems feeding those messages upstream to alterdial.  This is not a
rare occurance unfortunately.  A while back I contacted the CNRI
admins about this, but I don't remember hearing a positive resolution.

I'm reluctant to point python-list's newserver directly at the
alterdial machine because I remember there were worse problems with
that approach (like postings actually getting lost as opposed to just
sitting behind a log jam -- ournews never expires c.l.py.* postings).

Andrew, maybe you can ping the folks at CNRI more directly to at least
make sure that there are no problems between ournews<->news?

-Barry


From akuchlin@mems-exchange.org  Fri Nov 17 18:08:56 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 13:08:56 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <14869.29192.697233.746198@anthem.concentric.net>; from barry@digicool.com on Fri, Nov 17, 2000 at 12:59:36PM -0500
References: <3A156746.D2B59356@lemburg.com> <20001117124507.G25735@kronos.cnri.reston.va.us> <14869.29192.697233.746198@anthem.concentric.net>
Message-ID: <20001117130856.C7763@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 12:59:36PM -0500, Barry A. Warsaw wrote:
>My prediction is that the news is piling up on ournews and there are
>problems feeding those messages upstream to alterdial.  This is not a

And I believe you're correct; pointing a newsreader at ournews, I can
see postings (such as MAL's "ANN: New URL for the mx-Extensions") that
I never saw when reading news on uunet's server, so this is another
example of CNRI's incompetent sysadmining.  I've sent a note off
asking them to look into it.

--amk


From barry@digicool.com  Fri Nov 17 18:10:03 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 13:10:03 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
 <3A15640D.C2412AB1@tismer.com>
Message-ID: <14869.29819.597320.836410@anthem.concentric.net>

>>>>> "CT" == Christian Tismer <tismer@tismer.com> writes:

    CT> I still don't have root access to the new machine via
    CT> ssh. This might be by chance, at least I hope so!

Gotta be, because /nobody/ had root access via ssh until moments ago
(I have root access to the underlying machine -- outside the chroot
jail).  I just cat'd my authorized_keys file, as well as pirx's and
jwt's to starship's root account, so Chris, Jim Tittsler and I at
least should be able to get in as root@starship.python.net.

This should let Jim do any necessary unpacking of transferred data.

-Barry


From tismer@tismer.com  Fri Nov 17 17:19:07 2000
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 19:19:07 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
 <3A15640D.C2412AB1@tismer.com> <14869.29819.597320.836410@anthem.concentric.net>
Message-ID: <3A15688B.B69C7878@tismer.com>


"Barry A. Warsaw" wrote:
> 
> >>>>> "CT" == Christian Tismer <tismer@tismer.com> writes:
> 
>     CT> I still don't have root access to the new machine via
>     CT> ssh. This might be by chance, at least I hope so!
> 
> Gotta be, because /nobody/ had root access via ssh until moments ago
> (I have root access to the underlying machine -- outside the chroot
> jail).  I just cat'd my authorized_keys file, as well as pirx's and
> jwt's to starship's root account, so Chris, Jim Tittsler and I at
> least should be able to get in as root@starship.python.net.
> 
> This should let Jim do any necessary unpacking of transferred data.

Ahhh, puuuh, that's great. :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From mal@lemburg.com  Fri Nov 17 19:55:13 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 20:55:13 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com> <3A15640D.C2412AB1@tismer.com>
Message-ID: <3A158D21.134A44EE@lemburg.com>

Christian Tismer wrote:

[This trick got me started again... thanks.]
> BTW, you might have ssh access via port 10022, at least
> for me this works just fine:
> 
> ssh -v -l pirx -p 10022 starship.python.net
> 
> Marc, you are the first Starship user! If you leave,
> I consider the project dead. Please don't.

Sorry, Christian, but things got so much out of hand
lately, that I think a new site for my stuff will make
my users happier.

With the help of Andrew I have added redirects for all
my files on starship to the new site, so the move should
be painless (the old URLs will continue to work). I'm
sort of a virtual starship user now ;-)

Anyway, starship is not dead -- it has been a great
community site for the past three years and I hope that
it will now regain the status it had until early this
year when things started to go wrong due to the various
well-known problems.

Let's take this relaunch as a chance to get things moving
again. I'd suggest to restart providing everybody in the Python 
community should with an account on the server to host Python
software in return for the annual fee (what was the PSA-membership
fee until recently). This should by the sysadmins enough
pizza to keep the server running for a looong time ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Sat Nov 18 12:51:16 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 18 Nov 2000 13:51:16 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A167B44.4D53B796@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Help someone, please.

Thank you to everyone who jumped to the rescue :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From ping@lfw.org  Sat Nov 18 21:57:31 2000
From: ping@lfw.org (Ka-Ping Yee)
Date: Sat, 18 Nov 2000 13:57:31 -0800 (PST)
Subject: [Python-Dev] socket.error should be an IOError?
Message-ID: <Pine.LNX.4.10.10011181352250.504-100000@skuld.kingmanhall.org>

In socketmodule, socket.error is currently just a plain old error
(derived only from Exception).  Looks to me like it should be an IOError.
Do you agree?

 line 2379 of socketmodule.c:

­­­­­­­d = PyModule_GetDict(m);
­­­­­­­PySocket_Error = PyErr_NewException("socket.error", NULL, NULL);
­­­­­­­if (PySocket_Error == NULL)

...just change that first NULL to PyExc_IOError.



-- ?!ng

"All models are wrong; some models are useful."
    -- George Box



From jack@oratrix.nl  Sun Nov 19 22:27:14 2000
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 19 Nov 2000 23:27:14 +0100
Subject: [Python-Dev] site.py, .pth files and unix-dependencies
Message-ID: <20001119222719.5980A1301E4@oratrix.oratrix.nl>

In the course of getting distutils to behave on the Mac I came across
.pth files for the first time (all packages I ever tried until now
used the __init__.py method of advertising themselves) and it turns
out they don't work on the Mac.

There's some funny code in site.py that triggers on "os.sep ==
'/'". I've added an "elif os.sep == ':'" for the mac-specific location 
where site-python is (sys.prefix + :lib:site-python), but checking for 
os.sep in stead of simply checking that sys.platform==posix makes me
wonder whether something infinitely more deep than my understanding
allows me to grasp is going on.

Can someone enlighten me? Is my fix okay and should I check it in?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++


From akuchlin@mems-exchange.org  Mon Nov 20 22:32:30 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 20 Nov 2000 17:32:30 -0500
Subject: [Python-Dev] Too much silence
Message-ID: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>

All the PEPs for 2.1 are supposed to be complete for Dec. 16, and some
of those PEPs are pretty complicated.  I'm a bit worried that it's
been so quiet on python-dev lately, especially after the previous two
weeks of lively discussion.  (And no comments on my Python/Distutils
PEP at all -- not even an assigned number... <sniffle>)

--amk


From nas@arctrix.com  Mon Nov 20 16:52:16 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 08:52:16 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001120085216.A30685@glacier.fnational.com>

No doubt.  I'm still waiting for some feedback on my coercion
patch.  I'd write a PEP but I don't think it would help.  Marc's
web page already explains things in detail.  Also, I don't want
to spend too much time going down a path that Guido and the other
developers are not going to accept.

The patch makes some pretty major changes to Python's internals
(for the better I think).  If something is going to be done for
2.1 then it should happen sooner rather than later.

  Neil


From jeremy@alum.mit.edu  Mon Nov 20 23:39:25 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Mon, 20 Nov 2000 18:39:25 -0500 (EST)
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120085216.A30685@glacier.fnational.com>
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
 <20001120085216.A30685@glacier.fnational.com>
Message-ID: <14873.46637.76321.845888@bitdiddle.concentric.net>

Should Marc's Web page be turned into the PEP then?  I don't have time
to read a patch, but I do have time to read a PEP.

Jeremy


From mal@lemburg.com  Mon Nov 20 23:58:10 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 00:58:10 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
 <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net>
Message-ID: <3A19BA92.1A555299@lemburg.com>

Jeremy Hylton wrote:
> 
> Should Marc's Web page be turned into the PEP then?  I don't have time
> to read a patch, but I do have time to read a PEP.

If someone could do this, I'd appreciate it. I like Neil's
idea of using a type flag to signal "new style number", but apart
from that, I think that the web page pretty much makes things
clear.

The patch would need to be updated to Python 2.0, though -- don't
have time for this myself, so it's up to you.

I completely agree with Neil, that these things should be
done now rather than later: the current coercion mechanism is
a major bottleneck when it comes to implementing new types in C
that are supposed to interoperate with existing builtin types.

What the patch basically does is add Python's instance coercion
support to the C level -- in a backward compatible way.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From nas@arctrix.com  Mon Nov 20 17:54:45 2000
From: nas@arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 09:54:45 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <3A19BA92.1A555299@lemburg.com>; from mal@lemburg.com on Tue, Nov 21, 2000 at 12:58:10AM +0100
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com>
Message-ID: <20001120095445.A30774@glacier.fnational.com>

On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> Jeremy Hylton wrote:
> > Should Marc's Web page be turned into the PEP then?  I don't have time
> > to read a patch, but I do have time to read a PEP.

> The patch would need to be updated to Python 2.0, though -- don't
> have time for this myself, so it's up to you.

My patch _is_ up to date.  My original message also included a
brief description of what the patch does.  For more details look
at the URL to Marc's page that I provided.  There is also PEP 208
(which should probably be updated).

Marc and Moshe: who is currently championing PEP 207 and 208?

Barry or Jeremy: the formatting of 208 on sourceforge seems to be
screwy.  Part of the extended print PEP seems to be included.

I'll increase the traffic on this list yet. :)

  Neil


From tim.one@home.com  Tue Nov 21 01:21:15 2000
From: tim.one@home.com (Tim Peters)
Date: Mon, 20 Nov 2000 20:21:15 -0500
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120095445.A30774@glacier.fnational.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOFIAAA.tim.one@home.com>

[Neil Schemenauer]
> My patch _is_ up to date.  My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).

Yes; PEP 208 should be self-contained.

> Marc and Moshe: who is currently championing PEP 207 and 208?
>
> Barry or Jeremy: the formatting of 208 on sourceforge seems to be
> screwy.  Part of the extended print PEP seems to be included.

PEP 208 is in a severely broken state.  This is the full content of
pep-0208.txt:

====================================
PEP: 208
Title: Reworking the Coercion Model
Version: $Revision: 1.1 $
Owner: davida@activestate.com (David Ascher)
Python-Version: 2.1
Status: Incomplete



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
====================================

No change has been checked in since this original skeleton was added in the
middle of July.  So I guess somebody has been uploading damaged HTML for 208
without going thru CVS and the HTML generation process.  Nothing Barry or
Jeremy can do about that:  somebody has to update the .txt file.

> I'll increase the traffic on this list yet. :)

Guido will eventually chime in on the coercions patch, but is on vacation
until next Monday (however, AFAIK, he didn't release anything first <wink>).
I'll chime in too, but am almost in the same boat.

here-yet-absent-ly y'rs  - tim



From moshez@zadka.site.co.il  Tue Nov 21 13:06:31 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 15:06:31 +0200
Subject: [Python-Dev] Too much silence
In-Reply-To: Message from Neil Schemenauer <nas@arctrix.com>
 of "Mon, 20 Nov 2000 08:52:16 PST." <20001120085216.A30685@glacier.fnational.com>
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>  <20001120085216.A30685@glacier.fnational.com>
Message-ID: <E13yD84-0004hI-00@darjeeling.zadka.site.co.il>

> No doubt.  I'm still waiting for some feedback on my coercion
> patch.  I'd write a PEP but I don't think it would help.  Marc's
> web page already explains things in detail.  Also, I don't want
> to spend too much time going down a path that Guido and the other
> developers are not going to accept.

Neil, you can take the coercion patch from me. I thought I'd
have more time to deal with it then I do. Please let me know.
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From thomas@xs4all.net  Tue Nov 21 08:47:44 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 21 Nov 2000 09:47:44 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001121094744.D540@xs4all.nl>

On Mon, Nov 20, 2000 at 05:32:30PM -0500, Andrew Kuchling wrote:

> I'm a bit worried that it's been so quiet on python-dev lately, especially
> after the previous two weeks of lively discussion.

I'll appologize for my silence, but I can't speak for the rest of -dev ;)
I'm going to be silent for a few months more, though, as I'm swamped in work
and supposed to replace my boss during his 2-month vacation. I'll spend what
little time I have on catalog-sig rather than -dev issues, though I'll try
to keep up with the messages on -dev at least ;P

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From mal@lemburg.com  Tue Nov 21 09:22:56 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 10:22:56 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>
Message-ID: <3A1A3EF0.3584B425@lemburg.com>

Neil Schemenauer wrote:
> 
> On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> > Jeremy Hylton wrote:
> > > Should Marc's Web page be turned into the PEP then?  I don't have time
> > > to read a patch, but I do have time to read a PEP.
> 
> > The patch would need to be updated to Python 2.0, though -- don't
> > have time for this myself, so it's up to you.
> 
> My patch _is_ up to date. 

Cool... sorry, but I didn't have time to look closely.

> My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).
> 
> Marc and Moshe: who is currently championing PEP 207 and 208?

Not me -- got no time to spare.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov 21 11:31:34 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 21 Nov 2000 13:31:34 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
Message-ID: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>

I've looked at the people bugs can be assigned to, and Greg Stein is
missing. Is this purposeful or just an oversight?

Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
would look at it -- it's his code originally, so maybe he does not think
it's a bug.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From gstein@lyra.org  Tue Nov 21 12:34:44 2000
From: gstein@lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 04:34:44 -0800
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 21, 2000 at 01:31:34PM +0200
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
Message-ID: <20001121043444.K21426@lyra.org>

On Tue, Nov 21, 2000 at 01:31:34PM +0200, Moshe Zadka wrote:
> I've looked at the people bugs can be assigned to, and Greg Stein is
> missing. Is this purposeful or just an oversight?
> 
> Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
> would look at it -- it's his code originally, so maybe he does not think
> it's a bug.

Probably an oversight. I've had bugs assigned before, but a month or so ago
I accidentally removed myself from the Python project (damn trash can on the
"my" page is located in a horrible place, with no conformation). Tim put me
back, but there is probably a magic switch somewhere to enable bug
assignments.

I'm going to mail Roy Fielding to see if servers or clients are allowed to
send lower-cased HTTP version strings. Looking at the Apache code, it will
assume HTTP/1.0 if a client sends a lower-cased version. I'm suspicious that
the RFC didn't clarify the case-sensitivity on that string (yes, the bug
submitter is correct that the RFC allows a lower-cased version, but that
just seems wrong/mis-intended).

In any case... pending the input from Roy, I'll deal with the bug.

But one of the SF Python Admins will need to enable my bug db privs to work
on that bug properly.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From fdrake@acm.org  Tue Nov 21 14:41:31 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 09:41:31 -0500 (EST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <20001121043444.K21426@lyra.org>
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
 <20001121043444.K21426@lyra.org>
Message-ID: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > Probably an oversight. I've had bugs assigned before, but a month or so ago
 > I accidentally removed myself from the Python project (damn trash can on the
 > "my" page is located in a horrible place, with no conformation). Tim put me
 > back, but there is probably a magic switch somewhere to enable bug
 > assignments.

  I've fixed the permissions; you're back!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From Moshe Zadka <moshez@math.huji.ac.il>  Tue Nov 21 21:35:49 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Tue, 21 Nov 2000 23:35:49 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011212334230.10616-100000@sundial>

On Tue, 21 Nov 2000, Fred L. Drake, Jr. wrote:

>   I've fixed [Greg Stein's] permissions; you're back!  ;-)

Well, the bright side is that it generated some noise on Python-Dev
<wink>.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From akuchlin@mems-exchange.org  Tue Nov 21 22:06:04 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:06:04 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121170604.B15925@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
>Update of /cvsroot/python/python/dist/src/Lib/xml/dom
>In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
>
>Modified Files:
>	minidom.py 
...

Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
has wrong semantics".

Does the patch check for illegal children (bug #116678).

--amk


From fdrake@acm.org  Tue Nov 21 22:16:52 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:16:52 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121170604.B15925@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121170604.B15925@kronos.cnri.reston.va.us>
Message-ID: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
 > has wrong semantics".

  Sorry; that would have taken just enough more that I didn't want to
get into that today, but it shouldn't be hard.  See my comments on the
bug page (just added).

 > Does the patch check for illegal children (bug #116678).

  No.  There's still seriously little parameter checking in minidom,
and I'm not sure I want to add that.  One of the problems people had
with PyOM was the general weight of the code, and adding all the
checks contributes to that (though I suspect the proxies in PyDOM were
a far more serious contributor).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From akuchlin@mems-exchange.org  Tue Nov 21 22:22:33 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:22:33 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:16:52PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172233.A16224@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:16:52PM -0500, Fred L. Drake, Jr. wrote:
>and I'm not sure I want to add that.  One of the problems people had
>with PyDOM was the general weight of the code, and adding all the
>checks contributes to that (though I suspect the proxies in PyDOM were
>a far more serious contributor).

Indeed; I think the proxies really obfuscated the code.  Some simple
parameter checking, though, shouldn't add too much of a burden, and
will protect users from common mistakes that will result in invalid
trees.

--amk



From fdrake@acm.org  Tue Nov 21 22:20:47 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:20:47 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121172233.A16224@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121170604.B15925@kronos.cnri.reston.va.us>
 <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
 <20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Indeed; I think the proxies really obfuscated the code.  Some simple
 > parameter checking, though, shouldn't add too much of a burden, and
 > will protect users from common mistakes that will result in invalid
 > trees.

  So when should we be expecting your patch?  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From akuchlin@mems-exchange.org  Tue Nov 21 22:25:08 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:25:08 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:20:47PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172508.A16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:20:47PM -0500, Fred L. Drake, Jr. wrote:
>  So when should we be expecting your patch?  ;-)

Expect it when you see it. 

--amk


From paulp@ActiveState.com  Tue Nov 21 22:34:29 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 14:34:29 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <3A1AF875.4110F856@activestate.com>

Andrew Kuchling wrote:
> 
> ...
> 
> Indeed; I think the proxies really obfuscated the code.  Some simple
> parameter checking, though, shouldn't add too much of a burden, and
> will protect users from common mistakes that will result in invalid
> trees.

Those checks would slow down the original tree building unless we split
the interface into "internal" methods that we use ourself and "external
methods" that do the extra checking. Worth the effort and extra code
complexity? Maybe...maybe not.

 Paul Prescod


From jeremy@alum.mit.edu  Tue Nov 21 22:26:14 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:26:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <3A1AF875.4110F856@activestate.com>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121170604.B15925@kronos.cnri.reston.va.us>
 <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
 <20001121172233.A16224@kronos.cnri.reston.va.us>
 <3A1AF875.4110F856@activestate.com>
Message-ID: <14874.63110.889245.738951@bitdiddle.concentric.net>

>>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:

  PP> Andrew Kuchling wrote:
  >>
  >> ...
  >>
  >> Indeed; I think the proxies really obfuscated the code.  Some
  >> simple parameter checking, though, shouldn't add too much of a
  >> burden, and will protect users from common mistakes that will
  >> result in invalid trees.

  PP> Those checks would slow down the original tree building unless
  PP> we split the interface into "internal" methods that we use
  PP> ourself and "external methods" that do the extra checking. Worth
  PP> the effort and extra code complexity? Maybe...maybe not.

Could those checks be implemented as assertions?  If so, people who
care about speed can use "python -O"

Jeremy



From akuchlin@mems-exchange.org  Tue Nov 21 22:43:50 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:43:50 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121174350.B16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
>Could those checks be implemented as assertions?  If so, people who
>care about speed can use "python -O"

Making error checks optional and leaving it up to the user to avoid
problems...  did I get subscribed to the perl5-porters list by
mistake?

--amk


From fdrake@acm.org  Tue Nov 21 22:41:32 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:41:32 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121170604.B15925@kronos.cnri.reston.va.us>
 <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
 <20001121172233.A16224@kronos.cnri.reston.va.us>
 <3A1AF875.4110F856@activestate.com>
 <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <14874.64028.658410.793457@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Could those checks be implemented as assertions?  If so, people who
 > care about speed can use "python -O"

  Yes, but it is not clear that the checks are expensive.
  Another issue is compliance with the spec -- DOM level 1 states that
certain exceptions will be raised for various conditions, and using
assertions to check those would mean that exceptions would *not* be
raised in those cases.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From jeremy@alum.mit.edu  Tue Nov 21 22:38:14 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:38:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121174350.B16259@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121170604.B15925@kronos.cnri.reston.va.us>
 <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
 <20001121172233.A16224@kronos.cnri.reston.va.us>
 <3A1AF875.4110F856@activestate.com>
 <14874.63110.889245.738951@bitdiddle.concentric.net>
 <20001121174350.B16259@kronos.cnri.reston.va.us>
Message-ID: <14874.63830.803853.153636@bitdiddle.concentric.net>

>>>>> "AMK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

  AMK> On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
  >> Could those checks be implemented as assertions?  If so, people
  >> who care about speed can use "python -O"

  AMK> Making error checks optional and leaving it up to the user to
  AMK> avoid problems...  did I get subscribed to the perl5-porters
  AMK> list by mistake?

Not quite what I asked about: Enabling error checks by default, but
allowing users to turn them off in optmized mode.  If the checks are
expensive, which Fred subsequently said he wasn't sure about, this
might not be unreasonable.

Perhaps I'm just odd when it comes to -O.  I've never used it.

Jeremy



From gstein@lyra.org  Tue Nov 21 22:55:39 2000
From: gstein@lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:55:39 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121145539.B21426@lyra.org>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
> >>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:
>   PP> Andrew Kuchling wrote:
>   >> ...
>   >> Indeed; I think the proxies really obfuscated the code.  Some
>   >> simple parameter checking, though, shouldn't add too much of a
>   >> burden, and will protect users from common mistakes that will
>   >> result in invalid trees.
> 
>   PP> Those checks would slow down the original tree building unless
>   PP> we split the interface into "internal" methods that we use
>   PP> ourself and "external methods" that do the extra checking. Worth
>   PP> the effort and extra code complexity? Maybe...maybe not.
> 
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

+1 ... that would be the way to do it.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gstein@lyra.org  Tue Nov 21 22:56:52 2000
From: gstein@lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:56:52 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121145652.C21426@lyra.org>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
> Update of /cvsroot/python/python/dist/src/Lib/xml/dom
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
> 
> Modified Files:
> 	minidom.py 
> Log Message:
>...
> *** minidom.py	2000/10/23 18:09:50	1.13
> --- minidom.py	2000/11/21 22:02:22	1.14
> ***************
> *** 15,22 ****
>   """
>   
> - import pulldom
>   import string

This "import string" plus numerous uses within minidom seem a bit extraneous
since this is part of the Python 2.0 library...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From tim.one@home.com  Tue Nov 21 23:09:39 2000
From: tim.one@home.com (Tim Peters)
Date: Tue, 21 Nov 2000 18:09:39 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145539.B21426@lyra.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>

[Andrew Kuchling]
> Indeed; I think the proxies really obfuscated the code.  Some
> simple parameter checking, though, shouldn't add too much of a
> burden, and will protect users from common mistakes that will
> result in invalid trees.

[Paul Prescod]
> Those checks would slow down the original tree building unless
> we split the interface into "internal" methods that we use
> ourself and "external methods" that do the extra checking. Worth
> the effort and extra code complexity? Maybe...maybe not.

[Jeremy Hylton]
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

[Greg Stein]
> +1 ... that would be the way to do it.

-1.  User input is never trustworthy.  Your and your users' software lives
will be a lot happier if you stick to the rule that an assertion failure
always (always!) announces a bug in the implementation -- assertion failure
is never a user's fault.  This makes assertions *possibly* suitable for
Paul's hypothesized "internal methods", but never for checking that
user-supplied arguments satisfy preconditions.

pinning-the-blame-is-90%-of-debugging-and-"assert"-should-pin-it-
    exactly-ly y'rs  - tim



From paulp@ActiveState.com  Wed Nov 22 00:32:50 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 16:32:50 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>
Message-ID: <3A1B1432.E900C248@activestate.com>

Tim Peters wrote:
> 
> ...
> 
> -1.  User input is never trustworthy.  Your and your users' software lives
> will be a lot happier if you stick to the rule that an assertion failure
> always (always!) announces a bug in the implementation -- assertion failure
> is never a user's fault.  

So you prefer

if __debug__ and node.nodeType!=ELEMENT_TYPE: 
    raise TypeError

Unfortunately there's no way to turn that off at "compile time" so you
always incur the __debug__ lookup cost. That would send us back to two
versions of the methods.

Maybe testing would indicate that the performance implications are
minor. If so, I wouldn't mind having the type checks in there.

 Paul Prescod


From tim.one@home.com  Wed Nov 22 01:34:32 2000
From: tim.one@home.com (Tim Peters)
Date: Tue, 21 Nov 2000 20:34:32 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B1432.E900C248@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>

[Tim, objects to abusing assertions]

[Paul Prescod]
> So you prefer
>
> if __debug__ and node.nodeType!=ELEMENT_TYPE:
>     raise TypeError

I personally prefer

if node.nodeType != ELEMENT_TYPE:
    raise TypeError

if that is in fact a correct test of whatever user-input precondition it is
you're verifying.  An assert would be appropriate if it were "impossible"
for the test to fail.

> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.

Actually, there is:

if __debug__:
    if node.nodeType != ELEMENT_TYPE:
        raise TypeError

Python produces no code for that block under -O (btw, this is the same
mechanism that makes asserts vanish under -O:  it's __debug__ that's magic,
not asserts).

As a user, though, I don't expect -O to turn off argument verification!
Same as the Python implementation in these respects:  public API functions
*always* check their arguments, while some private API functions check only
in Debug builds (and via the C library's assert() function, as it's a bug in
the implementation if a private API is misused).

do-s-right-thing-ly y'rs  - tim



From paulp@ActiveState.com  Wed Nov 22 01:56:35 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 17:56:35 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>
Message-ID: <3A1B27D3.8ECDA690@activestate.com>

Tim Peters wrote:
> 
>...
> 
> As a user, though, I don't expect -O to turn off argument verification!
> Same as the Python implementation in these respects:  public API functions
> *always* check their arguments, while some private API functions check only
> in Debug builds (and via the C library's assert() function, as it's a bug in
> the implementation if a private API is misused).

As a user, I don't expect much argument verification from the Python
library at all! C-level verification makes sense because the alternative
is core dumps. That's not acceptable.

For the rare Python-coded function that DOES do argument verification, I
wouldn't have much of an expectation of the affect of "-O" on it
because, like Jeremy, I hardly ever use -O. So maybe the argument is not
worth having -- if nobody ever uses -O then we should always just
balance safety and performance rather than expecting the user to choose
one or the other.

 Paul Prescod


From tim.one@home.com  Wed Nov 22 03:12:44 2000
From: tim.one@home.com (Tim Peters)
Date: Tue, 21 Nov 2000 22:12:44 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B27D3.8ECDA690@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>

[Paul Prescod]
> As a user, I don't expect much argument verification from the Python
> library at all! C-level verification makes sense because the alternative
> is core dumps. That's not acceptable.

Why not?  I don't see any real difference between a core dump and an
uncaught & unexpected Python exception:  in either case the program didn't
get the job done, and left stuff in an unknown state.  Nothing
supernaturally evil about a core dump in that respect; if one is
unacceptable, so is the other.  In the good old days a core dump often
crashed the OS too, but that's rare even on Windows now.

> For the rare Python-coded function that DOES do argument verification, I
> wouldn't have much of an expectation of the affect of "-O" on it
> because, like Jeremy, I hardly ever use -O. So maybe the argument is not
> worth having -- if nobody ever uses -O then we should always just
> balance safety and performance rather than expecting the user to choose
> one or the other.

I don't care about argument verification except to the extent that it
validates preconditions.  If the preconditions for using a public function
aren't stated, the docs are inadequate (what's the user supposed to do then?
guess?).  If the preconditions aren't verified, then a user error may lead
to an incomprehensible error somewhere in the bowels.  Most library
functions have preconditions of the form "x is a sequence" or "y supports
.sort()", and for such builtin types & operations laziness is usually
tolerable because the specific exception raised by accident (in the absence
of checking and the presence of a bad argument) says "not a sequence" or
"can't be .sort()ed" more or less directly.  If you've got fancier
preconditions, an accidental exception likely makes no sense at all to the
user.

Andrew's claim was "some simple parameter checking, though, shouldn't add
too much of a burden, and will protect users from common mistakes that will
result in invalid trees".  I'm arguing both that validation is valuable to
the user and that "-O" is a rotten way to turn it off (even if people *did*
use it -- and I agree that few ever do).  Without any validation first,
though, an argument about how to turn it off-- or whether there's even a
need to --is at best premature.  If Andrew is wrong (that user mistakes
aren't common, or that simple checking couldn't protect users in a useful
way), I haven't heard anyone say so.

if-not-the-rest-is-just-a-question-of-biting-the-bullet-ly y'rs  - tim



From paulp@ActiveState.com  Wed Nov 22 04:13:22 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 20:13:22 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <3A1B47E2.E8E57AE0@activestate.com>

Tim Peters wrote:
> 
>...
>
> Why not?  I don't see any real difference between a core dump and an
> uncaught & unexpected Python exception:  in either case the program didn't
> get the job done, and left stuff in an unknown state.  

A core dump would kill Zope, PythonWin, Alice etc. An exception does
not. To me, that's a big difference. Also, Py_Object type checks are
extremely cheap in C code. And once we put in the parameter checks the
user will get an unexpected Python exception. Presumably they are not
building faulty XML trees on purpose!

Anyhow, I am won over despite your unpersuasive argument.

I note that minidom will not always give you an exception for a poorly
formed tree. That means that the programmer may not find her error until
the XML is "out of Python's hands." It should give an exception sooner
or later but not never.

 Paul Prescod


From fdrake@acm.org  Wed Nov 22 05:19:18 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:19:18 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145652.C21426@lyra.org>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
 <20001121145652.C21426@lyra.org>
Message-ID: <14875.22358.807220.890291@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > This "import string" plus numerous uses within minidom seem a bit extraneous
 > since this is part of the Python 2.0 library...

  An identical copy should be shippable as part of PyXML, which needs
to support Python 1.5.2 as well.  Otherwise, a number of the changes
would be slightly different.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fdrake@acm.org  Wed Nov 22 05:33:38 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:33:38 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
 <3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <14875.23218.734329.350940@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I note that minidom will not always give you an exception for a poorly
 > formed tree. That means that the programmer may not find her error until
 > the XML is "out of Python's hands." It should give an exception sooner
 > or later but not never.

  I'd like to mention again that there's also a matter of compliance
with the specification.  The DOM level 1 recommendation includes
specific documentation about the exceptions that are raised in certain
conditions.  Perhaps we should "map" these to more Pythonic
exceptions, and perhaps not, but I think the exceptions should be
raised when the API specification says they will be.  This is an
important aspect of compliance, and the XML community has demonstrated
substantially more interest in standards compliance than the HTML
community ever did; we should reap the benefits and not end up having
Python discarded because the standard implementation isn't compliant.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fdrake@acm.org  Wed Nov 22 05:35:19 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:35:19 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
References: <3A1B27D3.8ECDA690@activestate.com>
 <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <14875.23319.386269.278252@cj42289-a.reston1.va.home.com>

Tim Peters writes:
 > need to --is at best premature.  If Andrew is wrong (that user mistakes
 > aren't common, or that simple checking couldn't protect users in a useful
 > way), I haven't heard anyone say so.

  Frankly, I found it very helpful to get exceptions when working with
PyDOM.  Some of them annoyed me, but it made me write better code.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From uche.ogbuji@fourthought.com  Wed Nov 22 06:23:14 2000
From: uche.ogbuji@fourthought.com (uche.ogbuji@fourthought.com)
Date: Tue, 21 Nov 2000 23:23:14 -0700
Subject: [Python-Dev] Assertions
In-Reply-To: Message from Paul Prescod <paulp@ActiveState.com>
 of "Tue, 21 Nov 2000 16:32:50 PST." <3A1B1432.E900C248@activestate.com>
Message-ID: <200011220623.XAA13573@localhost.localdomain>

> Tim Peters wrote:
> > 
> > ...
> > 
> > -1.  User input is never trustworthy.  Your and your users' software lives
> > will be a lot happier if you stick to the rule that an assertion failure
> > always (always!) announces a bug in the implementation -- assertion failure
> > is never a user's fault.  
> 
> So you prefer
> 
> if __debug__ and node.nodeType!=ELEMENT_TYPE: 
>     raise TypeError
> 
> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.
> 
> Maybe testing would indicate that the performance implications are
> minor. If so, I wouldn't mind having the type checks in there.

Of course, continuations would be perfect here, as one could set the 
continuation to resume after the check if he wished to bypass it.

Something tells me I should duck now...

not-very-seriously-mind-you'ly yrs

-- 
Uche Ogbuji                               Principal Consultant
uche.ogbuji@fourthought.com               +1 303 583 9900 x 101
Fourthought, Inc.                         http://Fourthought.com 
4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA
Software-engineering, knowledge-management, XML, CORBA, Linux, Python




From gstein@lyra.org  Wed Nov 22 06:33:03 2000
From: gstein@lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 22:33:03 -0800
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>; from paulp@ActiveState.com on Tue, Nov 21, 2000 at 08:13:22PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <20001121223302.L21426@lyra.org>

By definition, isn't minidom supposed to be cheap/easy/quick? The quick
answer to a problem?

If somebody wants an exception-conforming DOM implementation with all the
bells and whistles, then they can go elsewhere. If minidom starts getting
loaded up, then it has kind of defeated its purpose, no?

Cheers,
-g

On Tue, Nov 21, 2000 at 08:13:22PM -0800, Paul Prescod wrote:
> Tim Peters wrote:
> > 
> >...
> >
> > Why not?  I don't see any real difference between a core dump and an
> > uncaught & unexpected Python exception:  in either case the program didn't
> > get the job done, and left stuff in an unknown state.  
> 
> A core dump would kill Zope, PythonWin, Alice etc. An exception does
> not. To me, that's a big difference. Also, Py_Object type checks are
> extremely cheap in C code. And once we put in the parameter checks the
> user will get an unexpected Python exception. Presumably they are not
> building faulty XML trees on purpose!
> 
> Anyhow, I am won over despite your unpersuasive argument.
> 
> I note that minidom will not always give you an exception for a poorly
> formed tree. That means that the programmer may not find her error until
> the XML is "out of Python's hands." It should give an exception sooner
> or later but not never.
> 
>  Paul Prescod
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/


From moshez@zadka.site.co.il  Wed Nov 22 17:33:27 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 19:33:27 +0200
Subject: [Python-Dev] PEP Process Inefficient?
Message-ID: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>

The PEP process, while *miles* better then anything I've seen in any
other development process I've ever seen, has some deficiencies.
I'll try to point them out, and to suggest some partial solution:

1. Users are not sure who to post PEP questions/remarks to: Python dev?
   the original author? The Python devver they know best?
2. It is the responsiblity of the PEP author to add open questions/pertinent
   remarks to the PEP.
3. Mail about the PEP which contains important discussion is lost.

Proposal: (still some shaky areas though):

Why not use Squishdot for PEPs? The PEP would be the "article", and
remarks could be comments. I think (but I'm not sure) Squishdot
has the ability to edit/moderate comments, and retroactively change the
article. If not, then my second best hope is some kind of Wiki system,
with self imposed article/comments-like structure.

Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
has inspired a lot of discussion, and very interesting comments. However
for someone who doesn't remember all the e-mails by heart, wading through
the archives is much too time consuming. In addition, personal mail sent to
me is not available for public discussion. 

Problems with proposed solution: no e-mail notification. I think Zwiki
does have an option for e-mail notification, though. Don't know about
Squishdot.

-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From mal@lemburg.com  Wed Nov 22 09:56:23 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 10:56:23 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <3A1B9847.1E6205AC@lemburg.com>

Moshe Zadka wrote:
> 
> The PEP process, while *miles* better then anything I've seen in any
> other development process I've ever seen, has some deficiencies.
> I'll try to point them out, and to suggest some partial solution:
> 
> 1. Users are not sure who to post PEP questions/remarks to: Python dev?
>    the original author? The Python devver they know best?

Perhaps the PEP should specify this by e.g. using a header
Comments-To: email-address

The email address could then be made "usable" on the PEP web pages.

> 2. It is the responsiblity of the PEP author to add open questions/pertinent
>    remarks to the PEP.
> 3. Mail about the PEP which contains important discussion is lost.

The original idea was for PEP authors to include the important
parts of those discussions in summary in the PEP -- AFAIK, this
is not done :-(
 
> Proposal: (still some shaky areas though):
> 
> Why not use Squishdot for PEPs? The PEP would be the "article", and
> remarks could be comments. I think (but I'm not sure) Squishdot
> has the ability to edit/moderate comments, and retroactively change the
> article. If not, then my second best hope is some kind of Wiki system,
> with self imposed article/comments-like structure.
> 
> Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
> has inspired a lot of discussion, and very interesting comments. However
> for someone who doesn't remember all the e-mails by heart, wading through
> the archives is much too time consuming. In addition, personal mail sent to
> me is not available for public discussion.
> 
> Problems with proposed solution: no e-mail notification. I think Zwiki
> does have an option for e-mail notification, though. Don't know about
> Squishdot.

Hmm, wouldn't something like the SF bug manager be the ideal
platform ? Perhaps someone could get the SF guys to add something
like an "idea" manager with similar functionality but different
content to SF ?!

In the meantime, I think PEP discussions ought to taken to the
sig mailing lists using python-dev as fallback solution.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov 22 10:27:10 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 22 Nov 2000 12:27:10 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1B9847.1E6205AC@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>

On Wed, 22 Nov 2000, M.-A. Lemburg wrote:

> The original idea was for PEP authors to include the important
> parts of those discussions in summary in the PEP -- AFAIK, this
> is not done :-(

I'm trying to do this. However, summarising every e-mail sent to me
about this PEP is quite a burden.

> Hmm, wouldn't something like the SF bug manager be the ideal
> platform ? Perhaps someone could get the SF guys to add something
> like an "idea" manager with similar functionality but different
> content to SF ?!

Yeah, that would work to.

> In the meantime, I think PEP discussions ought to taken to the
> sig mailing lists using python-dev as fallback solution.

I think most of the PEPs are not in the charter of any new sig, and
python-dev is invitation only forum. Perhaps having pep-xxx-sig would
solve it too?

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Wed Nov 22 11:16:09 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 12:16:09 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>
Message-ID: <3A1BAAF8.231B3914@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 22 Nov 2000, M.-A. Lemburg wrote:
> 
> > The original idea was for PEP authors to include the important
> > parts of those discussions in summary in the PEP -- AFAIK, this
> > is not done :-(
> 
> I'm trying to do this. However, summarising every e-mail sent to me
> about this PEP is quite a burden.
> 
> > Hmm, wouldn't something like the SF bug manager be the ideal
> > platform ? Perhaps someone could get the SF guys to add something
> > like an "idea" manager with similar functionality but different
> > content to SF ?!
> 
> Yeah, that would work to.
> 
> > In the meantime, I think PEP discussions ought to taken to the
> > sig mailing lists using python-dev as fallback solution.
> 
> I think most of the PEPs are not in the charter of any new sig, and
> python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> solve it too?

I don't think people would sign up for each and every PEP...
and you know how hard it is to even move a discussion from
one forum to another -- this either doesn't work out or 
kills discussion.

I think the simplest solution would be something like the
SF bug manager. I'm pretty sure that other projects on SF
have similar problems, so this might be worthwhile dicussing
with the SF folks. The technology and user interface are there,
it would only take a few new tables in the database.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fdrake@users.sourceforge.net  Wed Nov 22 12:51:00 2000
From: fdrake@users.sourceforge.net (Fred L. Drake)
Date: Wed, 22 Nov 2000 04:51:00 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011221251.EAA16696@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/


From jeremy@alum.mit.edu  Wed Nov 22 14:50:22 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Wed, 22 Nov 2000 09:50:22 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <14875.56622.102311.641555@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez@zadka.site.co.il> writes:

  MZ> The PEP process, while *miles* better then anything I've seen in
  MZ> any other development process I've ever seen, has some

(Not that we've used the PEP process for much of anything yet.
Except, I guess, extended print, which I've grown quite fond of.)

  MZ> deficiencies.  I'll try to point them out, and to suggest some
  MZ> partial solution:

I don't like your partial solution, so I'll try to pick about the
problems with the process <0.8 wink>.  Put another way: I can't
envision Slashdot as a useful design forum, but am interested in
improving the PEP process.  

  MZ> 1. Users are not sure who to post PEP questions/remarks to:
  MZ>    Python dev?  the original author? The Python devver they know
  MZ>    best?

In the absence of a reason to post elsewhere, the comments ought to be
sent to the PEP author.  If the message is intended to provoke wider
discussion, then the user can post it on any relevent forum (and cc
the PEP author).  A Unicode PEP might be discussed on the i18n-sig; a
Web PEP on the python-web-modules list; a change to core Python on
python-dev. 

I don't think there will ever be a rule that says: "Comments on PEP
1812 must be posted to the pep-1812 web forum."  Discussion should
occur in the forum in which it is most relevent.  Perhaps we could all
send our comments to Andrew, and he could write a bi-weekly
pep-comment summary <wink>.

Is the problem that you don't know where to post comments on someone
else's PEP, or that you are having trouble keeping track of
discussions in multiple places?

  MZ> 2. It is the responsiblity of the PEP author to add open
  MZ>    questions/pertinent remarks to the PEP.

What is the deficiency with this approach?  The PEP is supposed to
present a coherent proposal and design for a new language feature.
It is the designer's responsibility to write a good design document;
several people can share responsibility for the design.

Are you saying it's a problem that the PEPs aren't open to
modification by anyone?  (The last thing we need is someone who
doesn't understand what a first-class function is messing with the
scoping PEP <0.2 wink>.)

The designer is responsible for discussing trade-offs and alternatives
to justify her design.  This is where the responsibility to address
questions and comments comes from.

  MZ> 3. Mail about the PEP which contains important discussion is
  MZ>    lost.

How?  I've got all the mail on the nested static scopes PEP.  Is the
problem just the personal burden of keeping track of lots of mail
messages discussing a PEP?

Any discussion that occurs on a mailing list will be preserved in the
mailing list archive.  That's not lost.  So the only problem would be
with discussion that occurs in private email, where everyone deletes
copies of the mail.

I wonder if the key problem is not having an authoritative mail
archive for the PEP.  One part of the Scheme RFI process that we did
not copy was having a mail archive for each document.  Perhaps this
would be sufficient to address your concerns.  If you, the PEP author,
receive a comment by private email, you could bounce it to the
archive.

the-biggest-problem-with-the-PEP-process-is-no-one-writing-PEPs-ly y'rs,
Jeremy


From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov 22 15:12:28 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:12:28 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.56622.102311.641555@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>

The key problem, as Jeremy noted, was that there is no archive of
discussion about the PEP. I had a much more long winded and roundabout
way of saying that. Also, a minor related problem is that when the
only address on the PEP points to me (for example), I get all the mail
about the PEP personally -- and I'm loath to bounce it to a public forum.
*Having* a public forum which is *indicated in the PEP* would solve that.

I could phrase that as a point-by-point answer to Jeremy (and originally
did) but I decided that that would be idiotic.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From fdrake@acm.org  Wed Nov 22 15:21:01 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:21:01 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
 <Pine.GSO.4.10.10011221657310.14321-100000@sundial>
Message-ID: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > only address on the PEP points to me (for example), I get all the mail
 > about the PEP personally -- and I'm loath to bounce it to a public forum.
 > *Having* a public forum which is *indicated in the PEP* would solve that.

  Having a public forum isn't quite what's needed (though useful);
much of the time, it's more important to have a public *archive*.  I'm
sure there's a way to set things up a mail handler that maintains an
mbox file with collected mails on a topic, and a pipermail browsable
version of that should be available, and each PEP should point to the
appropriate archive.  (It should also clearly state that relevant
emails may be added to the archive at the recipient's discretion!)
  Using a mail handler for this would allow the handler to be added to
the CC: on the emails and generally collect replies without any
additional work on the part of the PEP author.  The handler could be
CC'd when the PEP is posted so that replies to the posting go there as
well.
  Now we just need someone interested in writing a handler.  ;-{


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov 22 15:28:56 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:28:56 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>

On Wed, 22 Nov 2000, Fred L. Drake, Jr. wrote:

>   Using a mail handler for this would allow the handler to be added to
> the CC: on the emails and generally collect replies without any
> additional work on the part of the PEP author.  The handler could be
> CC'd when the PEP is posted so that replies to the posting go there as
> well.
>   Now we just need someone interested in writing a handler.  ;-{

Ummmmm....couldn't that be implemented as a mailman list with *no*
subscribers (subscription requests auto-bounce)
I probably missed something?
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From akuchlin@mems-exchange.org  Wed Nov 22 15:32:45 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:32:45 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>; from gstein@lyra.org on Tue, Nov 21, 2000 at 10:33:03PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com> <20001121223302.L21426@lyra.org>
Message-ID: <20001122103245.A17172@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 10:33:03PM -0800, Greg Stein wrote:
>If somebody wants an exception-conforming DOM implementation with all the
>bells and whistles, then they can go elsewhere. If minidom starts getting
>loaded up, then it has kind of defeated its purpose, no?

Checking for the correct children should be quite fast; in PyDOM it
was basically the line "if newnode.type not in
self._LEGAL_CHILDREN_TYPES: raise ...".  I don't know about the other
minidom bug report, but will try to look into it before too long.

--amk

   Some compilers allow a check during execution that subscripts do not
   exceed array dimensions. This is a help, but not sufficient. First,
   many programmers do not use such compilers because "They're not
   efficient." (Presumably, this means that it is vital to get the wrong
   answers quickly.)
 
   Kernighan and Plauger, in _The Elements of Programming Style_



From fredrik@effbot.org  Wed Nov 22 15:49:38 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Wed, 22 Nov 2000 16:49:38 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <14875.56622.102311.641555@bitdiddle.concentric.net><Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <012f01c0549b$d39cb320$3c6340d5@hagrid>

Fred L. Drake wrote:
>   Having a public forum isn't quite what's needed (though useful);
> much of the time, it's more important to have a public *archive*.  I'm
> sure there's a way to set things up a mail handler that maintains an
> mbox file with collected mails on a topic, and a pipermail browsable
> version of that should be available, and each PEP should point to the
> appropriate archive.  (It should also clearly state that relevant
> emails may be added to the archive at the recipient's discretion!)

http://www.lfw.org/ping/roundup.html

</F>



From akuchlin@mems-exchange.org  Wed Nov 22 15:34:36 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:34:36 -0500
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 22, 2000 at 10:21:01AM -0500
References: <14875.56622.102311.641555@bitdiddle.concentric.net> <Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <20001122103436.B17172@kronos.cnri.reston.va.us>

On Wed, Nov 22, 2000 at 10:21:01AM -0500, Fred L. Drake, Jr. wrote:
>  Using a mail handler for this would allow the handler to be added to
>the CC: on the emails and generally collect replies without any
>additional work on the part of the PEP author.  The handler could be
>CC'd when the PEP is posted so that replies to the posting go there as
>well.

This sounds suspiciously like the nosy lists in ?!ng's Roundup
bug-tracking tool.

--amk


From fdrake@acm.org  Wed Nov 22 15:33:04 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:33:04 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>
References: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
 <Pine.GSO.4.10.10011221727460.14321-100000@sundial>
Message-ID: <14875.59184.806912.900828@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Ummmmm....couldn't that be implemented as a mailman list with *no*
 > subscribers (subscription requests auto-bounce)
 > I probably missed something?

  Definately, but at that point, why disallow subscribers unless
there's already a forum for the PEP?
  I don't object to setting up a mailing list if there isn't one that
fits the bill, but I think Mailman is a fairly heavy way to do the
job, and server resources definately seem scarce these days, so we
don't want to overload what we have!
  (We'd probably also want to add checks that a message hasn't already
been sent along, and deal with forwards a little differently -- we
only want the forwarded message if someone is just archiving a message
received in private email.  Hmmm... this is starting to sound like an
AI problem!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From fdrake@acm.org  Wed Nov 22 15:35:19 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:35:19 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122103436.B17172@kronos.cnri.reston.va.us>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
 <Pine.GSO.4.10.10011221657310.14321-100000@sundial>
 <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
 <20001122103436.B17172@kronos.cnri.reston.va.us>
Message-ID: <14875.59319.351125.5491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > This sounds suspiciously like the nosy lists in ?!ng's Roundup
 > bug-tracking tool.

  Yes, this is very suspicious, indeed!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From trentm@ActiveState.com  Wed Nov 22 16:38:51 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:38:51 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1BAAF8.231B3914@lemburg.com>; from mal@lemburg.com on Wed, Nov 22, 2000 at 12:16:09PM +0100
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com>
Message-ID: <20001122083851.H16729@ActiveState.com>

On Wed, Nov 22, 2000 at 12:16:09PM +0100, M . -A . Lemburg wrote:
> Moshe Zadka wrote:
> > > In the meantime, I think PEP discussions ought to taken to the
> > > sig mailing lists using python-dev as fallback solution.
> > 
> > I think most of the PEPs are not in the charter of any new sig, and
> > python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> > solve it too?
> 
> I don't think people would sign up for each and every PEP...
> and you know how hard it is to even move a discussion from
> one forum to another -- this either doesn't work out or 
> kills discussion.
> 
> I think the simplest solution would be something like the
> SF bug manager. I'm pretty sure that other projects on SF
> have similar problems, so this might be worthwhile dicussing
> with the SF folks. The technology and user interface are there,
> it would only take a few new tables in the database.


A Roundup mailing list with a category for each PEP?

Trent

-- 
Trent Mick
TrentM@ActiveState.com


From trentm@ActiveState.com  Wed Nov 22 16:40:40 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:40:40 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>; from trentm@ActiveState.com on Wed, Nov 22, 2000 at 08:38:51AM -0800
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com> <20001122083851.H16729@ActiveState.com>
Message-ID: <20001122084040.I16729@ActiveState.com>

On Wed, Nov 22, 2000 at 08:38:51AM -0800, Trent Mick wrote:
> 
> A Roundup mailing list with a category for each PEP?
> 

Or I should just read all the thread first! Y'all are way ahead of me.


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From Moshe Zadka <moshez@math.huji.ac.il>  Wed Nov 22 16:41:50 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Wed, 22 Nov 2000 18:41:50 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>
Message-ID: <Pine.GSO.4.10.10011221840420.19237-100000@sundial>

On Wed, 22 Nov 2000, Trent Mick wrote:

> A Roundup mailing list with a category for each PEP?

OK, when we get over the make-Moshe-feel-stupid-for-not-proposing-roundup
part, just let me know <wink>. Yep, Roundup is perfect for this!

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From barry@digicool.com  Wed Nov 22 19:19:54 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Wed, 22 Nov 2000 14:19:54 -0500
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
 <20001120085216.A30685@glacier.fnational.com>
 <14873.46637.76321.845888@bitdiddle.concentric.net>
 <3A19BA92.1A555299@lemburg.com>
 <20001120095445.A30774@glacier.fnational.com>
Message-ID: <14876.7258.137276.784970@anthem.concentric.net>

>>>>> "NS" == Neil Schemenauer <nas@arctrix.com> writes:

    NS> Barry or Jeremy: the formatting of 208 on sourceforge seems to
    NS> be screwy.  Part of the extended print PEP seems to be
    NS> included.

Something's really screwed up here.  The pep-0208.txt file is almost
empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
that I see on the web page.  I just did a "cvs up" and a "cvs log" and
I don't see anything new checked in for pep 208.

Did somebody forget to commit their changes to CVS?
-Barry


From fdrake@acm.org  Wed Nov 22 20:44:59 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 15:44:59 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
 <3A1B47E2.E8E57AE0@activestate.com>
 <20001121223302.L21426@lyra.org>
Message-ID: <14876.12363.340244.877950@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > By definition, isn't minidom supposed to be cheap/easy/quick? The quick
 > answer to a problem?
 > 
 > If somebody wants an exception-conforming DOM implementation with all the
 > bells and whistles, then they can go elsewhere. If minidom starts getting
 > loaded up, then it has kind of defeated its purpose, no?

  I've started a new discussion about this over in the XML SIG; if
you're interested, please join in on that list.  If you haven't been
following it, you might want to check the archives starting with
yesterday.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From cgw@fnal.gov  Wed Nov 22 21:20:05 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Wed, 22 Nov 2000 15:20:05 -0600 (CST)
Subject: [Python-Dev] more Sourceforge CVS screwiness
Message-ID: <14876.14469.534078.512149@buffalo.fnal.gov>

Haven't seen this for a while - I was unable to build the most current
version of Python from CVS and was about to squawk, when I realized
that the problem was a bogus "config.h" in my dist/src/Include
directory:

shell$ pwd
/usr/local/src/Python-CVS/python/dist/src/Include
shell$ cvs status config.h                                                                             
===================================================================
File: config.h          Status: Up-to-date

   Working revision:    2.1
   Repository revision: 2.1     /cvsroot/python/python/dist/src/Include/Attic/config.h,v
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      (none)


A fresh checkout of the whole tree does not contain this file.  I know
how to fix this (edit CVS/Entries) but it troubles me that old files
from the Attic keep re-appearing when I do updates (without any
sticky tags or anything, I always do "cvs update -dP")

Am I the only person who sees this?  Does Sourceforge just not like me
for some reason?



From moshez@zadka.site.co.il  Thu Nov 23 06:46:51 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Thu, 23 Nov 2000 08:46:51 +0200
Subject: [Python-Dev] Too much silence
In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw)
 of "Wed, 22 Nov 2000 14:19:54 EST." <14876.7258.137276.784970@anthem.concentric.net>
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>  <14876.7258.137276.784970@anthem.concentric.net>
Message-ID: <E13yq9k-0007Ci-00@darjeeling.zadka.site.co.il>

> Something's really screwed up here.  The pep-0208.txt file is almost
> empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
> that I see on the web page.  I just did a "cvs up" and a "cvs log" and
> I don't see anything new checked in for pep 208.
> 
> Did somebody forget to commit their changes to CVS?

I did, when you define forget to include "didn't have the time yet
to translate the web page to PEP".
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From thomas@xs4all.net  Wed Nov 22 22:40:57 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 22 Nov 2000 23:40:57 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <14876.7258.137276.784970@anthem.concentric.net>; from barry@digicool.com on Wed, Nov 22, 2000 at 02:19:54PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com> <14876.7258.137276.784970@anthem.concentric.net>
Message-ID: <20001122234057.H540@xs4all.nl>

On Wed, Nov 22, 2000 at 02:19:54PM -0500, Barry A. Warsaw wrote:

[ The HTML version of PEP 208 is screwed ]

> Did somebody forget to commit their changes to CVS?

-rw-rw-r--   1 nowonder python      25515 Nov 10 09:31 pep-0208.html

Peter-SK wrote these versions to disk, it seems, but it could have been
edited later. Maybe he can check his .txt version to see whether that's
garbled as well ?

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From akuchlin@mems-exchange.org  Thu Nov 23 16:37:22 2000
From: akuchlin@mems-exchange.org (A.M. Kuchling)
Date: Thu, 23 Nov 2000 11:37:22 -0500
Subject: [Python-Dev] Regression testing
Message-ID: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>

This morning I was looking at test_minidom.py, which contains its own
simplistic framework for running a series of regression tests.  Maybe
another 2.1 task could be selecting a standard unit testing framework;
this should mostly be a matter of selecting one, since there are
already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
and the unittest.py from Quixote, and have a vague feeling I'm
forgetting one other module.  No prizes for guessing which one I'd
vote for. :)

--amk



From trentm@ActiveState.com  Thu Nov 23 16:48:30 2000
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 23 Nov 2000 08:48:30 -0800
Subject: [Python-Dev] Regression testing
In-Reply-To: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>; from akuchlin@mems-exchange.org on Thu, Nov 23, 2000 at 11:37:22AM -0500
References: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>
Message-ID: <20001123084830.I31441@ActiveState.com>

On Thu, Nov 23, 2000 at 11:37:22AM -0500, A.M. Kuchling wrote:
> This morning I was looking at test_minidom.py, which contains its own
> simplistic framework for running a series of regression tests.  Maybe
> another 2.1 task could be selecting a standard unit testing framework;
> this should mostly be a matter of selecting one, since there are
> already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
> and the unittest.py from Quixote, and have a vague feeling I'm
> forgetting one other module.  No prizes for guessing which one I'd
> vote for. :)

Sure, if they provide a clear benefit. I.e. who will want to translate
all the existing tests. As well, I have a patch (which, I admit, I haven't
been maintaining or pushing much) to allow direct testing of the C API). I
have no experience with either of the unit testing frameworks you mention.
Will they be able to do that? Presumably yes, using the same hack that I
proposed.

http://sourceforge.net/patch/?func=detailpatch&patch_id=101162&group_id=5470

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From lutz@rmi.net  Sat Nov 25 15:12:56 2000
From: lutz@rmi.net (Mark Lutz)
Date: Sat, 25 Nov 2000 08:12:56 -0700
Subject: [Python-Dev] A house upon the sand
Message-ID: <004401c056f2$34a9ee00$dc73fea9@vaio>

Hi again Guido,

On a totally unrelated topic: has anyone pushed the idea of a
Python language standard lately?  Things seem to be changing too
fast for many developers to keep up, and it seems to me that a
formal standard doc might help ease a few fears I've seen out 
there.  Books used to be a sort of de facto standard, but even
they aren't reliable anymore; and the manuals aren't useful as a 
standard if they are open to arbitrary change every few months.

Frankly, some people in my classes are very concerned by the rapid 
pace of Python change, and I think their fear is justified.  I get 
burned a little almost every time a new Python release rolls out
too.  Most recently, some new book examples that worked in 1.5.2 
this summer no longer work under 2.0 this fall; I understand that
most changes are improvements (and minor), but this is not a great
story to tell.

A prime example: the string module, used in almost every Python 
program ever written by the half-million Python users out there,
has suddenly been marked as deprecated.  I expect that it won't 
really go away, but has anyone considered the impact of even the
suggestion of its deprecation on Python's acceptance?

If using Python requires that programmers invest lots of time 
tracking the whims of python-dev, then Python will become much 
less useful, imo.  Most developers just don't have the extra time
to spare.  A formal standard doc could give us at least a baseline
Python definition that developers could cling to.  Companies need 
to see a solid and reliable foundation.

Unfortunately, I don't have the time or interest in pushing this
idea through to fruition myself.  Do you have any ideas along these 
lines?  Maybe this task belongs in whatever body eventually takes 
over ownership.  I'm copying this to python-dev in the hopes that 
it might trigger some sort of discussion.

Cheers,
--Mark Lutz  (http://rmi.net/~lutz)



From mal@lemburg.com  Sat Nov 25 16:21:06 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 17:21:06 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A1FE6F2.6A612EF9@lemburg.com>

Mark Lutz wrote:
> 
> Frankly, some people in my classes are very concerned by the rapid
> pace of Python change, and I think their fear is justified.  I get
> burned a little almost every time a new Python release rolls out
> too.  Most recently, some new book examples that worked in 1.5.2
> this summer no longer work under 2.0 this fall; I understand that
> most changes are improvements (and minor), but this is not a great
> story to tell.
> 
> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

It is depreciated because string methods provide a better
model of extensibility for future versions. string.py simply
interfaces to these new methods. It won't go away, but using the
methods directly will provide better performance and an
overall better experience... this doesn't mean that Python
programmers can no longer use string.py, but it should hint
them towards using the string methods instead.

Next to come are number methods, AFAICT ;-)
 
> If using Python requires that programmers invest lots of time
> tracking the whims of python-dev, then Python will become much
> less useful, imo.  Most developers just don't have the extra time
> to spare.  A formal standard doc could give us at least a baseline
> Python definition that developers could cling to.  Companies need
> to see a solid and reliable foundation.

The only major incompatbile changes in 2.0 are the .append()
et al. changes and the str(1L) change. These have been
highlighted in the changes paper.
 
> Unfortunately, I don't have the time or interest in pushing this
> idea through to fruition myself.  Do you have any ideas along these
> lines?  Maybe this task belongs in whatever body eventually takes
> over ownership.  I'm copying this to python-dev in the hopes that
> it might trigger some sort of discussion.

I don't see how a Python standard would do any good. Standards
have version numbers as well and change at about the same rate
(e.g. take Unicode 2.0 vs. Unicode 3.0).

Besides, who forces anyone to use 2.0 instead of 1.5.x which
has been around for many years now ?

Sorry, but I sense bureaucracy kreeping into the house...
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From tismer@tismer.com  Sat Nov 25 16:25:32 2000
From: tismer@tismer.com (Christian Tismer)
Date: Sat, 25 Nov 2000 18:25:32 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <3A1FE7FC.AF14EF85@tismer.com>

(Marc, your clock is wrong by 1 hour :)

"M.-A. Lemburg" wrote:
> 
> Mark Lutz wrote:
...
> > Unfortunately, I don't have the time or interest in pushing this
> > idea through to fruition myself.  Do you have any ideas along these
> > lines?  Maybe this task belongs in whatever body eventually takes
> > over ownership.  I'm copying this to python-dev in the hopes that
> > it might trigger some sort of discussion.
> 
> I don't see how a Python standard would do any good. Standards
> have version numbers as well and change at about the same rate
> (e.g. take Unicode 2.0 vs. Unicode 3.0).
> 
> Besides, who forces anyone to use 2.0 instead of 1.5.x which
> has been around for many years now ?
> 
> Sorry, but I sense bureaucracy kreeping into the house...

Saying things explicitly is Mark's point, IMHO.

I don't see the danger of bureaucracy so much. Instead, setting
a standard is a sign of maturity for a language. Python as it
is, and in former versions, was a per-se standard, set by
the implementation. The Python documentation explains how
things work, but this is a description of the current
implementation, not setting possible ranges, like a standard.
A standard would allow anybody to write a Python implementation
which conforms to it, without knowing all the
details of the given implementation. He would not have to
guess (or ask Guido) what design decisions are final and
considered a part of the language, or just by chance,
ease of implementation or whatever reason might have let to it.
There are many issues where we know how it works
in one implementation and the other, but it is undefined
which behavior is intended, by chance, enforced or forbidden.

Example: Python 2.0 has now a garbage collector. Not so long
ago, a garbage collector seemed out of reach, and before
Guido sketched a GC by himself, I even had the impression
that a gc was out of question. (should have borrowed the
time machine :)
JPython had garbage collection in the first place, inherited
from Java. So, with Python 1.5.2, 2.0, and JPython, we get
three different implementations, and three different
lifetimes of objects.
It would be helpful if a Python standard would define
how an implementation should do this: Do objects have
to be destructed at all, or which objects have to, or
is it completely up to the implementor to decide?
Even if so, then this would be stated, and people
who write scripts compliant to Python standard 2.0
would know that they can never rely on object destruction,
for instance.

Other Example:
There are certain design decisions in the object layout,
which are exposed to the user, like __dict__ for many
objects. It is not clear if this is a language feature
or an implementation detail. The current user practice
enforces us to support __dict__, I think, although
very different decision would be drawn if someone
was designing a true compiler for Python. Does it have
to exist, and if so, does it have to be a dictionary?
Statements like "it happens to be a dict, but anything
supported by getattr() would do, but it must be there"
would be an example statement, most helpful for alternative
implementations, which still fulfill the standard.

better-explicit-than-implicit - ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From mal@lemburg.com  Sat Nov 25 17:59:10 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 18:59:10 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com>
Message-ID: <3A1FFDEE.E2321B40@lemburg.com>

Christian Tismer wrote:
> 
> > Mark Lutz wrote:
> ...
> > > Unfortunately, I don't have the time or interest in pushing this
> > > idea through to fruition myself.  Do you have any ideas along these
> > > lines?  Maybe this task belongs in whatever body eventually takes
> > > over ownership.  I'm copying this to python-dev in the hopes that
> > > it might trigger some sort of discussion.
> >
> > I don't see how a Python standard would do any good. Standards
> > have version numbers as well and change at about the same rate
> > (e.g. take Unicode 2.0 vs. Unicode 3.0).
> >
> > Besides, who forces anyone to use 2.0 instead of 1.5.x which
> > has been around for many years now ?
> >
> > Sorry, but I sense bureaucracy kreeping into the house...
> 
> Saying things explicitly is Mark's point, IMHO.
>
> I don't see the danger of bureaucracy so much. Instead, setting
> a standard is a sign of maturity for a language. Python as it
> is, and in former versions, was a per-se standard, set by
> the implementation. The Python documentation explains how
> things work, but this is a description of the current
> implementation, not setting possible ranges, like a standard.
> A standard would allow anybody to write a Python implementation
> which conforms to it, without knowing all the
> details of the given implementation. He would not have to
> guess (or ask Guido) what design decisions are final and
> considered a part of the language, or just by chance,
> ease of implementation or whatever reason might have let to it.
> There are many issues where we know how it works
> in one implementation and the other, but it is undefined
> which behavior is intended, by chance, enforced or forbidden.
> 
> [GC Example, __dict__ example]

Ok. Maybe I misunderstood Mark... if what he wants is a language
definition like e.g. the one for Java and its VM then I agree.
This would be a good thing. OTOH, I don't see much of a point
in giving the user a feeling of fixed standards when we are 
constantly talking about ways to break code ;-) (e.g. take the
Py3K upgrade path PEP).

We could have a language definition for say the 2.0 version
series, but the definition would have to be extended for
the 3.0 series (possibly even within the 2.0 series). It would
gives us a feeling of what "Python" stands for and also
simplify the task of writing different Python implementations,
but nothing much else... certainly not the warm fuzzy feeling
which Mark probably had in mind.

I think the overall impression of Python 2.0 being a moving target
is a bit overrated: there aren't really all that many changes
which affect existing code, so a Python 1.5.2 introduction will
still remain mostly valid even when the students use Python 2.0.

It is true, that Python 2.0 offers a lot more features than
1.5.2 and that in some contexts (e.g. database interfacing
on Windows) these new features have made their way into third
party extensions rather fast (e.g. switching from strings to
Unicode). The visible core language hasn't changed much though...
and this is goodness.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From thomas@xs4all.net  Sat Nov 25 19:45:10 2000
From: thomas@xs4all.net (Thomas Wouters)
Date: Sat, 25 Nov 2000 20:45:10 +0100
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <3A1FE6F2.6A612EF9@lemburg.com>; from mal@lemburg.com on Sat, Nov 25, 2000 at 05:21:06PM +0100
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <20001125204509.I540@xs4all.nl>

On Sat, Nov 25, 2000 at 05:21:06PM +0100, M.-A. Lemburg wrote:
> Mark Lutz wrote:

> > A prime example: the string module, used in almost every Python
> > program ever written by the half-million Python users out there,
> > has suddenly been marked as deprecated.  I expect that it won't
> > really go away, but has anyone considered the impact of even the
> > suggestion of its deprecation on Python's acceptance?

> It is depreciated because string methods provide a better
> model of extensibility for future versions. string.py simply
> interfaces to these new methods. It won't go away, but using the
> methods directly will provide better performance and an
> overall better experience...

Unfortunately, the term 'deprecated' suggests to most people, including me,
that it will be removed in future versions. I'm with Mark here; marking it
deprecated causes a lot of confusions, IMHO.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From tismer@tismer.com  Sun Nov 26 11:35:12 2000
From: tismer@tismer.com (Christian Tismer)
Date: Sun, 26 Nov 2000 13:35:12 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com> <3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <3A20F570.6FCD09F2@tismer.com>


"M.-A. Lemburg" wrote:
> 
> Christian Tismer wrote:
> >
[favorizing a standard]
> > > Mark Lutz wrote:
[standard, to avoid confusion]
...
> Ok. Maybe I misunderstood Mark... if what he wants is a language
> definition like e.g. the one for Java and its VM then I agree.
> This would be a good thing. OTOH, I don't see much of a point
> in giving the user a feeling of fixed standards when we are
> constantly talking about ways to break code ;-) (e.g. take the
> Py3K upgrade path PEP).

Whatever other feelings come into play, a standard would
be a good thing, IMHO.

Aside, the PERL world is thinking into this direction as well.
See this rosler article (also interesting concerning Python:)
http://www.perl.com/pub/2000/06/rosler.html

Since Python is so much cleaner and smaller (still?), this
is a chance to be the winder here.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From tim.one@home.com  Sun Nov 26 22:39:42 2000
From: tim.one@home.com (Tim Peters)
Date: Sun, 26 Nov 2000 17:39:42 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>

On the deprecation of the string module:  where did this idea come from?
I've never seen anything saying that the string module is deprecated.  The
2.0 docs say that the three specific functions string.{atoi, atol, atof} are
deprecated as of 2.0, but that's it.

On the meaning of deprecation:  yes, it means they'll go away someday.

On gc semantics:  Section 3.1 ("Objects, values and types") of the Lang Ref
sez:

    Objects are never explicitly destroyed; however, when they become
    unreachable they may be garbage-collected. An implementation is
    allowed to postpone garbage collection or omit it altogether -- it is
    a matter of implementation quality how garbage collection is
    implemented, as long as no objects are collected that are still
    reachable. (Implementation note: the current implementation uses a
    reference-counting scheme which collects most objects as soon as
    they become unreachable, but never collects garbage containing
    circular references.)

If Python had a standard, the only thing that would change is that the
obsolete parenthetical comment would be removed.

On formal standardization:  not likely.  An ISO/ANSI std takes a minimum of
person-decades of work, and Guido would be unable to make the time even to
review committee documents.  IOW, he won't cooperate, because he can't, and
that's that.  Propose changes to the Lang Ref instead (for example, Guido's
intent was that binary operators of equal precedence be evaluated strictly
left to right in the absence of parens, but the Lang Ref never got around to
saying so).

On the pace of change:  by all objective measures, Python change ended the
day 2.0 was released <0.9 wink>.

the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim



From martin@loewis.home.cs.tu-berlin.de  Sun Nov 26 22:53:54 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 26 Nov 2000 23:53:54 +0100
Subject: [Python-Dev] A house upon the sand
Message-ID: <200011262253.XAA01869@loewis.home.cs.tu-berlin.de>

> the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim

Since Java has been cited as an example of a language that has a
"language definition" - it is just a language reference book, either.
Sun did a number of attempts to make Java an Internation Standard, and
they withdrew when they finally understood what it means.

For Python, it would most likely mean that the only implementations
don't conform to the standard.

Regards,
Martin


From mwh21@cam.ac.uk  Sun Nov 26 23:22:15 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 26 Nov 2000 23:22:15 +0000
Subject: [Python-Dev] A house upon the sand
In-Reply-To: "Tim Peters"'s message of "Sun, 26 Nov 2000 17:39:42 -0500"
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>
Message-ID: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>

"Tim Peters" <tim.one@home.com> writes:

> On the deprecation of the string module:  where did this idea come from?
> I've never seen anything saying that the string module is deprecated.

I thought this, and went looking.  I found on
http://www.python.org/1.6/, about four fifths of the way down:

Changed Modules

    string - most of this module is deprecated now that strings have
    methods. This no longer uses the built-in strop module, but takes
    advantage of the new string methods to provide transparent support
    for both Unicode and ordinary strings.

I hope (and believe) this is Wrong.

http://www.python.org/2.0/new-python.html says:

    The old string module is still around for backwards compatibility,
    but it mostly acts as a front-end to the new string methods.

which is IMHO better.

Cheers,
M.

-- 
  Every day I send overnight packages filled with rabid weasels to
  people who use frames for no good reason.
                             -- The Usenet Oracle, Oracularity #1017-1



From greg@cosc.canterbury.ac.nz  Sun Nov 26 23:56:22 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 27 Nov 2000 12:56:22 +1300 (NZDT)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>

Michael Hudson <mwh21@cam.ac.uk>:

> http://www.python.org/2.0/new-python.html says:
>
>    The old string module is still around for backwards
>    compatibility,

This still suggests that continuing to use it is frowned
upon, though.

I think there are still legitimate reasons for using some
parts of the string module. For example, if you're one of
those stubborn people who refuse to accept that ",".join(foo)
is a better way of writing string.join(foo,",").

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fdrake@acm.org  Mon Nov 27 01:00:48 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 26 Nov 2000 20:00:48 -0500 (EST)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
 <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>

Greg Ewing writes:
 > This still suggests that continuing to use it is frowned
 > upon, though.

  Backward compatible code is still being written, certainly, and not
everything is available as a method (just try ''.letters! ;).

 > I think there are still legitimate reasons for using some
 > parts of the string module. For example, if you're one of
 > those stubborn people who refuse to accept that ",".join(foo)
 > is a better way of writing string.join(foo,",").

  There will never be an excuse for that!  .join() should never have
been added as a method!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From cgw@fnal.gov  Mon Nov 27 01:08:15 2000
From: cgw@fnal.gov (Charles G Waldman)
Date: Sun, 26 Nov 2000 19:08:15 -0600 (CST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
 <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
 <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
Message-ID: <14881.46079.317598.101585@buffalo.fnal.gov>

Fred L. Drake, Jr. writes:

 >   There will never be an excuse for that!  .join() should never have
 > been added as a method!

On the other hand it is not clear to me why "capwords" and "zfill" did
not become methods of string objects. 



From fredrik@effbot.org  Mon Nov 27 01:30:10 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 02:30:10 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov>
Message-ID: <005601c05811$95f391f0$3c6340d5@hagrid>

Charles wrote:
> On the other hand it is not clear to me why "capwords" and "zfill" did
> not become methods of string objects. 

fwiw, they were both present in my original unicode
implementation:

>>> from unicode import unicode
>>> a = unicode("hello world")
>>> a.capwords()
'Hello World'
>>> a.zfill(20)
'000000000hello world'

</F>



From fdrake@acm.org  Mon Nov 27 06:00:01 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 01:00:01 -0500 (EST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.46079.317598.101585@buffalo.fnal.gov>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
 <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
 <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
 <14881.46079.317598.101585@buffalo.fnal.gov>
 <005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <14881.63585.563272.169607@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > On the other hand it is not clear to me why "capwords" and "zfill" did
 > not become methods of string objects. 

Fredrik Lundh writes:
 > fwiw, they were both present in my original unicode
 > implementation:

  Interesting.  I'd have expected capwords() on strings.  I wonder why
/F's implementation wasn't retained?  As for zfill(), I can imagine no
one thought it was sufficiently useful to keep around.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From moshez@zadka.site.co.il  Mon Nov 27 15:06:42 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 17:06:42 +0200
Subject: [Python-Dev] PEP-0218
Message-ID: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>

Well, another proof of the bugs in the PEP process -- my remarks were
lost, so I'll send them here.

Let me note that almost everything Greg Wilson wants to do can be done
via a Python class implementing a set using a dictionary mapping to None.
Almost?

* No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
* Convertors: if we want list/tuple to have a semblance of efficiency,
  we'll need to cache the element list as a list when accessed by
  index.
* Two different constructors: set() for building from sequence, Set()
  for building from elements. Might be confusing.
* Only possible elements of a set are immutables. OTOH, I'm not sure
  how Greg intends to implement his sets if these sets are allowed 
  to contain mutable elements. 


-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From esr@thyrsus.com  Mon Nov 27 07:30:33 2000
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 02:30:33 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>; from moshez@zadka.site.co.il on Mon, Nov 27, 2000 at 05:06:42PM +0200
References: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <20001127023033.A7616@thyrsus.com>

Moshe Zadka <moshez@zadka.site.co.il>:
> Well, another proof of the bugs in the PEP process -- my remarks were
> lost, so I'll send them here.
> 
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.

I agree.  I wrote a full-featured set library long ago.  All it's
waiting for is rich comparisons.  Wait...are those in 2.0?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

It will be of little avail to the people, that the laws are made by
men of their own choice, if the laws be so voluminous that they cannot
be read, or so incoherent that they cannot be understood; if they be
repealed or revised before they are promulgated, or undergo such
incessant changes that no man, who knows what the law is to-day, can
guess what it will be to-morrow. Law is defined to be a rule of
action; but how can that be a rule, which is little known, and less
fixed?
	-- James Madison, Federalist Papers 62


From Moshe Zadka <moshez@math.huji.ac.il>  Mon Nov 27 07:40:47 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Mon, 27 Nov 2000 09:40:47 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127023033.A7616@thyrsus.com>
Message-ID: <Pine.GSO.4.10.10011270938120.11335-100000@sundial>

On Mon, 27 Nov 2000, Eric S. Raymond wrote:

> I agree.  I wrote a full-featured set library long ago.  All it's
> waiting for is rich comparisons.  Wait...are those in 2.0?

Nope. I'm not even sure if they're going to be in 2.1. DA?
Anyway, can you at least give the URL? I wrote something like that
too (is this another Python ritual), and I'd be happy to try and
integrate them for 2.1.

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Mon Nov 27 09:32:46 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 10:32:46 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <3A222A3E.D53D6D54@lemburg.com>

Fredrik Lundh wrote:
> 
> Charles wrote:
> > On the other hand it is not clear to me why "capwords" and "zfill" did
> > not become methods of string objects.
> 
> fwiw, they were both present in my original unicode
> implementation:
> 
> >>> from unicode import unicode
> >>> a = unicode("hello world")
> >>> a.capwords()
> 'Hello World'
> >>> a.zfill(20)
> '000000000hello world'

.zfill() is implemented for both strings and Unicode, .capwords()
only for Unicode. Both are disabled, though. I talked with
Guido about these methods and we decided to leave those two
methods disabled in the implementation. They just don't provide
much extra generally useful functionality.

s.capwords() can be emulated with ' '.join(s.capitalize().split()),
BTW.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From gvwilson@nevex.com  Mon Nov 27 13:05:23 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:05:23 -0500 (EST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com>

(Note: PEP 0218 is very much a work in progress --- I just wanted to get
some preliminary thoughts down so that my conscience wouldn't nag me quite
so much... :-)

> Well, another proof of the bugs in the PEP process -- my remarks were lost,
> so I'll send them here.

Where did you send your comments?

Thanks,
Greg



From moshez@zadka.site.co.il  Mon Nov 27 21:24:42 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 23:24:42 +0200
Subject: [Python-Dev] PEP-0218
In-Reply-To: Message from Greg Wilson <gvwilson@nevex.com>
 of "Mon, 27 Nov 2000 08:05:23 EST." <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com>
References: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com>
Message-ID: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>

> > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > so I'll send them here.
> 
> Where did you send your comments?

To you personally?
-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From gvwilson@nevex.com  Mon Nov 27 13:30:13 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:30:13 -0500 (EST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>

> > > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > > so I'll send them here.
> > Where did you send your comments?
> To you personally?

Yes, I got those late last week.  Today's election day here in Canada, though,
so the weekend was a write-off.  I'll get to them as soon as I can...

Thanks,

Greg

p.s. emphasis on "day" --- as in, one 24 hour period... :-)



From barry@digicool.com  Mon Nov 27 13:46:14 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 08:46:14 -0500
Subject: [Python-Dev] PEP-0218
References: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
 <Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>
Message-ID: <14882.26022.860967.54489@anthem.concentric.net>

>>>>> "GW" == Greg Wilson <gvwilson@nevex.com> writes:

    GW> p.s. emphasis on "day" --- as in, one 24 hour period... :-)

Recount, recount!  But tell ya what.  If you have as much problems as
we had, we'll trade you Florida for a province or territory of your
choice.

-Barry


From barry@digicool.com  Mon Nov 27 14:46:40 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:46:40 -0500
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
 <3A1FE6F2.6A612EF9@lemburg.com>
 <3A1FE7FC.AF14EF85@tismer.com>
 <3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <14882.29648.538582.454004@anthem.concentric.net>

>>>>> "M" == M  <mal@lemburg.com> writes:

    M> I think the overall impression of Python 2.0 being a moving
    M> target is a bit overrated: there aren't really all that many
    M> changes which affect existing code, so a Python 1.5.2
    M> introduction will still remain mostly valid even when the
    M> students use Python 2.0.

    M> It is true, that Python 2.0 offers a lot more features than
    M> 1.5.2 and that in some contexts (e.g. database interfacing on
    M> Windows) these new features have made their way into third
    M> party extensions rather fast (e.g. switching from strings to
    M> Unicode). The visible core language hasn't changed much
    M> though...  and this is goodness.

I agree completely.  What I think freaks people out is all the new
features that cropped up in 2.0.  People think they have to learn all
that new fancy stuff like string methods, list comprehensions, and
extended print.  For those who watch python-dev from the sidelines,
pie-in-the-sky discussions on things like continuations freak people
out even more.  There's too much brain-blowing new things being added
to my nice little language!

But it doesn't have to be that way.  As others have pointed out,
almost all the code that worked for 1.5.2 works for 2.0 and 1.5.2 was
released what? 18 months ago?  That's a lot of stability these days.

I understand that adding new stuff to the language has an impact on
slow-update media like books, and that should not be ignored.  But
people don't have to learn (or be taught) the new features to be just
as productive in Python 2.0 as they were in 1.5.2.  They can learn the
new stuff once they've grasped and are comfortable with the old.

2.0 introduced a lot of new stuff, as much because of social reasons
and pressures as for technical benefits.  On the flip side, we've had
messages in this forum decrying the recent dearth of messages.  I
suspect that's partly a reaction to all the changes in 2.0, letting
the new things settle in.  I can't imagine that 2.1 will have nearly
as many new features.

-Barry


From barry@digicool.com  Mon Nov 27 14:52:19 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:52:19 -0500
Subject: [Python-Dev] A house upon the sand
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
 <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.29987.132557.43415@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

    GE> I think there are still legitimate reasons for using some
    GE> parts of the string module. For example, if you're one of
    GE> those stubborn people who refuse to accept that ",".join(foo)
    GE> is a better way of writing string.join(foo,",").

Actually, an even better way is

COMMA = ','
COMMA.join(foo)

To me, that is substantially easier to read than either of the above
two alternatives.

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    Fred>   There will never be an excuse for that!  .join() should
    Fred> never have been added as a method!

Of course, I completely disagree.  However, I have noticed that I
often define string constants like

SPACE = ' '
EMPTY = ''
NL = '\n'

just so I can write code like

NL.join(headers)
SPACE.join(names)
EMPTY.join(lines)

I doubt it's worth it, but maybe having a standard module called
stringconstants.py with some of the more obvious choices would make
things better?

toowtdi-my-foot-ly y'rs,
-Barry


From guido@python.org  Mon Nov 27 15:21:48 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:21:48 -0500
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 10:32:46 +0100."
 <3A222A3E.D53D6D54@lemburg.com>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>
 <3A222A3E.D53D6D54@lemburg.com>
Message-ID: <200011271521.KAA29437@cj20424-a.reston1.va.home.com>

> .zfill() is implemented for both strings and Unicode, .capwords()
> only for Unicode. Both are disabled, though. I talked with
> Guido about these methods and we decided to leave those two
> methods disabled in the implementation. They just don't provide
> much extra generally useful functionality.

Exactly.  There's a price to pay for adding methods, and I think these
two are below the threshold.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Nov 27 15:31:33 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:31:33 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: Your message of "Mon, 27 Nov 2000 09:40:47 +0200."
 <Pine.GSO.4.10.10011270938120.11335-100000@sundial>
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial>
Message-ID: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>

[Moshe, on rich comparisons]
> Nope. I'm not even sure if they're going to be in 2.1. DA?

Yes, they should go into 2.1.  Moshe, you are listed as co-author of
PEP 207.  What's up?  Also, I thought you were going to co-author PEP
208 (Reworking the Coercion Model).  What's up with that?  Both are
empty!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gvwilson@nevex.com  Mon Nov 27 16:29:58 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 11:29:58 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>

Hi, folks.  Is there a standard or usual way to provide version
information inside a Python module or class?  Special tokens in
the doc string, a method like "getVersion"...?

Thanks,
Greg


From fdrake@acm.org  Mon Nov 27 16:33:04 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 11:33:04 -0500 (EST)
Subject: [Python-Dev] versioning standards?
In-Reply-To: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
References: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
 <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <14882.36032.794185.285527@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Hi, folks.  Is there a standard or usual way to provide version
 > information inside a Python module or class?  Special tokens in
 > the doc string, a method like "getVersion"...?

  Modules sometimes offer a "__version__" attribute, which I've also
seen called "version" or "VERSION" in strange cases (xmllib and (I
think) PIL).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From mal@lemburg.com  Mon Nov 27 16:42:02 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 17:42:02 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <3A228EDA.56050B52@lemburg.com>

Greg Wilson wrote:
> 
> Hi, folks.  Is there a standard or usual way to provide version
> information inside a Python module or class?  Special tokens in
> the doc string, a method like "getVersion"...?

AFAIK, __version__ with a string value is in common usage both
in modules and classes.

BTW, while we're at it: with the growing number of dependencies
between modules, packages and the Python lib version... how about
creating a standard to enable versioning in module/package imports ?

It would have to meet (at least) these requirements:

* import of a specific version

* alias of the unversioned name to the most recent version
  available

Thoughts ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Mon Nov 27 18:11:54 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 13:11:54 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: Your message of "Mon, 27 Nov 2000 17:42:02 +0100."
 <3A228EDA.56050B52@lemburg.com>
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
 <3A228EDA.56050B52@lemburg.com>
Message-ID: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>

> AFAIK, __version__ with a string value is in common usage both
> in modules and classes.

Correct.  This was agreed upon as a standard long ago.  It's probably
not documented anywhere as such.

> BTW, while we're at it: with the growing number of dependencies
> between modules, packages and the Python lib version... how about
> creating a standard to enable versioning in module/package imports ?
> 
> It would have to meet (at least) these requirements:
> 
> * import of a specific version
> 
> * alias of the unversioned name to the most recent version
>   available
> 
> Thoughts ?

This is a major rathole.  I don't know of any other language or system
that has solved this in a satisfactory way.  Most shared library
implementations have had to deal with this, but it's generally a very
painful manual process to ensure compatibility.  I would rather stay
out of trying to solve this for Python in a generic way -- when
specific package develop incompatible versions, it's up to the package
authors to come up with a backwards compatibility strategy.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Nov 27 18:23:35 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 19:23:35 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
 <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <3A22A6A7.B06F603E@lemburg.com>

Guido van Rossum wrote:
> 
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

Probably should be... along with some other __special_name__
which are in use, e.g. __copyright__ or __author__.

I think it would be a good idea to add these attribute names
as virtually "reserved" attribute names.
 
> > BTW, while we're at it: with the growing number of dependencies
> > between modules, packages and the Python lib version... how about
> > creating a standard to enable versioning in module/package imports ?
> >
> > It would have to meet (at least) these requirements:
> >
> > * import of a specific version
> >
> > * alias of the unversioned name to the most recent version
> >   available
> >
> > Thoughts ?
> 
> This is a major rathole.  I don't know of any other language or system
> that has solved this in a satisfactory way.  Most shared library
> implementations have had to deal with this, but it's generally a very
> painful manual process to ensure compatibility.  I would rather stay
> out of trying to solve this for Python in a generic way -- when
> specific package develop incompatible versions, it's up to the package
> authors to come up with a backwards compatibility strategy.

Well, I think it should at least be possible to install
multiple versions of the same packages in some way which
makes it possible for other programs to choose which version
to take.

With packages and the default import mechanism this is possible
using some caching and __path__ trickery -- I think Pmw does
this. The new "from ... import ... as" also provides ways
for this:

from mx import DateTime160 as DateTime

The problem with the latter approach is that pickles will use the
versioned name and thus won't port easily to later versions. But
you're probably right: only the package itself will know if it's
ok to use the newer version or not.

Would still be nice if we had some kind of documented naming
scheme for versioned modules and packages though, so that the
user will at least recognize these as being versioned import
names.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Mon Nov 27 18:50:10 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Mon, 27 Nov 2000 20:50:10 +0200 (IST)
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011272048300.10126-100000@sundial>

On Mon, 27 Nov 2000, Guido van Rossum wrote:

> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1.  Moshe, you are listed as co-author of
> PEP 207.  What's up?  Also, I thought you were going to co-author PEP
> 208 (Reworking the Coercion Model).  What's up with that?  Both are
> empty!

Ooops, the PEP-0207 thing is a mistake -- I meant to change 0208.
No time now, I'll change 0208 back to DA later tonight. (Sorry 
for being so israelo-centric <wink>). The 0208 PEP should get
done before the weekend. (Your weekend, not mine).

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Mon Nov 27 19:31:43 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 20:31:43 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
Message-ID: <3A22B69F.D3C78D5C@lemburg.com>

> Date: 2000-Nov-27 10:12
> By: mwh
> 
> Comment:
> I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...

Hehe, that was indeed a trivial patch. What was that about trees
in a forest...

I still think that the PySequence_Fast_GETITEM() macro should at
least include a fall-back type check which causes some exception in
case the used object was not "fixed" using PySequence_Fast() (which
I only noticed in string_join() just now).

Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
used more often :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From paulp@ActiveState.com  Mon Nov 27 20:14:06 2000
From: paulp@ActiveState.com (Paul Prescod)
Date: Mon, 27 Nov 2000 12:14:06 -0800
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A22C08E.614A1A0@ActiveState.com>

Mark Lutz wrote:
> 

I read several different issues into your message:

 * standards doc
 * speed of change
 * deprecations
 * backwards incompatibilities

Standards doc: we have one but it is out of date. I think that it would
be great if someone has the bandwidth to improve it but I'm not going to
push Guido very hard to do that because there's a lot of more high
priority stuff I'd like to see him work on.

Speed of change: it wasn't really so radical if you think in terms of
how long these suggestions have been "queued". I would not like it if
Python kept up the same pace of language additions but I don't think
that's going to happen.

Deprecation: deprecations are a good thing. They are the only way that
Python will ever drop old features. If Python never drops old features
then it can only ever grow. If it only ever grows then it will only
become more and more difficult to learn. I don't think I have to justify
the importance of learnability...

Backwards Incompatibilities: I feel strongly that we need to be more
conservative about code breakage. Python 2 was not backwards compatible
enough. It should have deprecated the code and used warnings instead.
Here's my suggested solution for the future:

  http://python.sourceforge.net/peps/pep-0005.html

> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

Sometimes we make mistakes. Making mistakes is part of learning. String
functions should have been methods. We had to correct that mistake. What
other mechanism do we have for correcting mistakes other than
deprecation?

 Paul Prescod


From tim.one@home.com  Mon Nov 27 20:30:04 2000
From: tim.one@home.com (Tim Peters)
Date: Mon, 27 Nov 2000 15:30:04 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>

[Barry A. Warsaw]
> ...
> I have noticed that I often define string constants like
>
> SPACE = ' '
> EMPTY = ''
> NL = '\n'
>
> just so I can write code like
>
> NL.join(headers)
> SPACE.join(names)
> EMPTY.join(lines)
> 
> I doubt it's worth it, but maybe having a standard module called
> stringconstants.py with some of the more obvious choices would make
> things better?

-0.  Then I'd expect to see

    from stringconstants import *

at least once, which is at least once too often.  Sick trick:

SPACE, TAB, NL = " \t\n"

sequence-unpacking-is-more-general-than-you-think<wink>-ly y'rs
    - tim



From mwh21@cam.ac.uk  Mon Nov 27 20:30:31 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:30:31 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 20:31:43 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com>
Message-ID: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> > Date: 2000-Nov-27 10:12
> > By: mwh
> > 
> > Comment:
> > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> 
> Hehe, that was indeed a trivial patch. What was that about trees
> in a forest...

The way I found it was perhaps instructive.  I was looking at the
function, and thought "that's a bit complicated" so I rewrote it (My
rewrite also seems to be bit quicker so I'll upload it as soon as make
test has finished[*]).  In the course of rewriting it, I saw the line
my patch touched and went "duh!".

> I still think that the PySequence_Fast_GETITEM() macro should at
> least include a fall-back type check which causes some exception in
> case the used object was not "fixed" using PySequence_Fast() (which
> I only noticed in string_join() just now).

It's hard to see how; you're not going to check each invocation of
PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
that PySequence_Fast should raise an exception on being passed a
string or Unicode object... but probably not.

> Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> used more often :-)

Yes.  But they're pretty new, aren't they?  I find them a bit
unsatisfactory that it's not possible to hoist the type check out of
the inner loop.  Still, it seems my PII's branch predictor nails that
one... (i.e. changing it so that it didn't check inside the loop made
naff-all difference to the running time).

Cheers,
M.

[*] which reminds me: test_unicodedata is failing for me at the
    moment.  Anyone else seeing this?  It looks like a python
    regrtest.py -g is all that's needed...

-- 
7. It is easier to write an incorrect program than understand a
   correct one.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From mwh21@cam.ac.uk  Mon Nov 27 20:38:35 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:38:35 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: Michael Hudson's message of "27 Nov 2000 20:30:31 +0000"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21@cam.ac.uk> writes:

> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > > 
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > 
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).

It's at

    https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102549&group_id=5470

I want a way of submitting patches by email.  Is roundup in a usable
state yet?

Cheers,
M.

-- 
  "The future" has arrived but they forgot to update the docs. 
                                        -- R. David Murray, 9 May 2000



From mal@lemburg.com  Mon Nov 27 21:08:41 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 22:08:41 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A22CD59.BC091CAD@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > >
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> >
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).  In the course of rewriting it, I saw the line
> my patch touched and went "duh!".

Yeah. The bug must have sneaked in there when the function was
updated to the PySequence_Fast_* implementation.

BTW, could you also add a patch for the test_string.py and
test_unicode.py tests ?

> > I still think that the PySequence_Fast_GETITEM() macro should at
> > least include a fall-back type check which causes some exception in
> > case the used object was not "fixed" using PySequence_Fast() (which
> > I only noticed in string_join() just now).
> 
> It's hard to see how; you're not going to check each invocation of
> PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> that PySequence_Fast should raise an exception on being passed a
> string or Unicode object... but probably not.

Since not using PySequence_Fast() to initialize the protocol,
I'd suggest doing a Py_FatalError() with some explanatory
text which gets printed to stderr -- still better than a
segfault at some later point due to some dangling pointers...

> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they? 

Yep. Fredrik added them quite late in the 2.0 release process.

> I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.  Still, it seems my PII's branch predictor nails that
> one... (i.e. changing it so that it didn't check inside the loop made
> naff-all difference to the running time).

I think Fredrik speculated on having the compiler optimizers
taking care of the check... hmm, it would probably also help
to somehow declare PyTypeObject slots "const" -- is this possible
in a struct definition ?
 
> Cheers,
> M.
> 
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

Is that for the CVS version or the release version ?
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fredrik@effbot.org  Mon Nov 27 21:29:25 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:29:25 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <000f01c058b9$1fc03c60$3c6340d5@hagrid>

Michael Hudson wrote:
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).

your patch changes the behaviour of join on non-standard sequences.

if you use join on a long synthetic sequence, your new version may run
out of memory where the old one worked just fine.

not sure that's a good idea.

-0

</F>



From tim.one@home.com  Mon Nov 27 21:18:35 2000
From: tim.one@home.com (Tim Peters)
Date: Mon, 27 Nov 2000 16:18:35 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEIDIBAA.tim.one@home.com>

[Michael Hudson]
> ...
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

All tests pass for me under current CVS, Windows.  More info?



From fredrik@effbot.org  Mon Nov 27 21:35:08 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:35:08 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
Message-ID: <000901c058b9$ec78aee0$3c6340d5@hagrid>

> your patch changes the behaviour of join on non-standard sequences.
> 
> if you use join on a long synthetic sequence, your new version may run
> out of memory where the old one worked just fine.

looking again, this was obviously only true for the unicode version
(which uses PySequence_GetItem), not for the 8-bit string version
(which uses the PySequence_Fast API).

maybe we should change the 8-bit implementation instead?

if not, +1.

</F>



From barry@digicool.com  Mon Nov 27 21:30:06 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:30:06 -0500
Subject: [Python-Dev] A house upon the sand
References: <14882.29987.132557.43415@anthem.concentric.net>
 <LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>
Message-ID: <14882.53854.350972.738517@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

    TP> at least once, which is at least once too often.  Sick trick:

    TP> SPACE, TAB, NL = " \t\n"

Oh, that is perversely delicious!  I love it.

super-cali-fragi-listic-expi-ali-chomp-chomp-ly y'rs,
-Barry


From fredrik@effbot.org  Mon Nov 27 21:41:17 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:41:17 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <002701c058ba$c99ced40$3c6340d5@hagrid>

Michael Hudson wrote:
> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they?  I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.

if you can figure out a way to do that with C macros,
I'd love to see it...

on the other hand, PySequence_Fast guarantees that the
returned object is either a list or a tuple, so you can easily
move the test outside the loop yourself:

    if (PyTuple_Check(seq))
        for (...)
            item = PyTuple_GET_ITEM(seq, i)
            ...
    else
        for (...)
            item = PyList_GET_ITEM(seq, i)
            ...

> Still, it seems my PII's branch predictor nails that one... (i.e. changing
> it so that it didn't check inside the loop made naff-all difference to the
> running time).

yeah, the type isn't likely to change inside the loop...

</F>



From barry@digicool.com  Mon Nov 27 21:32:05 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:32:05 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
 <3A22B69F.D3C78D5C@lemburg.com>
 <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
 <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <14882.53973.591033.946156@anthem.concentric.net>

>>>>> "MH" == Michael Hudson <mwh21@cam.ac.uk> writes:

    MH> I want a way of submitting patches by email.

I want a way of submitting bugs by email too.
-Barry


From greg@cosc.canterbury.ac.nz  Mon Nov 27 23:32:55 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 12:32:55 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>

Anyone with an interest in the string functions vs.
string methods debate should read this article, which
was referenced in comp.lang.python recently:

    How Non-Member Functions Improve Encapsulation
    by Scott Meyers
    http://www.cuj.com/archive/1802/feature.html

Mr. Meyers presents some very well-reasoned arguments
against the everything-should-be-a-method mentality.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From mal@lemburg.com  Mon Nov 27 23:45:44 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 00:45:44 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <3A22F228.810F7B29@lemburg.com>

Greg Ewing wrote:
> 
> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Note that the motivation for turning to string methods was
that of migrating from strings to Unicode. Adding Unicode
support to the strop C module would have caused very complicated
code -- methods helped by enabling polymorphic code which is
one of the great advantages of writing software for an interface
rather than an implementation.

Note that functions can make very good use of methods and
thus implement polymorphic functionality -- this is not
about methods vs. functions it's about methods to enable
polymorphic functions.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Mon Nov 27 23:58:52 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 18:58:52 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 12:32:55 +1300."
 <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>

> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Just skimmed it -- seems to be a good argument.  (Also for why
capwords and zfill aren't worth adding as methods. :-)

It would be easy to propose join() as a built-in, and this looks
attractive, if it weren't for the existence of os.path.join().  Some
people probably write ``from os.path import join'' and once join() is
a built-in function, this may be confusing for readers who miss that a
different join is imported.  (I'm not saying that it breaks existing
code -- just that it's confusing to have two joins.)  But maybe this
is a moot argument since string.join() is already a function.

I wonder what Java enthusiasts would say after reading Meyers'
article: there *are* no non-member non-friend functions in Java.  The
best approximation is static methods, but they still live in a
class...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry@digicool.com  Tue Nov 28 00:10:56 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:10:56 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
 <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.63504.970688.304915@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

    GE> Anyone with an interest in the string functions vs.
    GE> string methods debate should read this article, which
    GE> was referenced in comp.lang.python recently:

    |     How Non-Member Functions Improve Encapsulation
    |     by Scott Meyers
    |     http://www.cuj.com/archive/1802/feature.html

    GE> Mr. Meyers presents some very well-reasoned arguments
    GE> against the everything-should-be-a-method mentality.

Remember that Meyers is talking about C++ here, where there's a
distinction in the `friendliness' of a function.  He himself says that
the choice is between non-member non-friend functions and member
functions (i.e. methods).  Friend functions have the same dependency
on implementation changes as methods.

In Python, there is no such friend distinction, although you can come
close to faking it with private names.  So the argument based on
degrees of encapsulation doesn't hold water in Python.

in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,
-Barry


From guido@python.org  Tue Nov 28 00:07:38 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:07:38 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 00:45:44 +0100."
 <3A22F228.810F7B29@lemburg.com>
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
 <3A22F228.810F7B29@lemburg.com>
Message-ID: <200011280007.TAA30005@cj20424-a.reston1.va.home.com>

> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

Of course.  Meyers starts by saying that if it needs implementation
details it must be a method.  This is true (if only for efficiency
reasons) for most string methods.

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

Meyers also says that if it needs to be virtual it needs to be a
method.  Polymorphism is roughly equivalent to virtual in this
context, and this alone justifies the move to methods.

But join() is special: it is polymorphic in two arguments, and making
it a method of the separator argument doesn't help.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@effbot.org  Tue Nov 28 00:24:42 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 01:24:42 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com>
Message-ID: <042401c058d1$9b61f030$3c6340d5@hagrid>

mal wrote:
> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

as Alex pointed out on comp.lang.python, the first if-clause
in Meyers algorithm is:

    if (f needs to be virtual)
       make f a member function of C;

which probably covers the join case pretty well...

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

a bit further down, the algorithm goes:

    else if (f can be implemented via C's
             public interface)
       make f a non-member function;

which covers all remaining non-trivial functions in the string
module...

(okay, you could get around this by renaming the method to
something a bit more annoying; let's say the "public interface"
contains a virtual method called "use_myself_to_join_sequence",
in which "join" could be a non-member convenience function
using nothing but the public interface.  but that's just silly...)

:::

fwiw, I still think that Meyers article cannot be fully applied
to Python, since he's ignoring the case:

    else if (f can be implemented via C's
             public interface, but may have
             to be overridden by a subclass
             to C)
       make f a member function;

this isn't a problem in a language that supports function over-
loading (see the interfaces/packages section in meyers article),
but as we all know, Python doesn't...

</F>



From barry@digicool.com  Tue Nov 28 00:17:32 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:17:32 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
 <3A22F228.810F7B29@lemburg.com>
 <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <14882.63900.127012.804027@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> But join() is special: it is polymorphic in two arguments,
    GvR> and making it a method of the separator argument doesn't
    GvR> help.

What about calling the built-in strjoin().  That avoids the conflict
with os.path.join() and strengthens the connection with the intended
behavior -- that the return value is a string and the elements are
str()-ified.

or-use-sjoin()-ly y'rs,
-Barry


From guido@python.org  Tue Nov 28 00:12:01 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:12:01 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 19:10:56 EST."
 <14882.63504.970688.304915@anthem.concentric.net>
References: <14882.29987.132557.43415@anthem.concentric.net> <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
 <14882.63504.970688.304915@anthem.concentric.net>
Message-ID: <200011280012.TAA30062@cj20424-a.reston1.va.home.com>

> Remember that Meyers is talking about C++ here, where there's a
> distinction in the `friendliness' of a function.  He himself says that
> the choice is between non-member non-friend functions and member
> functions (i.e. methods).  Friend functions have the same dependency
> on implementation changes as methods.
> 
> In Python, there is no such friend distinction, although you can come
> close to faking it with private names.  So the argument based on
> degrees of encapsulation doesn't hold water in Python.
> 
> in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,

I disagree.  While non-methods *can* look inside an implementation,
it's still implied that they break encapsulation if they do.  (This
may explain why you like to use private names -- they make the
encapsulation explicit.  For me, it exists even when it is implicit.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@digicool.com  Tue Nov 28 00:26:25 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:26:25 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
 <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
 <14882.63504.970688.304915@anthem.concentric.net>
 <200011280012.TAA30062@cj20424-a.reston1.va.home.com>
Message-ID: <14882.64433.691041.119209@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> I disagree.  While non-methods *can* look inside an
    GvR> implementation, it's still implied that they break
    GvR> encapsulation if they do.  (This may explain why you like to
    GvR> use private names -- they make the encapsulation explicit.
    GvR> For me, it exists even when it is implicit.)

True, but it still means that changing the implementation has an
unknown effect on existing code, because there /could/ be a lot of
breakage.  That can be (and has been! :) written off as the fault of
the 3rd party author, but still, it isn't enforcible like it is in
C++, where, if it ain't a friend, it's not going to break.  In C++ you
can count the number of functions that will break to determine the
degree of encapsulation, but you can't in Python.

Witness the recent change to cgi.FieldStorage.  We removed the
self.lines attribute because it was a memory bloat, and we /thought/
that it wouldn't affect existing code, but there was no way to really
know for sure because the intent of the cgi module author wasn't at
all clear.  Was self.lines intended to be public but just accidentally
undocumented?  Was it a debugging tool?

As an aside, this argument's not really true with built-in types such
as strings because the internal implementation isn't publically
available to the Python programmer.

-Barry


From greg@cosc.canterbury.ac.nz  Tue Nov 28 00:24:07 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:24:07 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>
Message-ID: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz>

Guido said:

> It would be easy to propose join() as a built-in, and this looks
> attractive, if it weren't for the existence of os.path.join().

and

> join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

How about an operator, then?

We already have % rather than a string.format or some such.
Presumably this is because Guido thought it was such a handy
thing that it should be instantly available at our fingertips.
I think a similar argument could be made for string.join, and
also its inverse string.split.

So, how about:

   x & s   == string.join(x, s)

   s1 | s2 == string.split(s1, s2)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Tue Nov 28 00:31:35 2000
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:31:35 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <042401c058d1$9b61f030$3c6340d5@hagrid>
Message-ID: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>

Fredrik Lundh <fredrik@effbot.org>:

> he's ignoring the case:
>
>    else if (f can be implemented via C's
>             public interface, but may have
>             to be overridden by a subclass
>             to C)

Isn't that the same as "f needs to be virtual"?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fredrik@effbot.org  Tue Nov 28 01:11:48 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 02:11:48 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>
Message-ID: <044a01c058d8$3144b320$3c6340d5@hagrid>

Greg Ewing wrote:
> > he's ignoring the case:
> >
> >    else if (f can be implemented via C's
> >             public interface, but may have
> >             to be overridden by a subclass
> >             to C)
> 
> Isn't that the same as "f needs to be virtual"?

in Python, sure.

in C++, it depends.  thanks to function overloading, you can have
polymorphism without having to make everything virtual...

</F>



From guido@python.org  Tue Nov 28 01:22:07 2000
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 20:22:07 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 13:24:07 +1300."
 <200011280024.NAA03278@s454.cosc.canterbury.ac.nz>
References: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz>
Message-ID: <200011280122.UAA30268@cj20424-a.reston1.va.home.com>

> How about an operator, then?
> 
> We already have % rather than a string.format or some such.
> Presumably this is because Guido thought it was such a handy
> thing that it should be instantly available at our fingertips.
> I think a similar argument could be made for string.join, and
> also its inverse string.split.
> 
> So, how about:
> 
>    x & s   == string.join(x, s)
> 
>    s1 | s2 == string.split(s1, s2)

I don't see the mnemonic.  (For %, the mnemonic is clear: %s, %d etc.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From esr@thyrsus.com  Tue Nov 28 01:25:03 2000
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 20:25:03 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 10:31:33AM -0500
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial> <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <20001127202503.A1013@thyrsus.com>

Guido van Rossum <guido@python.org>:
> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1. 

Good.  Within a few hours after I see that feature in the beta I'll
have a very rich Set class for the standard library.  It includes all
the standard boolean operations, symmetric difference, powerset, and
useful overloads for most of the standard operators.  As the header
comment says:

# A set-algebra module for Python
#
# The functions work on any sequence type and return lists.
# The set methods can take a set or any sequence type as an argument.

Most of the code is tested already.  All it's waiting on is rich comparisons
so I can handle partial-ordering of sets properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

When only cops have guns, it's called a "police state".
        -- Claire Wolfe, "101 Things To Do Until The Revolution" 


From jeremy@alum.mit.edu  Tue Nov 28 02:10:17 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Mon, 27 Nov 2000 21:10:17 -0500 (EST)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <000f01c058b9$1fc03c60$3c6340d5@hagrid>
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
 <3A22B69F.D3C78D5C@lemburg.com>
 <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
 <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
 <000f01c058b9$1fc03c60$3c6340d5@hagrid>
Message-ID: <14883.5129.304362.407763@bitdiddle.concentric.net>

test_unicodedata is failing for me on Linux.  Fredrik, you made a
change on Nov. 3 that changed test/output/test_unicodedata but not
test/test_unicodedata.pu.  Did you verify that the test suite worked
then?

Jeremy


From mwh21@cam.ac.uk  Tue Nov 28 08:52:00 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 28 Nov 2000 08:52:00 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in   <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 22:08:41 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Michael Hudson wrote:
> > 
> > "M.-A. Lemburg" <mal@lemburg.com> writes:
> > 
> > > > Date: 2000-Nov-27 10:12
> > > > By: mwh
> > > >
> > > > Comment:
> > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > >
> > > Hehe, that was indeed a trivial patch. What was that about trees
> > > in a forest...
> > 
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).  In the course of rewriting it, I saw the line
> > my patch touched and went "duh!".
> 
> Yeah. The bug must have sneaked in there when the function was
> updated to the PySequence_Fast_* implementation.
> 
> BTW, could you also add a patch for the test_string.py and
> test_unicode.py tests ?

Here's an effort, but it's a memory scribbling bug.  If I change
test_unicode.py thus:

Index: test_unicode.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
retrieving revision 1.22
diff -c -r1.22 test_unicode.py
*** test_unicode.py     2000/10/23 17:22:08     1.22
--- test_unicode.py     2000/11/28 08:49:11
***************
*** 70,76 ****
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = 'wxyz'
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
--- 70,76 ----
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
***************
*** 78,83 ****
--- 78,87 ----
  test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
  test('join', u' ', u'w x y z', Sequence())
  test('join', u' ', TypeError, 7)
+ test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
+ test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
+ test('join', ' ', u'w x y z', Sequence())
+ test('join', ' ', TypeError, 7)
  
  class BadSeq(Sequence):
      def __init__(self): self.seq = [7, u'hello', 123L]

and back out the fix for the join bug, this happens:

...
...
Testing Unicode formatting strings... done.
Testing builtin codecs...
Traceback (most recent call last):
  File "test_unicode.py", line 378, in ?
    assert unicode('hello','utf8') == u'hello'
  File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
 ?
    import codecs,aliases
SystemError: compile.c:185: bad argument to internal function
Segmentation fault

i.e. it crashes miles away from the problem.

I'll reply to the other stuff later - no time now.

Cheers,
M.

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                         and quoted by Aahz Maruch on comp.lang.python



From mal@lemburg.com  Tue Nov 28 09:17:29 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:17:29 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com> <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A237829.EA13903D@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> > Michael Hudson wrote:
> > >
> > > "M.-A. Lemburg" <mal@lemburg.com> writes:
> > >
> > > > > Date: 2000-Nov-27 10:12
> > > > > By: mwh
> > > > >
> > > > > Comment:
> > > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > > >
> > > > Hehe, that was indeed a trivial patch. What was that about trees
> > > > in a forest...
> > >
> > > The way I found it was perhaps instructive.  I was looking at the
> > > function, and thought "that's a bit complicated" so I rewrote it (My
> > > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > > test has finished[*]).  In the course of rewriting it, I saw the line
> > > my patch touched and went "duh!".
> >
> > Yeah. The bug must have sneaked in there when the function was
> > updated to the PySequence_Fast_* implementation.
> >
> > BTW, could you also add a patch for the test_string.py and
> > test_unicode.py tests ?
> 
> Here's an effort, but it's a memory scribbling bug.  If I change
> test_unicode.py thus:
> 
> Index: test_unicode.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
> retrieving revision 1.22
> diff -c -r1.22 test_unicode.py
> *** test_unicode.py     2000/10/23 17:22:08     1.22
> --- test_unicode.py     2000/11/28 08:49:11
> ***************
> *** 70,76 ****
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = 'wxyz'
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> --- 70,76 ----
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> ***************
> *** 78,83 ****
> --- 78,87 ----
>   test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
>   test('join', u' ', u'w x y z', Sequence())
>   test('join', u' ', TypeError, 7)
> + test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
> + test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
> + test('join', ' ', u'w x y z', Sequence())
> + test('join', ' ', TypeError, 7)
> 
>   class BadSeq(Sequence):
>       def __init__(self): self.seq = [7, u'hello', 123L]
> 
> and back out the fix for the join bug, this happens:
> 
> ...
> ...
> Testing Unicode formatting strings... done.
> Testing builtin codecs...
> Traceback (most recent call last):
>   File "test_unicode.py", line 378, in ?
>     assert unicode('hello','utf8') == u'hello'
>   File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
>  ?
>     import codecs,aliases
> SystemError: compile.c:185: bad argument to internal function
> Segmentation fault
> 
> i.e. it crashes miles away from the problem.

The test is only supposed to assure that we don't trip again. It's
not intended to work in some way *before* applying your patch.

I always try to integrate tests for bugs into the test suites
for my mx stuff and AFAICTL this also seems to be the Python
dev style.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Tue Nov 28 09:21:30 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:21:30 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
 <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <3A23791A.9C5ACD60@lemburg.com>

Guido van Rossum wrote:
> 
> But join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

join() is special indeed, but what about the semantics we talked
about last year (?)...

join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]

This should fit all uses of join() (accept maybe os.path.join).

How about naming the beast concat() with sep defaulting to '' to
avoid the problems with os.path.join() ?!


-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fredrik@effbot.org  Tue Nov 28 09:54:31 2000
From: fredrik@effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 10:54:31 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com><3A22B69F.D3C78D5C@lemburg.com><m3bsv16tug.fsf@atrus.jesus.cam.ac.uk><m38zq56th0.fsf@atrus.jesus.cam.ac.uk><000f01c058b9$1fc03c60$3c6340d5@hagrid> <14883.5129.304362.407763@bitdiddle.concentric.net>
Message-ID: <00a401c05923$73ca2950$3c6340d5@hagrid>

Jeremy wrote:

> test_unicodedata is failing for me on Linux.  Fredrik, you made a
> change on Nov. 3 that changed test/output/test_unicodedata but not
> test/test_unicodedata.pu.  Did you verify that the test suite worked
> then?

I added 38,000 missing characters to the unicode database.
that didn't change the test, only the test output...

the november 3 checkin included the following files:

    Lib/test/output/test_unicodedata
    Modules/unicodedata_db.h
    Objects/unicodetype_db.h
    Tools/unicode/makeunicodedata.py

:::

hmm.  looks like the makefile dependencies haven't been
updated (is this done by hand on Unix???)

in Modules/Makefile.pre.in

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h

should be

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h unicodedata_db.h

and in Objects/Makefile.in

    unicodectype.o: unicodectype.c

should be:

    unicodectype.o: unicodectype.c unicodectype_db.h
    
</F>



From mwh21@cam.ac.uk  Tue Nov 28 10:19:14 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:19:14 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
In-Reply-To: <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>

On Mon, 27 Nov 2000, M.-A. Lemburg wrote:

> Michael Hudson wrote:
> 
> > > I still think that the PySequence_Fast_GETITEM() macro should at
> > > least include a fall-back type check which causes some exception in
> > > case the used object was not "fixed" using PySequence_Fast() (which
> > > I only noticed in string_join() just now).
> > 
> > It's hard to see how; you're not going to check each invocation of
> > PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> > that PySequence_Fast should raise an exception on being passed a
> > string or Unicode object... but probably not.
> 
> Since not using PySequence_Fast() to initialize the protocol,
> I'd suggest doing a Py_FatalError() with some explanatory
> text which gets printed to stderr -- still better than a
> segfault at some later point due to some dangling pointers...

So you'd want PySequence_Fast_GETITEM to look like

#define PySequence_Fast_GETITEM(s,i) \
    (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
          (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
                  Py_FatalError("muffin!") ))

?  That'd probably be fair - you'd want to check any performance hit, but
I wouldn't be surprised if it was neglible.

> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they? 
> 
> Yep. Fredrik added them quite late in the 2.0 release process.
> 
> > I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.  Still, it seems my PII's branch predictor nails that
> > one... (i.e. changing it so that it didn't check inside the loop made
> > naff-all difference to the running time).
> 
> I think Fredrik speculated on having the compiler optimizers
> taking care of the check... 

gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like

.stabn 68,0,825,.LM312-string_join
.LM312:
        movl PyList_Type@GOT(%ebx),%eax
        cmpl %eax,4(%edi)
        jne .L875
        movl 12(%edi),%eax
        jmp .L898
        .p2align 4,,7
.L875:
        leal 12(%edi),%eax
.L898:
        movl -32(%ebp),%edx
        movl (%eax,%edx,4),%esi

... but I'm no expert in assembly reading.

> hmm, it would probably also help
> to somehow declare PyTypeObject slots "const" -- is this possible
> in a struct definition ?

I have no idea!

Cheers,
M.



From mwh21@cam.ac.uk  Tue Nov 28 10:35:57 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:35:57 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <002701c058ba$c99ced40$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281025490.11335-100000@localhost.localdomain>

On Mon, 27 Nov 2000, Fredrik Lundh wrote:

> Michael Hudson wrote:
> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they?  I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.
> 
> if you can figure out a way to do that with C macros,
> I'd love to see it...

No.  You can consider my above comment as a round-about way of saying "I
hate C" :-)

> on the other hand, PySequence_Fast guarantees that the
> returned object is either a list or a tuple, so you can easily
> move the test outside the loop yourself:
> 
>     if (PyTuple_Check(seq))
>         for (...)
>             item = PyTuple_GET_ITEM(seq, i)
>             ...
>     else
>         for (...)
>             item = PyList_GET_ITEM(seq, i)
>             ...

But doing that would slightly eliminate the point of the PySequenceFast*
API wouldn't it?  Rhetorical question; it doesn't really seem to matter.

Slightly less rhetorical question: string_join currently just assumes a
list of strings.  But it is conceivable that one day it might be changed
to call __str__ on instances it finds, and then you'd have the issue of
__str__ methods changing the length of the list you're iterating over
(this might more plausibly apply in other situations where you might want
to use PySequence_Fast but I haven't gone looking for them). So would it
be feasible to change PySequence_Fast to smash the type of a list it's
given with the "immutable_list_type" from listobject.c and then have a
PySequence_Fast_End function that would decref the sequence and un-smash
the type as appropriate?

Cheers,
M.



From mwh21@cam.ac.uk  Tue Nov 28 10:42:10 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:42:10 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <00a401c05923$73ca2950$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281036320.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, Fredrik Lundh wrote:

> Jeremy wrote:
> 
> > test_unicodedata is failing for me on Linux.  Fredrik, you made a
> > change on Nov. 3 that changed test/output/test_unicodedata but not
> > test/test_unicodedata.pu.  Did you verify that the test suite worked
> > then?
> 
> I added 38,000 missing characters to the unicode database.
> that didn't change the test, only the test output...
> 
> the november 3 checkin included the following files:
> 
>     Lib/test/output/test_unicodedata
>     Modules/unicodedata_db.h
>     Objects/unicodetype_db.h
>     Tools/unicode/makeunicodedata.py
> 
> :::
> 
> hmm.  looks like the makefile dependencies haven't been
> updated (is this done by hand on Unix???)

I think so.  I've done a complete rebuild and the test now passes (I was
sure I'd done one of those more recently than Nov 3...).  So, false alarm,
sorry.

Cheers,
M.



From mwh21@cam.ac.uk  Tue Nov 28 10:43:49 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:43:49 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in    <stringobject>.join(<unicodestring>)
In-Reply-To: <3A237829.EA13903D@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, M.-A. Lemburg wrote:

> The test is only supposed to assure that we don't trip again. It's
> not intended to work in some way *before* applying your patch.

Yes.  I don't know what I was thinking earlier.  Too many beers last
night, perhaps.

> I always try to integrate tests for bugs into the test suites
> for my mx stuff and AFAICTL this also seems to be the Python
> dev style.

Do you want to check it in then?  I can't.

Cheers,
M.



From mal@lemburg.com  Tue Nov 28 11:07:22 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:07:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>
Message-ID: <3A2391EA.133A2CEF@lemburg.com>

Michael Hudson wrote:
> 
> On Tue, 28 Nov 2000, M.-A. Lemburg wrote:
> 
> > The test is only supposed to assure that we don't trip again. It's
> > not intended to work in some way *before* applying your patch.
> 
> Yes.  I don't know what I was thinking earlier.  Too many beers last
> night, perhaps.
> 
> > I always try to integrate tests for bugs into the test suites
> > for my mx stuff and AFAICTL this also seems to be the Python
> > dev style.
> 
> Do you want to check it in then?  I can't.

Best is to put it up on SF as patch. Someone will then pick it up and
check it in sooner or later. In any case that'll prevent it
getting lost.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mal@lemburg.com  Tue Nov 28 11:12:56 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:12:56 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <3A239338.800E84E8@lemburg.com>

Michael Hudson wrote:
> 
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair - you'd want to check any performance hit, but
> I wouldn't be surprised if it was neglible.

Yes.
 
> [Fredrik's PySequence_Fast_* APIs]
> >
> > I think Fredrik speculated on having the compiler optimizers
> > taking care of the check...
> 
> gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like
> 
> .stabn 68,0,825,.LM312-string_join
> .LM312:
>         movl PyList_Type@GOT(%ebx),%eax
>         cmpl %eax,4(%edi)
>         jne .L875
>         movl 12(%edi),%eax
>         jmp .L898
>         .p2align 4,,7
> .L875:
>         leal 12(%edi),%eax
> .L898:
>         movl -32(%ebp),%edx
>         movl (%eax,%edx,4),%esi
> 
> ... but I'm no expert in assembly reading.
> 
> > hmm, it would probably also help
> > to somehow declare PyTypeObject slots "const" -- is this possible
> > in a struct definition ?
> 
> I have no idea!

Me neither -- it could help the compiler when moving type
checks out of inner loops: the type slot of PyObject normally
doesn't change (except on Windows where you have to patch it
in order make VC++ happy).

Anyone got a working idea in this direction ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fredrik@pythonware.com  Tue Nov 28 11:32:22 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2000 12:32:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <012c01c0592e$e0b463e0$0900a8c0@SPIFF>

michael wrote:
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair

sorry, but that's just silly -- the function is called GET_ITEM,
not GetItem.

it's no less safe than PyList_GET_ITEM or PyTuple_GET_ITEM or
any other "trade speed for safety" macro.

</F>



From guido@python.org  Tue Nov 28 12:13:57 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 07:13:57 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 10:21:30 +0100."
 <3A23791A.9C5ACD60@lemburg.com>
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
 <3A23791A.9C5ACD60@lemburg.com>
Message-ID: <200011281213.HAA02449@cj20424-a.reston1.va.home.com>

> join() is special indeed, but what about the semantics we talked
> about last year (?)...
> 
> join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> 
> This should fit all uses of join() (accept maybe os.path.join).

This is much more general than the current definition --
e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
the definition this much.

> How about naming the beast concat() with sep defaulting to '' to
> avoid the problems with os.path.join() ?!

Hm... if we can stick to the string semantics this would be okay.  But
we'd lose the symmetry of split/join.  Note that string.join has a
default separator of ' ' to roughly match the default behavoir of
split.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Nov 28 15:10:51 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 10:10:51 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: Your message of "26 Nov 2000 23:22:15 GMT."
 <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>
 <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>

----------
My feelings about this topic have been expressed adequately by others.

I'm not sure what to say about the code that worked in 1.5.2 but broke
in 2.0 -- if these were book examples, I suspect that the book was
probably using undocumented (if commonly seen) features, line
multi-argument append() or connect().

As someone else said:  Change happens.  Get over it. :-)

> "Tim Peters" <tim.one@home.com> writes:
> 
> > On the deprecation of the string module:  where did this idea come from?
> > I've never seen anything saying that the string module is deprecated.

Michael Hudson:

> I thought this, and went looking.  I found on
> http://www.python.org/1.6/, about four fifths of the way down:
> 
> Changed Modules
> 
>     string - most of this module is deprecated now that strings have
>     methods. This no longer uses the built-in strop module, but takes
>     advantage of the new string methods to provide transparent support
>     for both Unicode and ordinary strings.
> 
> I hope (and believe) this is Wrong.

Because of its importance, the deprecation time of the string module
will be longer than that of most deprecated modules.  I expect it
won't be removed until Python 3000.

> http://www.python.org/2.0/new-python.html says:
> 
>     The old string module is still around for backwards compatibility,
>     but it mostly acts as a front-end to the new string methods.
> 
> which is IMHO better.

Yes.

And Greg Ewing:

> I think there are still legitimate reasons for using some
> parts of the string module. For example, if you're one of
> those stubborn people who refuse to accept that ",".join(foo)
> is a better way of writing string.join(foo,",").

This has been discussed -- jut note that continuing to use the string
module *is* frowned upon, and such stubborn code will get its just
desserts when Py3K arrives.

I suggest adding the following to the string module's documentation
(rather than marking it as explicitly deprecated):

    This module exists for backwards compatibility only.  It will
    eventually be deprecated and its use should be avoided in new code.

I also suggest that someone go through the standard library and get
rid of all uses of the string module.  (What to do with string.letters
c.s.?)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Tue Nov 28 15:22:18 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 16:22:18 +0100
Subject: [Python-Dev] A house upon the sand
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>
 <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <3A23CDAA.F14E0011@lemburg.com>

Guido van Rossum wrote:
> 
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
> 
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new code.
> 
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

We will need some place to put constants and other non-method
based string related functions (much like the Unicode 
database module which serves a similar purpose)... 
the ideal complement would be a stringdata module which provides
the same interfaces as the unicodedata module if reasonably 
possible (we'd get into encoding problems again...) !

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From tismer@tismer.com  Tue Nov 28 14:33:38 2000
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 28 Nov 2000 16:33:38 +0200
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
 <3A23791A.9C5ACD60@lemburg.com> <200011281213.HAA02449@cj20424-a.reston1.va.home.com>
Message-ID: <3A23C242.C24BE5D9@tismer.com>


Guido van Rossum wrote:
> 
> > join() is special indeed, but what about the semantics we talked
> > about last year (?)...
> >
> > join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> >
> > This should fit all uses of join() (accept maybe os.path.join).
> 
> This is much more general than the current definition --
> e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
> the definition this much.

No, if n is the length of the sequence, the above definition
would calculate 10 and then raise IndexError :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com


From akuchlin@mems-exchange.org  Tue Nov 28 22:17:35 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 17:17:35 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282213.OAA31146@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Tue, Nov 28, 2000 at 02:13:47PM -0800
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
Message-ID: <20001128171735.A21996@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
>229
>is not sufficiently worked out and doesn't seem a priority.  (This can
>still make it into 2.1 if the author does the work.)

I was waiting for further comments on the idea before actually doing
the work.  Anyone see gaping holes in the basic idea?  (I *can* just
charge ahead and work on it anyway...)

--amk


From guido@python.org  Tue Nov 28 23:01:38 2000
From: guido@python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 18:01:38 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: Your message of "Tue, 28 Nov 2000 17:17:35 EST."
 <20001128171735.A21996@kronos.cnri.reston.va.us>
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
 <20001128171735.A21996@kronos.cnri.reston.va.us>
Message-ID: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>

> On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
> >229
> >is not sufficiently worked out and doesn't seem a priority.  (This can
> >still make it into 2.1 if the author does the work.)

Andrew:
> I was waiting for further comments on the idea before actually doing
> the work.  Anyone see gaping holes in the basic idea?  (I *can* just
> charge ahead and work on it anyway...)

I'd say charge ahead, but check where you are every once in a while.
Some things I'm not sure about include:

- Always shared libs.  What about Unixish systems that don't have
  shared libs?  What if you just want something to be hardcoded as
  statically linked, e.g. for security reasons?  (On the other hand
  there are some reasons why defaulting to shared libs is a good
  idea -- e.g. it makes it easier to upgrade a single extension module
  in an installation, e.g. for a critical fix.)

- Doesn't distutils currently always look for an existing installed
  Python installation to get its Makefile info?  How to force it to
  use the info from the build tree?

BTW off-topic: I didn't demote your PEP 222 (Web Library Enhancements)
but I'm pretty neutral on this -- except that it's probably a good
idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
than making the existing cgi.py even more complicated.  (Also, it's
very fragile -- you never know who uses what mis-feature.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Tue Nov 28 23:21:44 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 28 Nov 2000 18:21:44 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
 <20001128171735.A21996@kronos.cnri.reston.va.us>
 <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <14884.15880.264559.125483@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > idea to start from scratch with a new cgi-ish module (cgi2.py?) rather

  Or it could be called "cgilib"...


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From exim-users@exim.org  Tue Nov 28 23:42:27 2000
From: exim-users@exim.org (Jeffrey C. Ollie)
Date: Tue, 28 Nov 2000 17:42:27 -0600
Subject: [Python-Dev] Python embedded in Exim
Message-ID: <20001128174227.A24208@ollie.clive.ia.us>

--aM3YZ0Iwxop3KEKx
Content-Type: multipart/mixed; boundary="FL5UXtIhxfXey3p5"
Content-Disposition: inline


--FL5UXtIhxfXey3p5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Well, I had an epiphany over the US Thanksgiving holiday weekend (or
maybe it was a brain aneurism) so I've spent some late nights and some
time stolen from work and come up with a first pass at embedding
Python in Exim.

I imagine that embedding Python in Exim will be interesting to those
folks writing virus scanners or for VERY tight integration of Mailman
with Exim.  You can also do all of those things that you can do with
the Perl string expansions.

Note: this code is VERY lightly tested. It compiles on my home
hacked-up RedHat Linux system and works on a couple of simple tests
that I've done, but no real e-mails have passed through this code yet.
I'm tossing this out for public comment on the design and
implementation...  It's been a while since I've done a lot of C
coding.

Unfortunately, you must embed Python 2.0.  This is because previous
versions of Python used a hacked copy of PCRE as it's regular
expression engine that conflicts with the copy of PCRE in Exim.

Starting in Python 2.0 the default regular expression is a completely
new one called SRE that supports Unicode.  The PCRE-based regular
expression engine is still included in Python 2.0 but I get around
that by creating a private copy of the Python library and delete the
offending object modules.  You could do that in versions of Python
prior to 2.0 but a lot of the standard library depends on regular
expressions I didn't think that there was much point in trying to make
the embedding work with 1.5.2 or 1.6.

Well, on to the patch itself.  The patch is made against Exim v3.20.
After you've patched the source code, you need to set four variables
in the Local/Makefile:

EXIM_PYTHON=python.o
PYTHON_INC=-I/usr/local/include/python2.0
PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a
PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil

Then build Exim as usual.

There are three runtime directives that control the embedding, all of which
are optional:

python_at_start		boolean - force startup of Python interpreter
python_module_paths	colon separated list of paths to append to sys.path
python_initial_import   colon separated list of modules to import at
                        interpreter initialization time

There are also two new command line switches -ys and -yd that will
force an immediate startup of Python or delay startup, overriding
python_at_start.

Then you use it:

${python{<module>.<function>}{<arg1>}...}

You must specify the module name and the function name.  The named
module will be imported automatically.  And currently you must specify
a function that returns a string (I'll look into adding accessing
attributes, methods of classes, and converting non-strings into
strings).  There can be up to eight arguments.  Each argument will be
expanded by Exim before it's passed to the Python interpreter.

Your python code has access to a module called "exim", which currently
defines two things:

exim.expand_string(s) which calls back to Exim to expand a string and
exim.error            which is an exception object used by expand_string

One simple example:

${python{string.upper}{$local_part}}

Error reporting isn't the best right now, I'm going to look into
formatting exceptions and tracebacks for more meaningful error
messages.

Well, I think that's the gist of it.  I'll work on patches to the
documentation when this thing becomes more stable.  As I said before
I've only lightly tested this... beware!

Comments, criticisms, suggestions, flames welcome...

Jeff


--FL5UXtIhxfXey3p5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="exim-python.patch"
Content-Transfer-Encoding: quoted-printable

Index: exim/OS/Makefile-Base
diff -u exim/OS/Makefile-Base:1.1.1.1 exim/OS/Makefile-Base:1.1.1.1.2.3
--- exim/OS/Makefile-Base:1.1.1.1	Mon Nov 27 08:18:15 2000
+++ exim/OS/Makefile-Base	Tue Nov 28 16:49:25 2000
@@ -192,6 +192,16 @@
 	@chmod a+x ../util/convert4r3
 	@echo ">>> convert4r3 script built in util directory"; echo ""
=20
+libpython.a: $(PYTHON_LIB)
+	if [ -n $(PYTHON_LIB) ] ;\
+	then \
+		cp $(PYTHON_LIB) libpython.a ;\
+		ar d libpython.a pcremodule.o pypcre.o ;\
+		ranlib libpython.a ;\
+	else \
+		ar cq libpython.a ;\
+		ranlib libpython.a ;\
+	fi
=20
 # Targets for final binaries; the main one has a build number which is
 # updated each time. We don't bother with that for the auxiliaries.
@@ -201,11 +211,12 @@
         header.o host.o log.o match.o moan.o os.o parse.o queue.o \
         readconf.o retry.o rewrite.o \
         route.o search.o smtp_in.o smtp_out.o spool_in.o spool_out.o \
-        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE=
RL)
+        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PE=
RL) \
+	$(EXIM_PYTHON)
=20
 exim:   libident/libident.a pcre/libpcre.a lookups/lookups.a auths/auths.a=
 \
         directors/directors.a routers/routers.a transports/transports.a \
-        $(OBJ_EXIM) version.c
+        $(OBJ_EXIM) version.c libpython.a
 	awk '{ print ($$1+1) }' cnumber.h > cnumber.temp
 	/bin/rm -f cnumber.h; mv cnumber.temp cnumber.h
 	$(CC) -c $(CFLAGS) $(INCLUDE) $(IPV6_INCLUDE) $(TLS_INCLUDE) version.c
@@ -215,7 +226,8 @@
 	  routers/routers.a transports/transports.a lookups/lookups.a \
 	  auths/auths.a \
 	  $(LIBS) $(LIBS_EXIM) $(IPV6_LIBS) $(EXTRALIBS) $(EXTRALIBS_EXIM) \
-	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) $(TLS_LIBS)
+	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) libpython.a $(PYTHON=
_EXTRA_LIBS) \
+	  $(TLS_LIBS)
 	$(EXIM_CHMOD)
 	@echo " "
 	@echo ">>> exim binary built"
@@ -316,6 +328,11 @@
=20
 perl.o:          $(HDRS) perl.c
 	$(PERL_CC) $(PERL_CCOPTS) $(CFLAGS) $(INCLUDE) -c perl.c
+
+# Build instructions for python.o for when EXIM_PYTHON is set
+
+python.o:          $(HDRS) python.c
+	$(CC) -c $(CFLAGS) $(INCLUDE) $(PYTHON_INC) -I. python.c
=20
 # Dependencies for the "ordinary" exim modules
=20
Index: exim/scripts/MakeLinks
diff -u exim/scripts/MakeLinks:1.1.1.1 exim/scripts/MakeLinks:1.1.1.1.2.1
--- exim/scripts/MakeLinks:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/scripts/MakeLinks	Mon Nov 27 08:27:46 2000
@@ -189,6 +189,7 @@
 ln -s ../src/moan.c            moan.c
 ln -s ../src/parse.c           parse.c
 ln -s ../src/perl.c            perl.c
+ln -s ../src/python.c          python.c
 ln -s ../src/queue.c           queue.c
 ln -s ../src/readconf.c        readconf.c
 ln -s ../src/retry.c           retry.c
Index: exim/src/EDITME
diff -u exim/src/EDITME:1.1.1.2 exim/src/EDITME:1.1.1.2.2.4
--- exim/src/EDITME:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/EDITME	Tue Nov 28 16:20:19 2000
@@ -260,6 +260,10 @@
 # PERL_CCOPTS=3D
 # PERL_LIBS=3D
=20
+# EXIM_PYTHON=3Dpython.o
+# PYTHON_INC=3D-I/usr/local/include/python2.0
+# PYTHON_LIB=3D/usr/local/lib/python2.0/config/libpython2.0.a
+# PYTHON_EXTRA_LIBS=3D-lpthread -ldl -lutil
=20
 # This parameter sets the maximum length of the header portion of a message
 # that Exim is prepared to process. The default setting is one megabyte. T=
here
Index: exim/src/config.h.defaults
diff -u exim/src/config.h.defaults:1.1.1.1 exim/src/config.h.defaults:1.1.1=
.1.2.1
--- exim/src/config.h.defaults:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/config.h.defaults	Mon Nov 27 08:43:04 2000
@@ -36,6 +36,8 @@
=20
 #define EXIM_PERL
=20
+#define EXIM_PYTHON
+
 #define HAVE_SA_LEN
 #define HEADER_MAXSIZE  (1024*1024)
=20
Index: exim/src/exim.c
diff -u exim/src/exim.c:1.1.1.2 exim/src/exim.c:1.1.1.2.2.4
--- exim/src/exim.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/exim.c	Tue Nov 28 00:46:52 2000
@@ -351,6 +351,9 @@
 #ifdef EXIM_PERL
 int  perl_start_option =3D 0;
 #endif
+#ifdef EXIM_PYTHON
+int  python_start_option =3D 0;
+#endif
 int  recipients_arg =3D argc;
 int  sender_address_domain =3D 0;
 int  test_retry_arg =3D -1;
@@ -583,6 +586,17 @@
 simply recorded for checking and handling afterwards. Do a high-level swit=
ch
 on the second character (the one after '-'), to save some effort. */
=20
+#ifdef EXIM_PYTHON
+opt_python_original_argc =3D argc;
+opt_python_original_argv =3D store_get(sizeof(char *) * (argc + 1));
+opt_python_original_argv[argc] =3D NULL;
+
+for (i =3D 0; i < argc; i++)
+  {
+  opt_python_original_argv[i] =3D string_copy(argv[i]);
+  }
+#endif
+
 for (i =3D 1; i < argc; i++)
   {
   BOOL badarg =3D FALSE;
@@ -1521,6 +1535,17 @@
     break;
     #endif
=20
+    /* -ys: force Python startup; -yd force delayed Python startup */
+
+    #ifdef EXIM_PYTHON
+    case 'y':
+    if (*argrest =3D=3D 's' && argrest[1] =3D=3D 0) python_start_option =
=3D 1;
+    else
+      if (*argrest =3D=3D 'd' && argrest[1] =3D=3D 0) python_start_option =
=3D -1;
+    else badarg =3D TRUE;
+    break;
+    #endif
+
=20
     case 'q':
=20
@@ -1967,6 +1992,27 @@
   opt_perl_started =3D TRUE;
   }
 #endif /* EXIM_PERL */
+
+/* Start up Python interpreter if Python support is configured and
+there is a and the configuration or the command line specifies
+initializing starting. Note that the global variables are actually
+called opt_python_xxx. */
+
+#ifdef EXIM_PYTHON
+if (python_start_option !=3D 0)
+  opt_python_at_start =3D (python_start_option > 0);
+if (opt_python_at_start)
+  {
+  char *errstr;
+  DEBUG(9) debug_printf("Starting Python interpreter\n");
+  errstr =3D init_python();
+  if (errstr !=3D NULL)
+    {
+    fprintf(stderr, "exim: error during Python startup: %s\n", errstr);
+    return EXIT_FAILURE;
+    }
+  }
+#endif /* EXIM_PYTHON */
=20
 /* Log the arguments of the call if the configuration file said so. This is
 a debugging feature for finding out what arguments certain MUAs actually u=
se.
Index: exim/src/expand.c
diff -u exim/src/expand.c:1.1.1.2 exim/src/expand.c:1.1.1.2.2.3
--- exim/src/expand.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/expand.c	Tue Nov 28 00:23:10 2000
@@ -1318,7 +1318,7 @@
=20
 Operators:
    domain                  extracts domain from an address
-   escape                  escapes non-printing characters
+   escape                  escapes non-printing characters=20
    expand                  expands the string one more time
    hash_<n>_<m>            hash the string, making one that is of length n,
                              using m (default 26) characters from hashcodes
@@ -1865,6 +1865,100 @@
     continue;
     }
   #endif /* EXIM_PERL */
+
+  /* If Perl support is configured, handle calling embedded perl subroutin=
es,
+  unless locked out at this time. Syntax is ${perl{sub}} or ${perl{sub}{ar=
g}}
+  or ${perl{sub}{arg1}{arg2}} or up to a maximum of EXIM_PERL_MAX_ARGS
+  arguments (defined below). */
+
+  #ifdef EXIM_PYTHON
+  #define EXIM_PYTHON_MAX_ARGS 8
+
+  if (strcmp(name, "python") =3D=3D 0)
+    {
+    int i =3D 0;
+    char *sub_name;
+    char *sub_arg[EXIM_PYTHON_MAX_ARGS + 1];
+    char *new_yield;
+
+    if (expand_forbid_python)
+      {
+      expand_string_message =3D "Python calls are not permitted";
+      goto EXPAND_FAILED;
+      }
+
+    while (isspace((uschar)*s)) s++;
+    if (*s !=3D '{') goto EXPAND_FAILED_CURLY;
+    sub_name =3D expand_string_internal(s+1, TRUE, &s, FALSE);
+    if (sub_name =3D=3D NULL) goto EXPAND_FAILED;
+    while (isspace((uschar)*s)) s++;
+    if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY;
+
+    while (isspace((uschar)*s)) s++;
+
+    while (*s =3D=3D '{')
+      {
+      if (i =3D=3D EXIM_PYTHON_MAX_ARGS)
+        {
+        expand_string_message =3D
+          string_sprintf("Too many arguments passed to Python subroutine \=
"%s\" "
+            "(max is %d)", sub_name, EXIM_PYTHON_MAX_ARGS);
+        goto EXPAND_FAILED;
+        }
+      sub_arg[i] =3D expand_string_internal(s+1, TRUE, &s, FALSE);
+      if (sub_arg[i++] =3D=3D NULL) goto EXPAND_FAILED;
+      while (isspace((uschar)*s)) s++;
+      if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY;
+      while (isspace((uschar)*s)) s++;
+      }
+
+    if (*s++ !=3D '}') goto EXPAND_FAILED_CURLY;
+    sub_arg[i] =3D 0;
+
+    /* Start the interpreter if necessary */
+
+    if (!opt_python_started)
+      {
+      char *initerror;
+      DEBUG(9) debug_printf("Starting Python interpreter\n");
+      initerror =3D init_python();
+      if (initerror !=3D NULL)
+        {
+        expand_string_message =3D
+          string_sprintf("error during Python startup: %s\n", initerror);
+        goto EXPAND_FAILED;
+        }
+      }
+
+    /* Call the function */
+
+    new_yield =3D call_python_cat(yield, &size, &ptr, &expand_string_messa=
ge,
+      sub_name, sub_arg);
+
+    /* NULL yield indicates failure; if the message pointer has been set to
+    NULL, the yield was undef, indicating a forced failure. Otherwise the
+    message will indicate some kind of Perl error. */
+
+    if (new_yield =3D=3D NULL)
+      {
+      if (expand_string_message =3D=3D NULL)
+        {
+        expand_string_message =3D
+          string_sprintf("Python subroutine \"%s\" returned undef to force=
 "
+            "failure", sub_name);
+        expand_string_forcedfail =3D TRUE;
+        }
+      goto EXPAND_FAILED;
+      }
+
+    /* Yield succeeded. Ensure forcedfail is unset, just in case it got
+    set during a callback from Python. */
+
+    expand_string_forcedfail =3D FALSE;
+    yield =3D new_yield;
+    continue;
+    }
+  #endif /* EXIM_PYTHON */
=20
   /* Handle character translation for "tr" */
=20
Index: exim/src/functions.h
diff -u exim/src/functions.h:1.1.1.2 exim/src/functions.h:1.1.1.2.2.1
--- exim/src/functions.h:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/functions.h	Mon Nov 27 10:13:25 2000
@@ -16,6 +16,12 @@
 extern char *init_perl(char *);
 #endif
=20
+#ifdef EXIM_PYTHON
+extern char *call_python_cat(char *, int *, int *, char **, char *, char *=
*);
+extern void  cleanup_python(void);
+extern char *init_python(void);
+#endif
+
 #ifdef SUPPORT_TLS
 extern BOOL  tls_client_start(int, host_item *, address_item *, char *,
                char *, char *, char *, char *, int);
Index: exim/src/globals.c
diff -u exim/src/globals.c:1.1.1.1 exim/src/globals.c:1.1.1.1.2.2
--- exim/src/globals.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.c	Mon Nov 27 11:10:15 2000
@@ -35,6 +35,15 @@
 BOOL   opt_perl_started       =3D FALSE;
 #endif
=20
+#ifdef EXIM_PYTHON
+BOOL   opt_python_at_start       =3D FALSE;
+char  *opt_python_module_paths   =3D NULL;
+char  *opt_python_initial_import =3D NULL;
+BOOL   opt_python_started        =3D FALSE;
+int    opt_python_original_argc  =3D 0;
+char **opt_python_original_argv  =3D NULL;
+#endif
+
 #ifdef HAVE_AUTH
 BOOL   auth_always_advertise  =3D TRUE;
 char  *auth_hosts             =3D NULL;
@@ -310,6 +319,7 @@
 BOOL   expand_forbid_exists   =3D FALSE;
 BOOL   expand_forbid_lookup   =3D FALSE;
 BOOL   expand_forbid_perl     =3D FALSE;
+BOOL   expand_forbid_python   =3D FALSE;
 int    expand_nlength[EXPAND_MAXN+1];
 int    expand_nmax            =3D -1;
 char  *expand_nstring[EXPAND_MAXN+1];
Index: exim/src/globals.h
diff -u exim/src/globals.h:1.1.1.1 exim/src/globals.h:1.1.1.1.2.2
--- exim/src/globals.h:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.h	Mon Nov 27 11:10:15 2000
@@ -21,6 +21,15 @@
 extern BOOL   opt_perl_started;       /* Set once interpreter started */
 #endif
=20
+#ifdef EXIM_PYTHON
+extern BOOL   opt_python_at_start;       /* start Python at startup */
+extern char  *opt_python_module_paths;   /* list of paths to append to sys=
.path */
+extern char  *opt_python_initial_import; /* list of modules to import at i=
nterpreter initialization time */
+extern BOOL   opt_python_started;        /* set once Python interpreter st=
arted */
+extern int    opt_python_original_argc;
+extern char **opt_python_original_argv;
+#endif
+
 #ifdef HAVE_AUTH
 extern BOOL   auth_always_advertise;  /* If FALSE, advertise only when nee=
ded */
 extern char  *auth_hosts;             /* These must authenticate */
@@ -208,6 +217,7 @@
 extern BOOL   expand_forbid_exists;   /* Lock out exists checking */
 extern BOOL   expand_forbid_lookup;   /* Lock out lookups */
 extern BOOL   expand_forbid_perl;     /* Lock out Perl calls */
+extern BOOL   expand_forbid_python;   /* Lock out Python calls */
 extern int    expand_nlength[];       /* Lengths of numbered strings */
 extern int    expand_nmax;            /* Max numerical value */
 extern char  *expand_nstring[];       /* Numbered strings */
Index: exim/src/python.c
diff -u /dev/null exim/src/python.c:1.1.2.11
--- /dev/null	Tue Nov 28 16:50:15 2000
+++ exim/src/python.c	Tue Nov 28 16:22:03 2000
@@ -0,0 +1,291 @@
+#include "exim.h"
+
+#include "Python.h"
+
+static void init_exim_module(void);
+
+char *init_python(void)
+{
+   if (opt_python_started)
+   {
+      return NULL;
+   }
+
+   Py_SetProgramName(opt_python_original_argv[0]);
+  =20
+   Py_Initialize();
+  =20
+   PySys_SetArgv(opt_python_original_argc, opt_python_original_argv);
+
+   init_exim_module();
+
+   if (opt_python_module_paths)
+   {
+      char *listptr =3D opt_python_module_paths;
+      int separator =3D 0;
+      char buffer[256];
+      PyObject *path;
+      PyObject *sys_module;
+      PyObject *sys_dict;
+      PyObject *sys_path;
+      PyObject *name;
+
+      name =3D PyString_FromString("sys");
+      if (name =3D=3D NULL)
+      {
+	 PyErr_Clear();
+	 return "problem creating string \"sys\"";
+      }
+
+      sys_module =3D PyImport_Import(name);
+      Py_DECREF(name);
+
+      if (sys_module =3D=3D NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to import sys";
+      }
+
+      sys_dict =3D PyModule_GetDict(sys_module);
+      if (sys_dict =3D=3D NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying get dictionary from module sys";
+      }
+
+      sys_path =3D PyDict_GetItemString(sys_dict, "path");
+      if (sys_path =3D=3D NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to get sys.path";
+      }
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)=
))
+      {
+	 path =3D PyString_FromString(buffer);
+	 if (path =3D=3D NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while allocating Python string for \"%=
s\"", buffer);
+	 }
+
+	 if(!PyList_Append(sys_path, path))
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while insering string \"%s\"into sys.p=
ath", buffer);
+	 }
+      }
+   }
+
+   if (opt_python_initial_import)
+   {
+      char *listptr =3D opt_python_initial_import;
+      int separator =3D 0;
+      char buffer[256];
+      PyObject *module;
+      PyObject *name;
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)=
))
+      {
+	 name =3D PyString_FromString(buffer);
+	 if (name =3D=3D NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem creating string \"%s\"", buffer);
+	 }
+
+	 module =3D PyImport_Import(name);
+	 Py_DECREF(name);
+
+	 if (module =3D=3D NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem importing module %s", buffer);
+	 }
+      }
+   }
+
+     =20
+   opt_python_started =3D TRUE;
+   return NULL;
+}
+
+void cleanup_python(void)
+{
+  Py_Finalize();
+}
+
+static PyObject *find_function(char *name, char **errstrp)
+{
+   PyObject *module_name;
+   PyObject *function_name;
+   PyObject *module;
+   PyObject *dictionary;
+   PyObject *result;
+
+   char *ptr;
+
+   ptr =3D strrchr(name, '.');
+  =20
+   if (ptr =3D=3D NULL)
+   {
+      *errstrp =3D string_sprintf("function name must include module: <mod=
ule>.<name>");
+      return NULL;
+   }
+
+   function_name =3D PyString_FromString(ptr + 1);
+
+   if (function_name =3D=3D NULL)
+   {
+      PyErr_Clear();
+      *errstrp =3D string_sprintf("error trying to allocate function name"=
);
+      return NULL;
+   }
+
+   module_name =3D PyString_FromStringAndSize(name, ptr - name);
+
+   if (function_name =3D=3D NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp =3D string_sprintf("error trying to allocate module name");
+      return NULL;
+   }
+
+   module =3D PyImport_Import(module_name);
+
+   Py_DECREF(module_name);
+
+   if (module =3D=3D NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp =3D string_sprintf("error trying to import module while try=
ing to find %s", name);
+      return NULL;
+   }
+
+   dictionary =3D PyModule_GetDict(module);
+
+   result =3D PyDict_GetItem(dictionary, function_name);
+
+   Py_DECREF(function_name);
+
+   if (!PyCallable_Check(result))
+   {
+      PyErr_Clear();
+      *errstrp =3D string_sprintf("%s is not a callable object", name);
+      return NULL;
+   }
+
+   return result;
+}
+
+char *call_python_cat(char *yield, int *sizep, int *ptrp, char **errstrp,
+		      char *name, char **arg)
+{
+   PyObject *function;
+   PyObject *arguments;
+   PyObject *result;
+   int index;
+   int count=3D0;
+   char **p;
+   char *buffer;
+   int length;
+
+   function =3D find_function(name, errstrp);
+
+   if (function =3D=3D NULL)
+      return NULL;
+
+   p =3D arg;
+
+   while(*p)
+   {
+      count++;
+      p++;
+   }
+
+   arguments =3D PyTuple_New(count);
+
+   index =3D 0;
+   while(index < count)
+   {
+
+      PyTuple_SetItem(arguments, index, PyString_FromString(arg[index]));
+      index++;
+   }
+
+   result =3D PyObject_CallObject(function, arguments);
+
+   Py_DECREF(arguments);
+
+   if (result =3D=3D NULL)
+   {
+      PyErr_Clear();
+      *errstrp =3D string_sprintf("error in Python function %s", name);
+      return NULL;
+   }
+
+   if(!PyString_Check(result))
+   {
+      Py_DECREF(result);
+      *errstrp =3D string_sprintf("Python function %s returned non-string"=
, name);
+      return NULL;
+   }
+
+   if(PyString_AsStringAndSize(result, &buffer, &length))
+   {
+      PyErr_Clear();
+      Py_DECREF(result);
+      *errstrp =3D string_sprintf("could not extract the results of Python=
 function %s", name);
+      return NULL;
+   }
+
+   yield =3D string_cat(yield, sizep, ptrp, buffer, length);
+
+   Py_DECREF(result);
+
+   return yield;
+}
+
+static PyObject *exim_error =3D NULL;
+
+static PyObject *exim_expand_string(PyObject *self, PyObject *args)
+{
+   char *i;
+   char *o;
+
+   if(!PyArg_ParseTuple(args, "s", i))
+      return NULL;
+  =20
+   o =3D expand_string(i);
+  =20
+   if (o =3D=3D NULL)
+   {
+      PyErr_SetString(exim_error, expand_string_message);
+      return NULL;
+   }
+ =20
+   return PyString_FromString(o);
+}
+
+static PyMethodDef exim_methods[] =3D {
+   {"expand_string", exim_expand_string, 1},
+   {NULL,            NULL} /* sentinel */
+};
+
+static void init_exim_module(void)
+{
+   PyObject *m;
+   PyObject *d;
+   PyObject *o;
+
+   m =3D PyImport_AddModule("exim");
+   d =3D PyModule_GetDict(m);
+
+   Py_InitModule("exim", exim_methods);
+
+   exim_error =3D PyErr_NewException("exim.error", NULL, NULL);
+   PyDict_SetItemString(d, "error", exim_error);
+}
+
Index: exim/src/readconf.c
diff -u exim/src/readconf.c:1.1.1.1 exim/src/readconf.c:1.1.1.1.2.2
--- exim/src/readconf.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/readconf.c	Mon Nov 27 10:14:17 2000
@@ -160,6 +160,11 @@
   { "perl_at_start",            opt_bool,        &opt_perl_at_start },
   { "perl_startup",             opt_stringptr,   &opt_perl_startup },
 #endif
+#ifdef EXIM_PYTHON
+  { "python_at_start",          opt_bool,        &opt_python_at_start },
+  { "python_module_paths",      opt_stringptr,   &opt_python_module_paths =
},
+  { "python_initial_import",    opt_stringptr,   &opt_python_initial_impor=
t },
+#endif
 #ifdef LOOKUP_PGSQL
   { "pgsql_servers",            opt_stringptr,   &pgsql_servers },
 #endif

--FL5UXtIhxfXey3p5--

--aM3YZ0Iwxop3KEKx
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE6JELjmCSL+rEqeRsRArY9AJ9iw1RL79TcwrKoRxLCFGSvJ/hPqgCglwRc
JfzWMV388u0Hohv9S1xDxcE=
=430v
-----END PGP SIGNATURE-----

--aM3YZ0Iwxop3KEKx--


From gward@mems-exchange.org  Tue Nov 28 23:51:47 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Tue, 28 Nov 2000 18:51:47 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 01:11:54PM -0500
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com> <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <20001128185146.A19116@ludwig.cnri.reston.va.us>

On 27 November 2000, Guido van Rossum said:
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

I think that the "Distributing Python Modules" manual should have a
"Recommended Practices" section to cover things like this.  My personal
convention is that every single source file has a __revision__ variable,
and the "root" module [1] of a module distribution has __version__.

[1] the "root module" is the highest-level __init__.py in a package-ized
    distribution, or the module itself in a single-file distribution.
    Most non-packageized multi-module distributions have a fairly
    obvious place to put __version__.

        Greg


From akuchlin@mems-exchange.org  Wed Nov 29 02:57:48 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 21:57:48 -0500
Subject: [Python-Dev] PEP 229 and 222
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 28, 2000 at 06:01:38PM -0500
References: <200011282213.OAA31146@slayer.i.sourceforge.net> <20001128171735.A21996@kronos.cnri.reston.va.us> <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <20001128215748.A22105@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 06:01:38PM -0500, Guido van Rossum wrote:
>- Always shared libs.  What about Unixish systems that don't have
>  shared libs?  What if you just want something to be hardcoded as
>  statically linked, e.g. for security reasons?  (On the other hand

Beats me.  I'm not even sure if the Distutils offers a way to compile
a static Python binary.  (GPW: well, does it?)

>idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
>than making the existing cgi.py even more complicated.  (Also, it's
>very fragile -- you never know who uses what mis-feature.)

Indeed.  My inclination is to design two new Request and Response
classes; haven't done that yet, though...

--amk



From fdrake@users.sourceforge.net  Wed Nov 29 06:20:31 2000
From: fdrake@users.sourceforge.net (Fred L. Drake)
Date: Tue, 28 Nov 2000 22:20:31 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011290620.WAA09888@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/


From tim.one@home.com  Wed Nov 29 06:23:10 2000
From: tim.one@home.com (Tim Peters)
Date: Wed, 29 Nov 2000 01:23:10 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMDIBAA.tim.one@home.com>

[Guido]
> ...
> Because of its importance, the deprecation time of the string module
> will be longer than that of most deprecated modules.  I expect it
> won't be removed until Python 3000.

I see nothing in the 2.0 docs, code, or "what's new" web pages saying that
it's deprecated.  So I don't think you can even start the clock on this one
before 2.1 (a fuzzy stmt on the web page for the unused 1.6 release doesn't
count ...).

> ...
> This has been discussed -- jut note that continuing to use the string
> module *is* frowned upon, and such stubborn code will get its just
> desserts when Py3K arrives.
>
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
>
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new
>     code.
>
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

They have to be in a module.  They've always been in the string module.
Common sense thus dictates "leave them in the string module" <0.3 wink>.

change-for-the-sake-of-irritation-is-arguably-irritating-ly y'rs  - tim



From bckfnn@worldonline.dk  Wed Nov 29 21:37:38 2000
From: bckfnn@worldonline.dk (Finn Bock)
Date: Wed, 29 Nov 2000 21:37:38 GMT
Subject: [Python-Dev] long formatting
Message-ID: <3a2576da.50607279@smtp.worldonline.dk>

Hi,

While adding support to Jython for formatting of longs in "%[duxXo]", I
inadvertently introduced another difference between Python and Jython:

Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
000000000000000000037777777766
>>>

Jython 2.0alpha1 on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
-00000000000000000000000000012
>>>

The CPython behaviour looks as if the number have been through a C long
before it is formatted. It that intended? Would jython be wrong if it
used the result above?

regards,
finn


From fdrake@acm.org  Wed Nov 29 21:49:53 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 29 Nov 2000 16:49:53 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
References: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012
...
 > The CPython behaviour looks as if the number have been through a C long
 > before it is formatted. It that intended? Would jython be wrong if it
 > used the result above?

  It doesn't look like either of them is doing the right thing!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From tim_one@email.msn.com  Wed Nov 29 22:10:34 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:10:34 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com>

[Moshe Zadka]
> ...
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.
> Almost?
>
> * No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
> * Convertors: if we want list/tuple to have a semblance of efficiency,
>   we'll need to cache the element list as a list when accessed by
>   index.

Like everyone else, I have a Set class too.  Its __getitem__ is

    # so "for x in set:" works
    # see also set.tolist()
    def __getitem__(self, i):
        if i == 0:
            self.keys = self.d.keys()
        return self.keys[i]

Of course I don't *want* set[i] to mean anything at all; this is just a hack
to work with the current iteration protocol (i.e., Set.__getitem__ makes no
sense except in that it must be implemented so that "for x in set" works
today).  The one thing I could never get used to in Aaron's kjbuckets
implementation of sets is that for/in did not work as expected.

> * Two different constructors: set() for building from sequence, Set()
>   for building from elements. Might be confusing.

My Set.__init__:

    def __init__(self, seq=[]):
        self.d = d = {}
        for x in seq:
            d[x] = 1

That is, the constructor takes a sequence.  Same thing in Icon, which added
a set type late in the game.  No problem in practice.

> * Only possible elements of a set are immutables. OTOH, I'm not sure
>   how Greg intends to implement his sets if these sets are allowed
>   to contain mutable elements.

This needs to be addressed, but several approaches are possible.  For
example, my Set class allows for Sets of Sets (& so on), because I needed
them, and Sets are certainly mutable.

Now immutable objects are required because dicts require immutable objects
as keys.  However, a class (like Set!) can fool dicts, by supplying a
__hash__ and a __cmp__ "that work".  Dicts can be compared directly, so
__cmp__ is no problem.  The difficulty comes with the __hash__.  My solution
was to "freeze" a Set the instant __hash__ was invoked:  this allows
mutation up until the point a hash is captured, and disallows it thereafter.
Concretely,

    def __hash__(self):
        if self.frozen:
            hashcode = self.hashcode
        else:
            # The hash code must not depend on the order of the
            # keys.
            self.frozen = 1
            hashcode = 0
            _hash = hash
            for x in self.d.keys():
                hashcode = hashcode ^ _hash(x)
            self.hashcode = hashcode
        return hashcode

and all the mutating methods check self.frozen, raising ValueError if the
target set is currently frozen.  For example,

    # updating union
    def unionu(self, other):
        if self.frozen:
            raise ValueError(_frozenmsg)
        self.d.update(other.d)

Oddly enough, I almost never got a "frozen" error in practice; it seems that
by the time I was ready to build a set of sets, the constituent sets were at
their final values; and the few times I did get a frozen error, it was
actually a logic bug (I was mutating the set in error).  It's hard to guess
whether that's unique to me <0.5 wink>.

Icon has a different approach entirely:  an Icon set acts like a Python dict
*would* act if you inserted the id()s of the keys rather than the keys
themselves; that is, Icon's sets (and dicts) are based on object identity,
not object value.  Since object identity is invariant regardless of whether
the object is mutable, a hash table implementation has no difficulties.

I find identity semantics for sets less useful than value semantics, though.

[Eric Raymond]
> ...
> Within a few hours after I see [rich comparisons] in the beta I'll
> have a very rich Set class for the standard library.  It includes
> all the standard boolean operations, symmetric difference, powerset,
> and useful overloads for most of the standard operators.  As the
> header comment says:
>
> # A set-algebra module for Python
> #
> # The functions work on any sequence type and return lists.
> # The set methods can take a set or any sequence type as an argument.
> ...

I'm afraid this is a good example of why a set type is unlikely to make it
into the std distribution:  whenever a data structure is added to Python,
the agitation for variations is endless.  For example, returning a list
makes your flavor of sets unsuitable (because inefficient) for many
applications (e.g., there's no efficient way to test a list for element
membership).  "Almost all" set implementations I've seen for Python use
dicts for that reason.

Another set of complaints will be due to some people wanting functional
versions of the set operations, while others want mutating versions.  My own
Set class supplies both, because both are really needed in practice; e.g.,
the updating union was shown above; the functional union builds on that:

    # functional union
    def union(self, other):
        answer = self.__copy__()
        answer.unionu(other)
        return answer

(subtlety:  the functional union has no problem with "frozen" sets;
Set.__copy__ always returns a melted set).

Since it's Greg's PEP, it's up to him to make everyone happy <wink> -- but
since there's so much variation possible, I'm not sure Guido will ever bless
any Set class as "the" Set class.

BTW, one lesson to take from SETL:  a vital set operation in practice is a
mutating "pick a 'random' element E from the set, remove it from the set,
and return it".  An enormous number of set algorithms are of the form

    while not S.empty():
        pick some element from S
        deal with it, possibly mutating S

This operation can't be done efficiently in Python code if the set is
represented by a dict (the best you can do is materialize the full list of
keys first, and pick one of those).  That means my Set class often takes
quadratic time for what *should* be linear-time algorithms.

if-the-set-type-is-a-toy-there's-no-need-to-put-it-in-the-
    distribution-but-if-it's-not-a-toy-it-can't-be-written-
    in-python-today-ly y'rs  - tim




From tim_one@email.msn.com  Wed Nov 29 22:32:29 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:32:29 -0500
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENHIBAA.tim_one@email.msn.com>

[Finn Bock]
> While adding support to Jython for formatting of longs in "%[duxXo]", I
> inadvertently introduced another difference between Python and Jython:
>
> Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> 000000000000000000037777777766
> >>>
>
> Jython 2.0alpha1 on java1.3.0 (JIT: null)
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> -00000000000000000000000000012
> >>>
>
> The CPython behaviour looks as if the number have been through a C long
> before it is formatted. It that intended?

Hard to say, but it is what the code does <wink>.  In 1.5.2, "true longs"
raised an error in this context, so the code first looked to see whether the
putative Python long actually fit in a C long first; if so, it did not raise
an error, and ran the value thru C long formatting instead (in
stringobject.c):

				if (PyLong_Check(v) && PyLong_AsLong(v) == -1
				    && PyErr_Occurred()) {
					/* Too big for a C long. */

That test survived in 2.0; the difference is that it no longer raises an
error if the value doesn't fit in a C long.

> Would jython be wrong if it used the result above?

Hard to say!  Since the size of a Python int varies across platforms, not
even CPython will display the same thing for all Python longs across all
platforms.  I'm in favor of changing the test above to

				if (PyLong_Check(v)) {

Then CPython will match JPython here, and CPython will produce the same
result across all platforms when given a Python long.  The downside is that
it will not produce the same result as 2.0 (or earlier).

Guido, you can Pronounce now <wink>.




From guido@python.org  Thu Nov 30 03:14:47 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:14:47 -0500
Subject: [Python-Dev] Mail to guido@python.org lost or delayed
Message-ID: <200011300314.WAA18820@cj20424-a.reston1.va.home.com>

Folks, mail to guido@python.org may have been bouncing or dropped in a
black hole, since approximately 12:40 pm EST today -- due to a mail
server problem at my ISP (@home.com -- fast reliable cable bandwidth
but lousy mail and DNS service).  I found out about this and *thought*
I had fixed it by switching to guido@digicool.com, but in fact there
was still a forward to @home.com in effect.  I just finally fixed that
(around 10:10 pm EST).

You don't need to change the email address you have for me:
guido@python.org is still the preferred address.  However if you sent
me mail between the approximate times above, I probably haven't seen
it yet.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov 30 03:17:44 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:17:44 -0500
Subject: [Python-Dev] Re: long formatting
Message-ID: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>

I just skimmed the pipermail python-dev archives and found one item
to pronounce on: what "%o" should do with negative longs.  I agree
with Tim.  Also for "%x".  Reason: this matches oct() and hex().

>>> oct(-10L)
'-012L'
>>> oct(-10)
'037777777766'
>>> hex(-10L)
'-0xAL'
>>> hex(-10)
'0xfffffff6'
>>> 

Compatibility be damned (in this case).

(One could argue that for plain ints, the same result are desirable;
however when these are used as bitmasks, I as human reader prefer to
see '0xfffffff6' rather than '-0xA'.  Ah the insanity of consistency!)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Nov 30 05:41:03 2000
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 30 Nov 2000 00:41:03 -0500
Subject: [Python-Dev] Re: long formatting
In-Reply-To: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>

Alas, Guido and I have the same ISP (@home), and its mail server has been
dead all day.  Conveniently <snort>, my other ISP (MSN) coincidentally
stopped letting me send email over my cable connection too.

[Guido]
> I just skimmed the pipermail python-dev archives and found one item
> to pronounce on: what "%o" should do with negative longs.  I agree
> with Tim.  Also for "%x".  Reason: this matches oct() and hex().

Good point!

> >>> oct(-10L)
> '-012L'
> >>> oct(-10)
> '037777777766'

Note that this string varies across platforms.

> >>> hex(-10L)
> '-0xAL'
> >>> hex(-10)
> '0xfffffff6'

Ditto.

> >>>
>
> Compatibility be damned (in this case).

This became SF bug 123859, and has been closed.  Current CVS does:

>>> "%#o" % -10L
'-012'
>>> "%#X" % -10L
'-0XA'
>>>

> (One could argue that for plain ints, the same result are
> desirable; however when these are used as bitmasks, I as human
> reader prefer to see '0xfffffff6' rather than '-0xA'.  Ah the
> insanity of consistency!)

I doubt that we're done with this forever:  we're moving in the direction of
erasing user-visible distinctions between ints and longs.  For bitmask
output, we should probably introduce a new format code.

Fred, I see that you complained about Finn's output examples, but didn't say
why (just "they both look incorrect").  Were you hallucinating, or do you
have a legit complaint?




From fdrake@acm.org  Thu Nov 30 06:22:02 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 01:22:02 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
References: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
 <LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>
 <3a2576da.50607279@smtp.worldonline.dk>
 <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
Message-ID: <14885.61962.901996.857623@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012

Fred L. Drake, Jr. writes:
 >   It doesn't look like either of them is doing the right thing!

Tim Peters writes:
 > Fred, I see that you complained about Finn's output examples, but didn't say
 > why (just "they both look incorrect").  Were you hallucinating, or do you
 > have a legit complaint?

[Fred scrutinizes everything very carefully again... blushes when he
 notices that this was the 'o' format code, not 'd', and quietly
 slinks off to walk the dog...]


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From moshez@zadka.site.co.il  Thu Nov 30 14:52:59 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 16:52:59 +0200
Subject: [Python-Dev] PEP-0218
In-Reply-To: Message from "Tim Peters" <tim_one@email.msn.com>
 of "Wed, 29 Nov 2000 17:10:34 EST." <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com>
Message-ID: <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>

> BTW, one lesson to take from SETL:  a vital set operation in practice is a
> mutating "pick a 'random' element E from the set, remove it from the set,
> and return it".  An enormous number of set algorithms are of the form
> 
>     while not S.empty():
>         pick some element from S
>         deal with it, possibly mutating S
> 
> This operation can't be done efficiently in Python code if the set is
> represented by a dict (the best you can do is materialize the full list of
> keys first, and pick one of those).  That means my Set class often takes
> quadratic time for what *should* be linear-time algorithms.

Hmmm...actually, I've been wanting a method .key() for dictionaries 
a long time. So if we give dictionaries this one small method, then
we *can* do this in Python.

-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From fdrake@users.sourceforge.net  Thu Nov 30 07:33:55 2000
From: fdrake@users.sourceforge.net (Fred L. Drake)
Date: Wed, 29 Nov 2000 23:33:55 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011300733.XAA12728@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/

Today's changes are primarily in the xml.dom API documentation:

	http://python.sourceforge.net/devel-docs/lib/markup.html


From mal@lemburg.com  Thu Nov 30 08:30:22 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 09:30:22 +0100
Subject: [Python-Dev] PEP-0218
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>
Message-ID: <3A26101E.7FF2467B@lemburg.com>

Moshe Zadka wrote:
> 
> > BTW, one lesson to take from SETL:  a vital set operation in practice is a
> > mutating "pick a 'random' element E from the set, remove it from the set,
> > and return it".  An enormous number of set algorithms are of the form
> >
> >     while not S.empty():
> >         pick some element from S
> >         deal with it, possibly mutating S
> >
> > This operation can't be done efficiently in Python code if the set is
> > represented by a dict (the best you can do is materialize the full list of
> > keys first, and pick one of those).  That means my Set class often takes
> > quadratic time for what *should* be linear-time algorithms.
> 
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

Shouldn't be hard to do... the C API for this is already in place:
PyDict_Next(). I'd prefer a method .getitem(), though, which then
returns a tuple (key, value).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov 30 08:44:38 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 30 Nov 2000 10:44:38 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <3A26101E.7FF2467B@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011301043050.24575-100000@sundial>

[Me]
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

[MAL]
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

Well, I'm sure the C API allows it. And if we do it, we might as well
do it right: add .key(), .value() and .item().

--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From pf@artcom-gmbh.de  Thu Nov 30 09:25:37 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 10:25:37 +0100 (MET)
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: <3A26101E.7FF2467B@lemburg.com> from "M.-A. Lemburg" at "Nov 30, 2000  9:30:22 am"
Message-ID: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>

M.-A. Lemburg:
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

IMO '.pickitem()' would be a better name, since many people would
think, that 'getitem()' would take some kind of index as parameter.
Nevertheless I think this is a nice idea, though.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)


From mal@lemburg.com  Thu Nov 30 10:55:33 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 11:55:33 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <3A263225.E2FFDADB@lemburg.com>

Peter Funk wrote:
> 
> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Point taken. How about .fetchitem() -- this doesn't imply any
ordering and can't get confused with __getitem__() et al.

BTW, Moshe's .key() and .value() would break the strong binding
between dictionary keys and values -- I don't see much use in them.
Besides, .fetchitem() provides the same implied functionality.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From guido@python.org  Thu Nov 30 11:10:33 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:10:33 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:25:37 +0100."
 <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>

> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Pronouncement:

It is only efficient to get the *first* item, so let's make that
explicit.  The method names will be:

 .firstkey()
 .firstvalue()
 .firstitem()

Moshe will check in a patch.

Thinking aloud:

Would it be useful to also implement popkey(), popvalue(), popitem(),
which would remove the first item and then return the relevant part of
it?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Thu Nov 30 11:30:09 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 12:30:09 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <3A263A41.AFEFFB53@lemburg.com>

Guido van Rossum wrote:
> 
> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> >
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()

-1

>  .firstvalue()

-1

>  .firstitem()

+1

I really think that .firstitem() is both more intuitive and 
efficient -- .firstvalue() and .firstkey() are simply derivatives
of this method or how would you define the return value of
those two methods ?
 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

+1

Same comment as above... .popitem() suffices.
 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From pf@artcom-gmbh.de  Thu Nov 30 11:33:30 2000
From: pf@artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 12:33:30 +0100 (MET)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 30, 2000  6:10:33 am"
Message-ID: <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>

Guido van Rossum:
[...]
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

Since [].pop() takes in optional index as parameter which defaults
to the *LAST* element, it would be better to make the name explicit:
.popfirstitem()

This is however some what long.  DEJAVU: the {}.setdefault() discussion.
May it is better to leave these methods out for the sake of simplicity, 
although they look useful.

Regards, Peter


From guido@python.org  Thu Nov 30 11:52:10 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:52:10 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:30:09 +0100."
 <3A263A41.AFEFFB53@lemburg.com>
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
 <3A263A41.AFEFFB53@lemburg.com>
Message-ID: <200011301152.GAA26000@cj20424-a.reston1.va.home.com>

> > Pronouncement:
> > 
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> > 
> >  .firstkey()
> 
> -1
> 
> >  .firstvalue()
> 
> -1
> 
> >  .firstitem()
> 
> +1
> 
> I really think that .firstitem() is both more intuitive and 
> efficient -- .firstvalue() and .firstkey() are simply derivatives
> of this method or how would you define the return value of
> those two methods ?

Yes, firstvalue() and firstkey() return just the value or just the
key.  I can see that the first value is not very useful (since you
don't have the key, you can't mutate the dict).  firstkey() might
be just the ticket e.g. for set implementations.  OTOH you may be
right that it's best to add the minimal number of new methods.

> > Moshe will check in a patch.
> > 
> > Thinking aloud:
> > 
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> +1
> 
> Same comment as above... .popitem() suffices.

And for minimality reasons I'm at most -0 on it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov 30 12:09:03 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:09:03 -0500
Subject: [Python-Dev] Python track--O'Reilly Open Source Convention
Message-ID: <200011301209.HAA26079@cj20424-a.reston1.va.home.com>

Laura Lewin is looking for chairs for two Python tracks (or a Python
track and a Zope track) at the O'Reilly conference.  Anybody
interested in volunteering?  Some background info:

> We actually have two tracks, and there's room for strong differentiation 
> between the two.  Each track has a room for two days. We'll also want 
> tutorials on the Monday and Tuesday.
> 
> Like O'Reilly's books, we're not really aimed at the novice. We want to 
> give people who already use the technology more information. There should 
> be at least one introductory tutorial, but we don't want to focus 
> exclusively on that. However, if you, as Chair, have a vision for what you 
> want O'Reilly's conference to be, we'd be very happy to work with you to 
> realize that vision.
> 
> This year we're trying to raise the bar on our Python and Zope offerings. 
> We'd like to have a lot of editorial involvement from a program committee, 
> so that potential speakers have a chance to get their talks reviewed and 
> commented-upon by others.
> 
> The program chair would really be the point person for the conference, to 
> drum up papers and help publicize the conference to the Python community. 
> The chair would appoint a committee, receive submissions, select 
> presentations, seek out talks from people who
> didn't submit, and generally put together the material.
> 
> We can handle any combination of program committees and chairs: one person 
> overseeing two committees, two chairs two committees, whatever gets it all 
> done.
> 
> Does this sound like a job you're interested in? It doesn't pay, but it 
> would be great for the Python community, as I'm sure you know.

For more info, please write or call Laura Lewin LLewin@oreilly.com
(631) 262-9278.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Nov 30 12:40:06 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:40:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 04:31:07 PST."
 <200011301231.EAA22228@slayer.i.sourceforge.net>
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
Message-ID: <200011301240.HAA26261@cj20424-a.reston1.va.home.com>

Aargh!  I thought I told you to submit a patch (to SF), not to check
it in.

The code looks okay but we'll have to discuss whether we really want
all three (as opposed to only firstitem()).  I'm also not sure whether
KeyError might be a better exception.

Don't touch it for now, but let's think this over a bit.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh21@cam.ac.uk  Thu Nov 30 13:16:44 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 13:16:44 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Guido van Rossum's message of "Thu, 30 Nov 2000 06:10:33 -0500"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido@python.org> writes:

> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> > 
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()
>  .firstvalue()
>  .firstitem()
> 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

I must admit I can't think of a situation where .firstkey would be
more useful than .popkey; I thought that this discussion was
discussion was inspired by the wish to write code like:

while not set.empty():
    x = set.popfirstkey()
    ...

Maybe I'm jsut being dense, though.

FWIW, I think 

    dict.firstkey()

reads better than

    dict.firstitem()[0]

if the former's what you're actually trying to say.  Why not write

    for k,_ in dict.items():
        ...

? (Overstating the case slightly for effect, but I hope the point is
clear).

Cheers,
M.

-- 
  Arrrrgh, the braindamage!  It's not unlike the massively
  non-brilliant decision to use the period in abbreviations 
  as well as a sentence terminator.  Had these people no 
  imagination at _all_?                 -- Erik Naggum, comp.lang.lisp



From fdrake@acm.org  Thu Nov 30 13:34:07 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:34:07 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43
In-Reply-To: <200011301231.EAA22229@slayer.i.sourceforge.net>
References: <200011301231.EAA22229@slayer.i.sourceforge.net>
Message-ID: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > +   \lineiii{\var{a}.firstitem()}
 > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
 > +           {(2)}

  So what happens if the mapping is empty?  That's needed for all
three.
  You should also update UserDict.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From mal@lemburg.com  Thu Nov 30 13:40:12 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 14:40:12 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A2658BC.7922B8B1@lemburg.com>

Michael Hudson wrote:
> 
> Guido van Rossum <guido@python.org> writes:
> 
> > > M.-A. Lemburg:
> > > > Shouldn't be hard to do... the C API for this is already in place:
> > > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > > returns a tuple (key, value).
> > >
> > > IMO '.pickitem()' would be a better name, since many people would
> > > think, that 'getitem()' would take some kind of index as parameter.
> > > Nevertheless I think this is a nice idea, though.
> >
> > Pronouncement:
> >
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> >
> >  .firstkey()
> >  .firstvalue()
> >  .firstitem()
> >
> > Moshe will check in a patch.
> >
> > Thinking aloud:
> >
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> I must admit I can't think of a situation where .firstkey would be
> more useful than .popkey; I thought that this discussion was
> discussion was inspired by the wish to write code like:
> 
> while not set.empty():
>     x = set.popfirstkey()
>     ...
> 
> Maybe I'm jsut being dense, though.

Just curious: how would you get at the deleted value when
using .popfirstkey() ?

But you have a point: dict.pop() returning the first filled slot
in the dictionary as tuple (key,value) would probably be most
effective. It's also nice and short.

BTW, I don't get the inspiration for the "first" part in those
names... it makes you think that there's an order to dictionary
items when in reality there isn't.

> FWIW, I think
> 
>     dict.firstkey()
> 
> reads better than
> 
>     dict.firstitem()[0]
> 
> if the former's what you're actually trying to say.  Why not write
> 
>     for k,_ in dict.items():
>         ...
> 
> ? (Overstating the case slightly for effect, but I hope the point is
> clear).

No, the point is that you can modify the dictionary within
the loop. You wouldn't see any changes using the second technique.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov 30 13:43:10 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 30 Nov 2000 15:43:10 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>

On Thu, 30 Nov 2000, Fred L. Drake, Jr. wrote:

> 
> Moshe Zadka writes:
>  > +   \lineiii{\var{a}.firstitem()}
>  > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
>  > +           {(2)}
> 
>   So what happens if the mapping is empty?  That's needed for all
> three.
>   You should also update UserDict.
>

Right. Will be done soon.
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From fdrake@acm.org  Thu Nov 30 13:43:48 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:43:48 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
 <Pine.GSO.4.10.10011301542250.4645-100000@sundial>
Message-ID: <14886.22932.50399.879012@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Right. Will be done soon.

  Given Guido's surprise at the checkin, you might want to wait until
the discussions are over and just make one set of checkins, with all
the results.  ;)
  Hmm.  I need to think about adding version annotations to those
tables of operations on built-in types.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From guido@python.org  Thu Nov 30 13:52:07 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 08:52:07 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 14:40:12 +0100."
 <3A2658BC.7922B8B1@lemburg.com>
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>
 <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <200011301352.IAA26550@cj20424-a.reston1.va.home.com>

> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?
> 
> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Nice!

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

Actually there is -- the (arbitrary) order revealed by .keys().

--Guido van Rossum (home page: http://www.python.org/~guido/)


From moshez@zadka.site.co.il  Thu Nov 30 22:01:14 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 01 Dec 2000 00:01:14 +0200
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43
In-Reply-To: Your message of "Thu, 30 Nov 2000 08:43:48 EST."
 <14886.22932.50399.879012@cj42289-a.reston1.va.home.com>
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com> <Pine.GSO.4.10.10011301542250.4645-100000@sundial>  <14886.22932.50399.879012@cj42289-a.reston1.va.home.com>
Message-ID: <20001130220114.56E85A9BB@darjeeling.zadka.site.co.il>

>   Given Guido's surprise at the checkin, you might want to wait until
> the discussions are over and just make one set of checkins, with all
> the results.  ;)

Hence my not doing it right now.

-- 
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!


From mal@lemburg.com  Thu Nov 30 14:00:36 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 15:00:36 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>
 <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>
Message-ID: <3A265D84.9A80267A@lemburg.com>

Guido van Rossum wrote:
> 
> > Just curious: how would you get at the deleted value when
> > using .popfirstkey() ?
> >
> > But you have a point: dict.pop() returning the first filled slot
> > in the dictionary as tuple (key,value) would probably be most
> > effective. It's also nice and short.
> 
> Nice!
>
> > BTW, I don't get the inspiration for the "first" part in those
> > names... it makes you think that there's an order to dictionary
> > items when in reality there isn't.
> 
> Actually there is -- the (arbitrary) order revealed by .keys().

Hmm, but this can change if you modify the dictionary inside the
loop or if you modify the dictionary from some other thread.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From fredrik@pythonware.com  Thu Nov 30 14:02:17 2000
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 30 Nov 2000 15:02:17 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <02d301c05ad6$26851150$0900a8c0@SPIFF>

thanks, mal !

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

+1 on "pop" (short for dist.items().pop())

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

-0 on "first" (on the other hand, are we really
allowed to vote on pronouncements? ;-)

</F>



From mwh21@cam.ac.uk  Thu Nov 30 14:06:58 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 14:06:58 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 14:40:12 +0100"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <m3pujd5zb1.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal@lemburg.com> writes:
> Michael Hudson wrote:
> > I must admit I can't think of a situation where .firstkey would be
> > more useful than .popkey; I thought that this discussion was
> > discussion was inspired by the wish to write code like:
> > 
> > while not set.empty():
> >     x = set.popfirstkey()
> >     ...
> > 
> > Maybe I'm jsut being dense, though.
> 
> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?

You wouldn't.  I was thinking of the dict-as-sets code that was being
flung around earlier.

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Yes.

> > FWIW, I think
> > 
> >     dict.firstkey()
> > 
> > reads better than
> > 
> >     dict.firstitem()[0]
> > 
> > if the former's what you're actually trying to say.  Why not write
> > 
> >     for k,_ in dict.items():
> >         ...
> > 
> > ? (Overstating the case slightly for effect, but I hope the point is
> > clear).
> 
> No, the point is that you can modify the dictionary within
> the loop. You wouldn't see any changes using the second technique.

I was trying to make an analogy between .firstkey/.firstitem and
.keys/.items - the 

    for k,_ in dict.items():
        ...

was meant to be compared to 

    for k in dict.keys():
        ...

Sorry for being obscure.

Cheers,
M.

-- 
  "declare"?  my bogometer indicates that you're really programming
  in some other language and trying  to force Common Lisp into your
  mindset.  this won't work.            -- Erik Naggum, comp.lang.lisp



From guido@python.org  Thu Nov 30 14:10:34 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 09:10:34 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 15:00:36 +0100."
 <3A265D84.9A80267A@lemburg.com>
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>
 <3A265D84.9A80267A@lemburg.com>
Message-ID: <200011301410.JAA26719@cj20424-a.reston1.va.home.com>

> > > BTW, I don't get the inspiration for the "first" part in those
> > > names... it makes you think that there's an order to dictionary
> > > items when in reality there isn't.
> > 
> > Actually there is -- the (arbitrary) order revealed by .keys().
> 
> Hmm, but this can change if you modify the dictionary inside the
> loop or if you modify the dictionary from some other thread.

Let me explain.  When Moshe first proposed a .key() method, I was a
bit confused.  I thought he wanted a way to get to the i-th key (which
can't be done in O(1) time).  Then I thought he wanted a way to get a
random key.  Finally I realized that all he wanted was *a* key, and
the first key in the hash table would work fine.  Since that's also
the first key of .keys(), I figured "first" was an appropriate
qualification.  Sure, it's not guaranteed to remain the first key, but
neither is the first item in a list.

That said, getting rid of the first*() suite and adding popitem()
instead sounds attractive...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@digicool.com  Thu Nov 30 15:14:53 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:14:53 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
 <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14886.28397.991041.190430@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf@artcom-gmbh.de> writes:

    PF> Since [].pop() takes in optional index as parameter which
    PF> defaults to the *LAST* element, it would be better to make the
    PF> name explicit: .popfirstitem()

{}.pop() sounds like the right thing given its symmetry with [].pop().

It would have the semantics that it would remove and return a random
/item/ from the dictionary, which the implementation can make the
"first" item for convenience purposes.

It shouldn't dictate though, that {}.pop() == dict.items().pop() or
make any other guarantees about which item gets popped, because other
implementations (or indeed, other mapping-conformant objects) may use
different underlying representations that could make this inefficient.

For added symmetry, {}.pop() should take an optional argument which is
the key of the item to remove and return.

-Barry


From barry@digicool.com  Thu Nov 30 15:23:21 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:23:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
 <200011301240.HAA26261@cj20424-a.reston1.va.home.com>
Message-ID: <14886.28905.691007.74511@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Aargh!  I thought I told you to submit a patch (to SF), not
    GvR> to check it in.

    GvR> The code looks okay but we'll have to discuss whether we
    GvR> really want all three (as opposed to only firstitem()).  I'm
    GvR> also not sure whether KeyError might be a better exception.

    GvR> Don't touch it for now, but let's think this over a bit.

And don't forget that when we /do/ decide on the right thing, that
UserDict needs to map <ha ha> the change too.

-Barry


From guido@python.org  Thu Nov 30 15:22:06 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 10:22:06 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:14:53 EST."
 <14886.28397.991041.190430@anthem.concentric.net>
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>
 <14886.28397.991041.190430@anthem.concentric.net>
Message-ID: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>

> {}.pop() sounds like the right thing given its symmetry with [].pop().
> 
> It would have the semantics that it would remove and return a random
> /item/ from the dictionary, which the implementation can make the
> "first" item for convenience purposes.
> 
> It shouldn't dictate though, that {}.pop() == dict.items().pop() or
> make any other guarantees about which item gets popped, because other
> implementations (or indeed, other mapping-conformant objects) may use
> different underlying representations that could make this inefficient.
> 
> For added symmetry, {}.pop() should take an optional argument which is
> the key of the item to remove and return.

That seems a nice touch, until you realize that [].pop() returns the
value, while {}.pop() returns a (key, value) pair.  Also: if you
already know the key, the argument that you can't do it in O(1) time
is no longer valid.  So, a -0.5.

One more concern: if you repeatedly remove the *first* item, the hash
table will start looking lobsided.  Since we don't resize the hash
table on deletes, maybe picking an item at random (but not using an
expensive random generator!) would be better.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gvwilson@nevex.com  Thu Nov 30 15:21:24 2000
From: gvwilson@nevex.com (Greg Wilson)
Date: Thu, 30 Nov 2000 10:21:24 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <14886.28397.991041.190430@anthem.concentric.net>
Message-ID: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>

> Barry Warsaw wrote:
> {}.pop() sounds like the right thing given its symmetry with [].pop().
> [details]

Should we then add dict.push(), which would add a key/value pair to the
dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
but I think anyone who's gone through a CS course in data structures
and algorithms will expect push() where there's pop(), and it will allow
functions to be polymorphic over lists and dicts.

Greg (who's not lobbying for it, just wondering if it's a loose end)


From mwh21@cam.ac.uk  Thu Nov 30 15:35:31 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: Thu, 30 Nov 2000 15:35:31 +0000 (GMT)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re:
 {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <Pine.LNX.4.10.10011301534070.27976-100000@localhost.localdomain>

On Thu, 30 Nov 2000, Greg Wilson wrote:

> > Barry Warsaw wrote:
> > {}.pop() sounds like the right thing given its symmetry with [].pop().
> > [details]
> 
> Should we then add dict.push(), which would add a key/value pair to the
> dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
> but I think anyone who's gone through a CS course in data structures
> and algorithms will expect push() where there's pop(), and it will allow
> functions to be polymorphic over lists and dicts.

'cept there's no list.push - and I don't think you're proposing spelling
"dict.push" "dict.append", are you?

Cheers,
M.



From fdrake@acm.org  Thu Nov 30 15:34:50 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 10:34:50 -0500 (EST)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
References: <14886.28397.991041.190430@anthem.concentric.net>
 <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <14886.29594.947340.480556@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Should we then add dict.push(), which would add a key/value pair to the
 > dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),

  We don't have [].push(), and I don't think the metaphor really works
well with dictionaries.  There's also an implied ordering relationship
between push() and pop(), and this doesn't hold for dictionaries
either.

 > but I think anyone who's gone through a CS course in data structures
 > and algorithms will expect push() where there's pop(), and it will allow
 > functions to be polymorphic over lists and dicts.

  There's only a limited amount on conceptual polymorphism between the
two; artificial extending that may lead to a lot of programmers using
the wrong structure for their application.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations



From gstein@lyra.org  Thu Nov 30 17:08:10 2000
From: gstein@lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 09:08:10 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <200011301231.EAA22228@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Thu, Nov 30, 2000 at 04:31:07AM -0800
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
Message-ID: <20001130090810.A25840@lyra.org>

On Thu, Nov 30, 2000 at 04:31:07AM -0800, Moshe Zadka wrote:
>...
> *** dictobject.c	2000/09/01 23:29:27	2.65
> --- dictobject.c	2000/11/30 12:31:00	2.66
> ***************
> *** 710,713 ****
> --- 710,782 ----
>   
>   static PyObject *
> + dict_firstkey(register dictobject *mp, PyObject *args)
> + {
> + 	register int i;
> + 
> + 	if (!PyArg_NoArgs(args))
> + 		return NULL;

PyArg_NoArgs() is deprecated because you cannot specify the method name. The
code should be:

    if (!PyArg_ParseTuple(args, ":firstkey"))
        return NULL;

(same in the next two funcs)

>...
> + 		if (mp->ma_table[i].me_value != NULL) {
> + 			PyObject *key = mp->ma_table[i].me_key;
> + 			PyObject *value = mp->ma_table[i].me_value;
> + 			PyObject *item = PyTuple_New(2);
> + 			if (item == NULL) {
> + 				return NULL;
> + 			}
> + 			Py_INCREF(key);
> + 			PyTuple_SetItem(item, 0, key);
> + 			Py_INCREF(value);
> + 			PyTuple_SetItem(item, 1, value);

These can be PyTuple_SET_ITEM() since you know you have a tuple.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From jeremy@alum.mit.edu  Thu Nov 30 17:23:04 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 30 Nov 2000 12:23:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130090810.A25840@lyra.org>
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
 <20001130090810.A25840@lyra.org>
Message-ID: <14886.36088.34790.149929@bitdiddle.concentric.net>

Let's back out these changes and put them in a patch.  There is still
a lot of discussion going on.  The patch itself has some small
problems.  And it is incomplete -- no test cases, no documentation,
etc.

Jeremy


From guido@python.org  Thu Nov 30 17:39:16 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 12:39:16 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:23:04 EST."
 <14886.36088.34790.149929@bitdiddle.concentric.net>
References: <200011301231.EAA22228@slayer.i.sourceforge.net> <20001130090810.A25840@lyra.org>
 <14886.36088.34790.149929@bitdiddle.concentric.net>
Message-ID: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>

> Let's back out these changes and put them in a patch.  There is still
> a lot of discussion going on.  The patch itself has some small
> problems.  And it is incomplete -- no test cases, no documentation,
> etc.

Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
clever with cvs admin, just use -j.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@digicool.com  Thu Nov 30 18:13:24 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 13:13:24 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
Message-ID: <14886.39108.175356.483917@anthem.concentric.net>

I was just surprised by something I've never observed before from
string.capitalize().  There is clearly either a bug (really two) in
the code or in the documentation.

>>> '__Myfoo'.capitalize()
'__myfoo'

Huh?  The documentation says:

    capitalize () 
       Return a copy of the string with only its first character capitalized.

So it's only doing one of the three things it's promising!  For one
thing, it's modifying something other than the first character, and
for another, it's actually swapping the case of the character it /is/
changing.  At least it returns a copy. :)

Given the documentation, I would have expected the return value to be
the same as the original string, i.e. unchanged.

So which is it?  Does the description of the method need to be made
more complicated, or does the code need to be simplified <wink>?

-Barry


From mal@lemburg.com  Thu Nov 30 18:19:29 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:19:29 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A269A31.35FE0AEF@lemburg.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize ()
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)
> 
> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

Huh ? The "documentation" says that you will get a copy in which
only the first character is capitalized (and all others are
converted to lower-case).

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

I'd say this is a documentation bug (and I'm authorized since
I wrote that snippet called "documentation" ;-).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mwh21@cam.ac.uk  Thu Nov 30 18:21:42 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:21:42 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: barry@digicool.com's message of "Thu, 30 Nov 2000 13:13:24 -0500"
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>

barry@digicool.com (Barry A. Warsaw) writes:

> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize () 
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)

>>> 'AaAaAa'.capitalize()
'Aaaaaa'

That's not a ridiculous reading of the above docs.  It all depends
whether you think "being capitalized" is a property or an action, I
guess.

> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

I would probably have expected that, too.  But I'm not really sure
why.

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

Clarifying the docs won't break any code.  Not sure that changing the
code will much, either.

Oooh, here's something a bit more serious though:

>>> u'aAaAaA'.capitalize()
u'AAaAaA'

Something obviously Needs To Be Done.  My hunch is to change
string_capitalize, but that may be just me (and probably Barry).

Cheers,
M.

-- 
  Famous remarks are very seldom quoted correctly.
                                                    -- Simeon Strunsky



From mal@lemburg.com  Thu Nov 30 18:38:04 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:38:04 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A269E8C.6E1E0B05@lemburg.com>

> Oooh, here's something a bit more serious though:
> 
> >>> u'aAaAaA'.capitalize()
> u'AAaAaA'
> 
> Something obviously Needs To Be Done.  My hunch is to change
> string_capitalize, but that may be just me (and probably Barry).

Ouch. That's a bug.

Here's what the string.py module has to say about this BTW:

# Capitalize a string, e.g. "aBc  dEf" -> "Abc  def".
def capitalize(s):
    """capitalize(s) -> string

    Return a copy of the string s with only its first character
    capitalized.

    """
    return s.capitalize()


Note that .title() is very similar to the way string.capitalize()
works. unicode.title() also uses the title case information available
for Unicode characters.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From mwh21@cam.ac.uk  Thu Nov 30 18:47:37 2000
From: mwh21@cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:47:37 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 19:38:04 +0100"
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>
Message-ID: <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> > Oooh, here's something a bit more serious though:
> > 
> > >>> u'aAaAaA'.capitalize()
> > u'AAaAaA'
> > 
> > Something obviously Needs To Be Done.  My hunch is to change
> > string_capitalize, but that may be just me (and probably Barry).
> 
> Ouch. That's a bug.

Yes.
 
> Here's what the string.py module has to say about this BTW:

I said "string_capitalize", i.e. "stringobject.c:string_capitalize",
i.e I think 

>>> 'aAaA'.capitalize()

should result in

'AAaA'

We're not doing too well at understanding each other today, are we?

> Note that .title() is very similar to the way string.capitalize()
> works. unicode.title() also uses the title case information available
> for Unicode characters.

Ah yes.  So in the interests of reducing redunancy, .capitalize should
probably do something usefully different...

Cheers,
M.

-- 
59. In English every word can be verbed. Would that it were so in
    our programming languages.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From jim@interet.com  Thu Nov 30 18:49:28 2000
From: jim@interet.com (James C. Ahlstrom)
Date: Thu, 30 Nov 2000 13:49:28 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A26A138.350235C@interet.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:

I guess I am a total geek, but it seems right to me.  The
capital of "_" is "_", and the rest was converted to lower
case.

JimA


From Moshe Zadka <moshez@math.huji.ac.il>  Thu Nov 30 18:55:56 2000
From: Moshe Zadka <moshez@math.huji.ac.il> (Moshe Zadka)
Date: Thu, 30 Nov 2000 20:55:56 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects
 dictobject.c,2.65,2.66
In-Reply-To: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>

On Thu, 30 Nov 2000, Guido van Rossum wrote:

> > Let's back out these changes and put them in a patch.  There is still
> > a lot of discussion going on.  The patch itself has some small
> > problems.  And it is incomplete -- no test cases, no documentation,
> > etc.
> 
> Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> clever with cvs admin, just use -j.

OK, I'll try. If CVS gets the best of me, I'll ask for your help...
--
Moshe Zadka <moshez@math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez



From mal@lemburg.com  Thu Nov 30 18:56:45 2000
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:56:45 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com> <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A26A2ED.EFF487C3@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> > > Oooh, here's something a bit more serious though:
> > >
> > > >>> u'aAaAaA'.capitalize()
> > > u'AAaAaA'
> > >
> > > Something obviously Needs To Be Done.  My hunch is to change
> > > string_capitalize, but that may be just me (and probably Barry).
> >
> > Ouch. That's a bug.
> 
> Yes.
> 
> > Here's what the string.py module has to say about this BTW:
> 
> I said "string_capitalize", i.e. "stringobject.c:string_capitalize",

The string methods were built closely after the string module
functions, so that's what the reference is...

> i.e I think
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'
> 
> We're not doing too well at understanding each other today, are we?

:)
 
> > Note that .title() is very similar to the way string.capitalize()
> > works. unicode.title() also uses the title case information available
> > for Unicode characters.
> 
> Ah yes.  So in the interests of reducing redunancy, .capitalize should
> probably do something usefully different...

Depends... we'll still need some sort of compatibility with
string.py.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From gstein@lyra.org  Thu Nov 30 19:12:54 2000
From: gstein@lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 11:12:54 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 30, 2000 at 08:55:56PM +0200
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial>
Message-ID: <20001130111254.N25840@lyra.org>

On Thu, Nov 30, 2000 at 08:55:56PM +0200, Moshe Zadka wrote:
> On Thu, 30 Nov 2000, Guido van Rossum wrote:
> 
> > > Let's back out these changes and put them in a patch.  There is still
> > > a lot of discussion going on.  The patch itself has some small
> > > problems.  And it is incomplete -- no test cases, no documentation,
> > > etc.
> > 
> > Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> > clever with cvs admin, just use -j.
> 
> OK, I'll try. If CVS gets the best of me, I'll ask for your help...

Not sure what -j does, but you can just do a reverse diff. For example:

$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

Apply the patch to your working copy, then check it in.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From akuchlin@mems-exchange.org  Thu Nov 30 19:18:29 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 30 Nov 2000 14:18:29 -0500
Subject: [Python-Dev] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130111254.N25840@lyra.org>; from gstein@lyra.org on Thu, Nov 30, 2000 at 11:12:54AM -0800
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial> <20001130111254.N25840@lyra.org>
Message-ID: <20001130141829.B20259@kronos.cnri.reston.va.us>

On Thu, Nov 30, 2000 at 11:12:54AM -0800, Greg Stein wrote:
>Not sure what -j does, but you can just do a reverse diff. For example:
>
>$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

-j basically does the work for you; you could do:
$ cvs update -j2.66 -j2.65 dictobject.c 

And then do a commit.  See the CVS book: 
http://cvsbook.red-bean.com/cvsbook.html#Examining%20And%20Reverting%20Changes

--amk


From barry@digicool.com  Thu Nov 30 19:31:00 2000
From: barry@digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 14:31:00 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
 <3A269A31.35FE0AEF@lemburg.com>
Message-ID: <14886.43764.472482.888678@anthem.concentric.net>

    MAL> Huh ? The "documentation" says that you will get a copy in
    MAL> which only the first character is capitalized (and all others
    MAL> are converted to lower-case).

I think if the documentation actually said exactly that, I wouldn't
have been confused. :)

>>>>> "MH" == Michael Hudson <mwh21@cam.ac.uk> writes:

    MH> That's not a ridiculous reading of the above docs.  It all
    MH> depends whether you think "being capitalized" is a property or
    MH> an action, I guess.

Good way to look at it.

I vote to clarify the documentation.

-Barry


From guido@python.org  Thu Nov 30 19:53:23 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 14:53:23 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: Your message of "30 Nov 2000 18:47:37 GMT."
 <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>
 <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011301953.OAA27996@cj20424-a.reston1.va.home.com>

> i.e I think 
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'

No.  The capitalize() method, which descends from the
string.capitalize(), and which definitely should return 'Aaaa' here.

This means that (a) the documentation needs some clarification, and
(b) the Unicode capitalize() method is broken and needs to be fixed.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim.one@home.com  Thu Nov 30 22:46:52 2000
From: tim.one@home.com (Tim Peters)
Date: Thu, 30 Nov 2000 17:46:52 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEADICAA.tim.one@home.com>

[Guido]
> ...
> One more concern: if you repeatedly remove the *first* item, the hash
> table will start looking lobsided.  Since we don't resize the hash
> table on deletes, maybe picking an item at random (but not using an
> expensive random generator!) would be better.

Which is the reason SETL doesn't specify *which* set item is removed:  if
you always start looking at "the front" of a dict that's being consumed, the
dict fills with turds without shrinking, you skip over them again and again,
and consuming the entire dict is still quadratic time.

Unfortunately, while using a random start point is almost always quicker
than that, the expected time for consuming the whole dict remains quadratic.

The clearest way around that is to save a per-dict search finger, recording
where the last search left off.  Start from its current value.  Failure if
it wraps around.  This is linear time in non-pathological cases (a
pathological case is one in which it isn't linear time <wink>).



From gward@mems-exchange.org  Thu Nov 30 23:14:39 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Thu, 30 Nov 2000 18:14:39 -0500
Subject: [Python-Dev] PEP 229 and 222
In-Reply-To: <20001128215748.A22105@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Tue, Nov 28, 2000 at 09:57:48PM -0500
References: <200011282213.OAA31146@slayer.i.sourceforge.net> <20001128171735.A21996@kronos.cnri.reston.va.us> <200011282301.SAA03304@cj20424-a.reston1.va.home.com> <20001128215748.A22105@kronos.cnri.reston.va.us>
Message-ID: <20001130181438.A21596@ludwig.cnri.reston.va.us>

On 28 November 2000, Andrew Kuchling said:
> On Tue, Nov 28, 2000 at 06:01:38PM -0500, Guido van Rossum wrote:
> >- Always shared libs.  What about Unixish systems that don't have
> >  shared libs?  What if you just want something to be hardcoded as
> >  statically linked, e.g. for security reasons?  (On the other hand
> 
> Beats me.  I'm not even sure if the Distutils offers a way to compile
> a static Python binary.  (GPW: well, does it?)

It's in the CCompiler interface, but hasn't been exposed to the outside
world.  (IOW, it's mainly a question of desiging the right setup
script/command line interface: the implementation should be fairly
straightforward, assuming the existing CCompiler classes do the right
thing for generating binary executables.)

        Greg


From gward@mems-exchange.org  Thu Nov 30 23:19:38 2000
From: gward@mems-exchange.org (Greg Ward)
Date: Thu, 30 Nov 2000 18:19:38 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEMDIBAA.tim.one@home.com>; from tim.one@home.com on Wed, Nov 29, 2000 at 01:23:10AM -0500
References: <200011281510.KAA03475@cj20424-a.reston1.va.home.com> <LNBBLJKPBEHFEDALKOLCOEMDIBAA.tim.one@home.com>
Message-ID: <20001130181937.B21596@ludwig.cnri.reston.va.us>

On 29 November 2000, Tim Peters said:
> [Guido]
> > ...
> > Because of its importance, the deprecation time of the string module
> > will be longer than that of most deprecated modules.  I expect it
> > won't be removed until Python 3000.
> 
> I see nothing in the 2.0 docs, code, or "what's new" web pages saying that
> it's deprecated.  So I don't think you can even start the clock on this one
> before 2.1 (a fuzzy stmt on the web page for the unused 1.6 release doesn't
> count ...).

FWIW, I would argue against *ever* removing (much less "deprecating",
ie. threatening to remove) the string module.  To a rough approximation,
every piece of Python code in existence code prior to Python 1.6 depends
on the string module.  I for one do not want to have to change all
occurences of string.foo(x) to x.foo() -- it just doesn't buy enough to
make it worth changing all that code.  

Not only does the amount of code to change mean the change would be
non-trivial, it's not always the right thing, especially if you happen
to be one of the people who dislikes the "delim.join(list)" idiom.  (I'm
still undecided.)

        Greg


From gregor@hoffleit.de  Fri Nov  3 23:14:15 2000
From: gregor@hoffleit.de (Gregor Hoffleit)
Date: Sat, 4 Nov 2000 00:14:15 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Include pyport.h,2.20,2.21
In-Reply-To: <200010050142.SAA08326@slayer.i.sourceforge.net>; from tim_one@users.sourceforge.net on Wed, Oct 04, 2000 at 06:42:32PM -0700
References: <200010050142.SAA08326@slayer.i.sourceforge.net>
Message-ID: <20001104001415.A2093@53b.hoffleit.de>

FYI: This misdefinition with LONG_BIT was due to a bug in glibc's limits.h. It
has been fixed in glibc 2.96.

    Gregor
    


On Wed, Oct 04, 2000 at 06:42:32PM -0700, Tim Peters wrote:
> Update of /cvsroot/python/python/dist/src/Include
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv5758/python/dist/src/Include
> 
> Modified Files:
> 	pyport.h 
> Log Message:
> Move LONG_BIT from intobject.c to pyport.h.  #error if it's already been
> #define'd to an unreasonable value (several recent gcc systems have
> misdefined it, causing bogus overflows in integer multiplication).  Nuke
> CHAR_BIT entirely.


From moshez at math.huji.ac.il  Wed Nov  1 15:38:38 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 1 Nov 2000 16:38:38 +0200 (IST)
Subject: [Python-Dev] Bug Changes
Message-ID: <Pine.GSO.4.10.10011011635030.21805-200000@sundial>

I've noticed the SF-FAQ still has the old "Use Jitterbug" thing about
bugs, even though we've moved to SourceForge bug manager. Attached is
a patch to correct everything. 

I haven't checked it in, because I'm not sure I my explanations are
clear at all. I'd be happy if someone went over it and saw if it's all
right.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez
-------------- next part --------------
Index: sf-faq.html
===================================================================
RCS file: /cvsroot/python/python/nondist/sf-html/sf-faq.html,v
retrieving revision 1.15
diff -c -r1.15 sf-faq.html
*** sf-faq.html	2000/08/25 07:55:48	1.15
--- sf-faq.html	2000/11/01 14:33:50
***************
*** 57,63 ****
  <h2><a href="#bugs">6. Bug Reporting</a></h2>
  <ol>
    <li><a href="#b1">Where can I submit/view bugs for Python?</a></li>
!   <li><a href="#b2">How do I use jitterbug?</a></li>
  </ol>
  
  <h2><a href="#appendix">A. Appendix</a></h2>
--- 57,63 ----
  <h2><a href="#bugs">6. Bug Reporting</a></h2>
  <ol>
    <li><a href="#b1">Where can I submit/view bugs for Python?</a></li>
!   <li><a href="#b2">How do I use the sourceforge bug manager?</a></li>
  </ol>
  
  <h2><a href="#appendix">A. Appendix</a></h2>
***************
*** 446,472 ****
  <h4>Q: Where can I submit/view bugs for Python?</h4>
  
  <h4>A:</h4>
! As of now [25-Jul-200] the Python project does not use SourceForge's bug
! tracking facility. This may change when there is a way to import the existing
! jitterbug database. The jitterbug database can be accessed through the
! following interface:
  
- <blockquote>
-   <tt><a
-   href="http://www.python.org/python-bugs">http://www.python.org/python-bugs</a></tt></blockquote>
- 
  <h3><a name="b1" id="b1"></a>6.1.:</h3>
  
! <h4>Q: How do I use jitterbug?</h4>
  
  <h4>A:</h4>
! To get the list of open bugs click on <tt>open</tt> (hidden between the second
! last and the last horizontal ruler).
  
! <p>To get a list of the bugs which are related to some area, enter an
! appropriate regular expression and press "Select Messages". Then select
! <tt>open</tt> (or whatever category you would like to view) as described
! above.</p>
  
  <h1><a name="appendix" id="appendix"></a>A. Appendix</h1>
  
--- 446,466 ----
  <h4>Q: Where can I submit/view bugs for Python?</h4>
  
  <h4>A:</h4>
! The Python project uses SourceForge's bug
! tracking facility. Go to 
! <a href="http://sourceforge.net/bugs/?group_id=5470">http://sourceforge.net/bugs/?group_id=5470</a> for all bug management needs.
  
  <h3><a name="b1" id="b1"></a>6.1.:</h3>
  
! <h4>Q: How do I use the sourceforge bug manager?</h4>
  
  <h4>A:</h4>
! By default, you will see the list of all Open bugs. You can change
! which bugs you're viewing by selecting the assigned_to/status/area/type
! select boxs.
  
! <p>To submit a bug, use the "Submit a Bug" link, near the top of the page.
! </p>
  
  <h1><a name="appendix" id="appendix"></a>A. Appendix</h1>
  
***************
*** 600,616 ****
      mailing list address; this address should no longer be used for patch
      submission. The patch manager is for <b>patches</b> only; if you have a
      problem or suggestion but don't know how to write the code for it, use the
!     <a href="http://www.python.org/search/search_bugs.html">Python Bugs
!     List</a> instead. The bugs list is searchable; if you have a problem and
      you're not sure if it has been reported or fixed already, this is the
      first place to look. (There used to be a separate TODO list; we now prefer
      that you use the bugs list for suggestions and requests too.)</li>
    <li style="list-style: none"><b>Submit documentation patches the same
      way.</b> When adding the patch, be sure to set the "<b>Category</b>" field
      to "<b>documentation</b>". For documentation errors without patches,
!     please use the <a
!     href="http://www.python.org/search/search_bugs.html">Python Bugs List</a>
!     instead.</li>
    <li>We like context diffs. We grudgingly accept unified diffs. <b>Straight
      ("ed-style") diffs are right out!</b> If you don't know how to generate
      context diffs, you're probably not qualified to produce high-quality
--- 594,608 ----
      mailing list address; this address should no longer be used for patch
      submission. The patch manager is for <b>patches</b> only; if you have a
      problem or suggestion but don't know how to write the code for it, use the
!     <a href="#b1">bug reporting mechanism</a> instead. 
!     The bugs list is searchable; if you have a problem and
      you're not sure if it has been reported or fixed already, this is the
      first place to look. (There used to be a separate TODO list; we now prefer
      that you use the bugs list for suggestions and requests too.)</li>
    <li style="list-style: none"><b>Submit documentation patches the same
      way.</b> When adding the patch, be sure to set the "<b>Category</b>" field
      to "<b>documentation</b>". For documentation errors without patches,
!     please use the <a href="b1">bug reporting mechanism</a>.</li>
    <li>We like context diffs. We grudgingly accept unified diffs. <b>Straight
      ("ed-style") diffs are right out!</b> If you don't know how to generate
      context diffs, you're probably not qualified to produce high-quality

From mwh21 at cam.ac.uk  Wed Nov  1 19:13:07 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 01 Nov 2000 18:13:07 +0000
Subject: [Python-Dev] Python Call Mechanism 
In-Reply-To: Jeremy Hylton's message of "Mon, 30 Oct 2000 09:59:00 -0500 (EST)"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>
Message-ID: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>

Jeremy Hylton <jeremy at alum.mit.edu> writes:

> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> Jeremy Hylton wrote:
>   >>
>   >> Update of /cvsroot/python/python/dist/src/Python In directory
>   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
>   >>
>   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
>   >> implementation is getting really hairy; should review it to see
>   >> if it can be simplified.
> 
>   MAL> How about a complete redesign of the whole call mechanism ?!
>
[chomp]
> 
> I'd be interested in looking at it.

Random idea that occurred while answering a post on comp.lang.python:

How about dumping the CALL_FUNCTION* opcodes, and replacing them with
two non-argumented opcodes, called for the sake of argument NCALL_FUNC
and NCALL_FUNC_KW.

NCALL_FUNC would pop a function object and a tuple off the stack and
apply the function to the tuple.

NCALL_FUNC_KW would do the same, then pop a dictionary and then do
the moral equivalent of f(*args,**kw).

As a preliminary it would be sensible to rework BUILD_MAP so that it
built dictionaries off the stack (a bit like BUILD_LIST, and like
CALL_FUNCTION now does with keyword arguments...) (and extend the
compiler to use this for literal dictionaries).

This would add an opcode or so per function call, but would probably
make life much simpler.

No time for implementation tonight, but could probably knock something
up tomorrow (depending how hard it turns out to be).

Thoughts?  Is that like what you did, Marc?

M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp




From jeremy at alum.mit.edu  Wed Nov  1 20:06:46 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:06:46 -0500 (EST)
Subject: [Python-Dev] Python Call Mechanism 
In-Reply-To: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
References: <200010301715.JAA32564@slayer.i.sourceforge.net>
	<39FDB5EA.B60EA39A@lemburg.com>
	<14845.36020.17063.951147@bitdiddle.concentric.net>
	<m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <14848.27078.923932.758419@bitdiddle.concentric.net>

My first impression is that this sounds like a nice simplifcation.

One question is how expensive this is for the common case.  Right now
arguments are pushed on the interpreter stack before CALL_FUNCTION is
executed, which is just a pointer assignment.  The pointers on the
stack are then assigned into the fast locals of the function after the
call.  Your scheme sounds like it would increase all function calls by
the cost of a tuple allocation.

It certainly wouldn't hurt to implement this, as it would provide some
practical implementation experience that would inform a PEP on the
subject.

On a related note, I have proposed a pep to add nested lexical scopes
for Python 2.1.  Barry's away for the moment, so it hasn't been
assigned a number yet.  It's just a proposal, not sure what Guido will
say in the end, but it also involves revising the function call
architecture.  I'll send a copy of the current draft (just notes)
under a separate subject.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 20:07:10 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:07:10 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
Message-ID: <14848.27102.223001.369662@bitdiddle.concentric.net>

Title: Statically Nested Scopes
Author: Jeremy Hylton <jeremy at digicool.com>
Status: Draft
Type: Standards Track
Created: 01-Nov-2000

Abstract

    This PEP proposes the additional of statically nested scoping
    (lexical scoping) for Python 2.1.  The current language definition
    defines exactly three namespaces that are used to resolve names --
    the local, global, and built-in namespaces.  The addition of
    nested scopes would allow resolution of unbound local names in
    enclosing functions' namespaces.

    One consequence of this change that will be most visible to Python
    programs is that lambda statements could reference variables in
    the namespaces where the lambda is defined.  Currently, a lambda
    statement uses default arguments to explicitly creating bindings
    in the lambda's namespace.

Notes

    This section describes several issues that will be fleshed out and
    addressed in the final draft of the PEP.  Until that draft is
    ready, please direct comments to the author.

    This change has been proposed many times in the past.  It has
    always been stymied by the possibility of creating cycles that
    could not be collected by Python's reference counting garbage
    collector.  The additional of the cycle collector in Python 2.0
    eliminates this concern.

    Guido once explained that his original reservation about nested
    scopes was a reaction to their overuse in Pascal.  In large Pascal
    programs he was familiar with, block structure was overused as an
    organizing principle for the program, leading to hard-to-read
    code. 

    Greg Ewing developed a proposal "Python Nested Lexical Scoping
    Enhancement" in Aug. 1999.  It is available from
    http://www.cosc.canterbury.ac.nz/~greg/python/lexscope.html

    Michael Hudson's bytecodehacks projects at
    http://sourceforge.net/projects/bytecodehacks/
    provides facilities to support nested scopes using the closure
    module. 

    Examples:

    def make_adder(n):
        def adder(x):
            return x + n
        return adder
    add2 = make_adder(2)
    add2(5) == 7

    
    from Tkinter import *
    root = Tk()
    Button(root, text="Click here",
           command = lambda : root.test.configure(text="..."))

    
    One controversial issue is whether it should be possible to modify
    the value of variables defined in an enclosing scope.  

    One part of the issue is how to specify that an assignment in the
    local scope should reference to the definition of the variable in
    an enclosing scope.  Assignment to a variable in the current scope
    creates a local variable in the scope.  If the assignment is
    supposed to refer to a global variable, the global statement must
    be used to prevent a local name from being created.  Presumably,
    another keyword would be required to specify "nearest enclosing
    scope."

    Guido is opposed to allow modifications (need to clarify exactly
    why).  If you are modifying variables bound in enclosing scopes,
    you should be using a class, he says.

    The problem occurs only when a program attempts to rebind the name
    in the enclosing scope.  A mutable object, e.g. a list or
    dictionary, can be modified by a reference in a nested scope; this
    is an obvious consequence of Python's reference semantics.  The
    ability to change mutable objects leads to an inelegant
    workaround: If a program needs to rebind an immutable object,
    e.g. a number or tuple, store the object in a list and have all
    references to the object use this list:

    def bank_account(initial_balance):
        balance = [initial_balance]
        def deposit(amount):
            balance[0] = balance[0] + amount
        def withdraw(amount):
            balance[0] = balance[0] - amount
        return deposit, withdraw

    I would prefer for the language to support this style of
    programming directly rather than encouraging programs to use this
    somewhat obfuscated style.  Of course, an instance would probably
    be clearer in this case.
 
    One implementation issue is how to represent the environment that
    stores variables that are referenced by nested scopes.  One
    possibility is to add a pointer to each frame's statically
    enclosing frame and walk the chain of links each time a non-local
    variable is accessed.  This implementation has some problems,
    because access to nonlocal variables is slow and causes garbage to
    accumulate unncessarily.  Another possibility is to construct an
    environment for each function that provides access to only the
    non-local variables.  This environment would be explicitly passed
    to nested functions.



From mal at lemburg.com  Wed Nov  1 21:31:08 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:31:08 +0100
Subject: [Python-Dev] Python Call Mechanism
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <3A007D8C.A9943D90@lemburg.com>

Michael Hudson wrote:
> 
> Jeremy Hylton <jeremy at alum.mit.edu> writes:
> 
> > >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> >
> >   MAL> Jeremy Hylton wrote:
> >   >>
> >   >> Update of /cvsroot/python/python/dist/src/Python In directory
> >   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
> >   >>
> >   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
> >   >> implementation is getting really hairy; should review it to see
> >   >> if it can be simplified.
> >
> >   MAL> How about a complete redesign of the whole call mechanism ?!
> >
> [chomp]
> >
> > I'd be interested in looking at it.
> 
> Random idea that occurred while answering a post on comp.lang.python:
> 
> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).
> 
> As a preliminary it would be sensible to rework BUILD_MAP so that it
> built dictionaries off the stack (a bit like BUILD_LIST, and like
> CALL_FUNCTION now does with keyword arguments...) (and extend the
> compiler to use this for literal dictionaries).
> 
> This would add an opcode or so per function call, but would probably
> make life much simpler.
> 
> No time for implementation tonight, but could probably knock something
> up tomorrow (depending how hard it turns out to be).
> 
> Thoughts?  Is that like what you did, Marc?

No, I just cleaned up the intertwine calling scheme currently
implemented in ceval.c. This allows a few improvments, one of
them being the possibility to inline C function calls in the
main loop (anyone ever trace the path Python takes when calling
a builtin function or method... you'd be surprised).

About your idea with the new opcodes: you could be touching
a performance relevant section there -- a ceval round-trip
may cost more than the added if()s in the CALL_FUNCION opcode.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Wed Nov  1 21:37:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:37:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <3A007EF8.2D9BDCF5@lemburg.com>

[pre-PEP]

This will break code... I'm not sure whether it's worth going
down this path just for the sake of being able to define
functions within functions.

Wouldn't it be a better idea to somehow add native acqusition
to Python's objects ?

We already have a slot which implements the "contains"
relationship. All we'd need is a way for a contained
object to register itself with the container in a 
way that doesn't produce cycles.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jeremy at alum.mit.edu  Wed Nov  1 21:48:53 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 15:48:53 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A007EF8.2D9BDCF5@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
Message-ID: <14848.33205.361821.679508@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.

How will this break code?  Any code written to use the scoping rules
will not work today.

Python already allows programs to define functions within functions.
That's not at issue.  The issue is how hard it is to use nested
functions, including lambdas.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?

No.

Seriously, I don't see how acquistion addresses the same issues at
all.  Feel free to explain what you mean.

  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.

The contains relationship has to do with container objects and their
contents.  A function's environment is not a container in the same
sense, so I don't see how this is related.

As I noted in the PEP, I don't see a compelling reason to avoid
cycles.

Jeremy



From mal at lemburg.com  Wed Nov  1 21:51:16 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:51:16 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
		<3A007EF8.2D9BDCF5@lemburg.com> <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <3A008244.8F558C8B@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> [pre-PEP] This will break code... I'm not sure whether it's
>   MAL> worth going down this path just for the sake of being able to
>   MAL> define functions within functions.
> 
> How will this break code?  Any code written to use the scoping rules
> will not work today.
> 
> Python already allows programs to define functions within functions.
> That's not at issue.  The issue is how hard it is to use nested
> functions, including lambdas.

The problem is that with nested scoping, a function defined
within another function will suddenly reference the variables
of the enclosing function as globals and not the module
globals... this could break code.

Another problem is that you can't reach out for the defining
module globals anymore (at least not in an easy way like today).

>   MAL> Wouldn't it be a better idea to somehow add native acqusition
>   MAL> to Python's objects ?
> 
> No.
> 
> Seriously, I don't see how acquistion addresses the same issues at
> all.  Feel free to explain what you mean.

It's not related to *statically* nested scopes, but is to
dynamically nested ones. Acquisition is basically about the
same thing: you acquire attributes from containers. The only
difference here is that the containment relationships are
defined at run-time.
 
>   MAL> We already have a slot which implements the "contains"
>   MAL> relationship. All we'd need is a way for a contained object to
>   MAL> register itself with the container in a way that doesn't
>   MAL> produce cycles.
> 
> The contains relationship has to do with container objects and their
> contents.  A function's environment is not a container in the same
> sense, so I don't see how this is related.
> 
> As I noted in the PEP, I don't see a compelling reason to avoid
> cycles.

Ok, we have cycle GC, but why create cycles when you don't
have to (Python can also be run without GC and then you'd run
into problems...) ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From skip at mojam.com  Wed Nov  1 22:57:17 2000
From: skip at mojam.com (Skip Montanaro)
Date: Wed, 1 Nov 2000 15:57:17 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <14848.37309.3616.710295@beluga.mojam.com>

    MAL> [pre-PEP] This will break code...

    Jeremy> How will this break code?

Suppose you have

    x = 1
    def f1():
        x = 2
        def inner():
            print x
        inner()

Today, calling f1() prints "1".  After your proposed changes I suspect it
would print "2".

Skip



From jeremy at alum.mit.edu  Wed Nov  1 22:18:49 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:18:49 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
	<3A008244.8F558C8B@lemburg.com>
Message-ID: <14848.35001.109767.606384@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.
  >>
  >> How will this break code?  Any code written to use the scoping
  >> rules will not work today.
  >>
  >> Python already allows programs to define functions within
  >> functions.  That's not at issue.  The issue is how hard it is to
  >> use nested functions, including lambdas.

  MAL> The problem is that with nested scoping, a function defined
  MAL> within another function will suddenly reference the variables
  MAL> of the enclosing function as globals and not the module
  MAL> globals... this could break code.

That's right it could, in the unlikely case that someone has existing
code today using nested functions where an intermediate function
defines a local variable that shadows a variable defined in an
enclosing scope.  

It should be straightfoward to build a tool that would detect this
case.

It would be pretty poor programming style, so I think it would be fine
to break backwards compatibility here.

  MAL> Another problem is that you can't reach out for the defining
  MAL> module globals anymore (at least not in an easy way like
  MAL> today).

I think we would want to keep globals implemented just the way they
are.  The compiler would need to determine exactly which variables are
access from enclosing scopes and which are globals.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?
  >>
  >> Seriously, I don't see how acquistion addresses the same issues
  >> at all.  Feel free to explain what you mean.

  MAL> It's not related to *statically* nested scopes, but is to
  MAL> dynamically nested ones. Acquisition is basically about the
  MAL> same thing: you acquire attributes from containers. The only
  MAL> difference here is that the containment relationships are
  MAL> defined at run-time.

Static scoping and dynamic scoping are entirely different beasts,
useful for different things.  I want to fix, among other things,
lambdas.  That's a static issue.
 
  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.
  >>
  >> The contains relationship has to do with container objects and
  >> their contents.  A function's environment is not a container in
  >> the same sense, so I don't see how this is related.
  >>
  >> As I noted in the PEP, I don't see a compelling reason to avoid
  >> cycles.

  MAL> Ok, we have cycle GC, but why create cycles when you don't have
  MAL> to (Python can also be run without GC and then you'd run into
  MAL> problems...) ?

If we can avoid cycles, sure.  I would prefer a simple design that
allowed cycles to a complicated design that avoided them.  Exactly
where to draw the line between simple and complex is a matter of
taste, of course.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 22:20:08 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:20:08 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.37309.3616.710295@beluga.mojam.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
	<14848.37309.3616.710295@beluga.mojam.com>
Message-ID: <14848.35080.73182.888834@bitdiddle.concentric.net>

Thanks.  I expect there is very little code that depends on this sort
of behavior, since it is confusing to read.  Many readers,
particularly novices, could reasonably expect Python to print 2 now.

As I explained to MAL, I think we would need to provide a code
analysis tool that identified these problems.  It's probably helpful
to generate warning about this right now, since it's rather
obfuscated.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 23:35:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:35:14 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
Message-ID: <14848.39586.832800.139182@bitdiddle.concentric.net>

I just did a clean configure and make from the latest CVS tree.  It
seems to get stuck in a loop calling makesetup over and over again.
(Output below.)

Any idea what's going wrong?

Jeremy

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
making Makefile in subdirectory .
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3'
making Makefile in subdirectory Parser
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
making Makefile in subdirectory Grammar
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
make[1]: Nothing to be done for `Makefile'.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
making Makefile in subdirectory Objects
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
making Makefile in subdirectory Python
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Python'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Python'
making Makefile in subdirectory Modules
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------

[etc.]



From thomas at xs4all.net  Wed Nov  1 23:41:34 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 1 Nov 2000 23:41:34 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <200011012237.OAA06642@slayer.i.sourceforge.net>; from twouters@users.sourceforge.net on Wed, Nov 01, 2000 at 02:37:39PM -0800
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
Message-ID: <20001101234134.O12812@xs4all.nl>

On Wed, Nov 01, 2000 at 02:37:39PM -0800, Thomas Wouters wrote:

> Modified Files:
> 	pep-0204.txt 
> Log Message:

> Update this PEP to current, harsh, reality. It's been rejected :) If at all
> possible, the reasoning should be extended to include the real reasons it
> was rejected -- this is just guesswork from my side. (This means you, Guido,
> or anyone who can channel Guido enough to call himself Guido.)

In addition to that, PEP 0 also needs to be updated. Shall I do that myself,
now that Barry is apparently away ? While I was at it, I also noticed PEP
0200 still says 'Incomplete', though that might be by design.

Yay-another-first-for-me---first-rejected-PEP-ly y'rs, ;)
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jeremy at alum.mit.edu  Wed Nov  1 23:54:07 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:54:07 -0500 (EST)
Subject: [Python-Dev] Move idle PEPs to status deferred
Message-ID: <14848.40719.914227.606929@bitdiddle.concentric.net>

Barry,

We should start working on any PEPs that are going to be considered
for Python 2.1.  There are a bunch of old PEPs that have been
assigned and then ignored.  You have marked many of them as deferred.
We should assign new authors for the PEPs we care about and move all
the other deferred PEPs somewhere else in the index.

The following PEPs have been inactive for at least two months:
 I     2  pep-0002.txt  Procedure for Adding New Modules       Raymond
 S   202  pep-0202.txt  List Comprehensions                    Peters
 SD  205  pep-0205.txt  Weak References                        Drake
 I   206  pep-0206.txt  2.0 Batteries Included                 Zadka
 SD  207  pep-0207.txt  Rich Comparisons                       Ascher
 SD  208  pep-0208.txt  Reworking the Coercion Model           Ascher
 SD  209  pep-0209.txt  Adding Multidimensional Arrays         Ascher
 SD  210  pep-0210.txt  Decoupling the Interpreter Loop        Ascher
 SD  211  pep-0211.txt  Adding New Linear Algebra Operators    Wilson
 SD  212  pep-0212.txt  Loop Counter Iteration                 Schneider-Kamp
 SD  213  pep-0213.txt  Attribute Access Handlers              Prescod
 SD  215  pep-0215.txt  String Interpolation                   Yee
 I   216  pep-0216.txt  Docstring Format                       Zadka
 SD  217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 SD  218  pep-0218.txt  Adding a Built-In Set Object Type      Wilson
 SD  219  pep-0219.txt  Stackless Python                       McMillan
 I   220  pep-0220.txt  Coroutines, Generators, Continuations  McMillan
 S   224  pep-0224.txt  Attribute Docstrings                   Lemburg

I think we should find new authors for PEPs 207 and 208 and work on
them for 2.1.  I imagine David would be happy to pass the torch on
these issues.

I assume Gordon will be working on the stackless PEPs, but we should
confirm this.

For all of the other PEPs, authors who want to have them considered
for 2.1 should provide some updates to their PEPs.

We should also simplfy the PEP index so that deferred PEPs are
collected at the bottom or something like that.

Jeremy



From greg at cosc.canterbury.ac.nz  Thu Nov  2 01:34:04 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:34:04 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <200011020034.NAA29444@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy at alum.mit.edu>:

> Seriously, I don't see how acquistion addresses the same issues at
> all.

My proposal for nested scopes was actually an acquisition-like
mechanism. The idea was to avoid unbreakable cycles by deferring
the creation of a closure from when the function is defined
to when it is used.

Guido rejected my implementation for various good reasons.
It could be modified to overcome most of those objections,
however.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Thu Nov  2 01:37:00 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:37:00 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
Message-ID: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> The problem is that with nested scoping, a function defined
> within another function will suddenly reference the variables
> of the enclosing function

This could be avoided by requiring that variables which are
to be visible in an inner scope be marked somehow in the
scope where they are defined.

I don't think it's a serious enough problem to be worth
fixing that way, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From gmcm at hypernet.com  Thu Nov  2 02:37:09 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Wed, 1 Nov 2000 20:37:09 -0500
Subject: [Python-Dev] Move idle PEPs to status deferred
In-Reply-To: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <3A007EF5.21540.6BA02C9@localhost>

Jeremy wrote:

> I assume Gordon will be working on the stackless PEPs, but we should
> confirm this.

Yes, I will.

- Gordon



From tim_one at email.msn.com  Thu Nov  2 08:02:04 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 02:02:04 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

Note that this problem will be much worse in Python:  in Pascal, you could
always "look up" for the closest-containing func/proc that explicitly
declares a referenced vrbl.  In Python, you have to indirectly *deduce*
which vrbls are local to a def, by searching the entire body for an
appearance as a binding target.  So you have to "look up" and "look down"
from the reference point, and it's easy to miss a binding target.

    i = 6
    def f(x):
        def g():
            print i
        # ...
        # skip to the next page
        # ...
        for i in x:  # ah, i *is* local to f, so this is what g sees
            pass
        g()

>    def bank_account(initial_balance):
>        balance = [initial_balance]
>        def deposit(amount):
>            balance[0] = balance[0] + amount
>        def withdraw(amount):
>            balance[0] = balance[0] - amount
>        return deposit, withdraw

Unfortunately for proponents, this is exactly the kind of SICP example that
is much better done via a class.  Not only is the closure version strained
by comparison, but as is usual it manages to create a bank account with a
write-only balance <0.9 wink>.

    def deposit(amount):
        global bank_account.balance
        balance += amount

is one old suggested way to explicitly declare non-local names and the
enclosing block to which they are local (and in analogy with current
"global", mandatory if you want to rebind the non-local name, optional if
you only want to reference it).  There are subtleties here, but explicit is
better than implicit, and the subtleties are only subtler if you refuse
(like Scheme) to make the intent explicit.

for-real-fun-think-about-"exec"-abuses-ly y'rs  - tim





From moshez at math.huji.ac.il  Thu Nov  2 09:32:11 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 10:32:11 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

[Tim Peters]
> Note that this problem will be much worse in Python:  in Pascal, you could
> always "look up" for the closest-containing func/proc that explicitly
> declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> which vrbls are local to a def, by searching the entire body for an
> appearance as a binding target.  So you have to "look up" and "look down"
> from the reference point, and it's easy to miss a binding target.

This is a tool problem, and should be solved with good tools.
Of course, installing the corret tools in people's minds will require
some technological discoveries.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From gstein at lyra.org  Thu Nov  2 09:46:04 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 00:46:04 -0800 (PST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
Message-ID: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>

On Thu, 2 Nov 2000, Moshe Zadka wrote:
> [Tim Peters]
> > Note that this problem will be much worse in Python:  in Pascal, you could
> > always "look up" for the closest-containing func/proc that explicitly
> > declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> > which vrbls are local to a def, by searching the entire body for an
> > appearance as a binding target.  So you have to "look up" and "look down"
> > from the reference point, and it's easy to miss a binding target.
> 
> This is a tool problem, and should be solved with good tools.
> Of course, installing the corret tools in people's minds will require
> some technological discoveries.

Bleck. Those tools are a crutch to deal with a poor language design /
feature. And are those tools portable? Are they part of everybody's
standard tool set? Will vi, emacs, and MS DevStudio all have those
capabilities?

Not a chance. Personally, I'll take Guido's point of view and say they are
inherently hard to deal with; therefore, punt them.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/





From mal at lemburg.com  Thu Nov  2 13:12:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 13:12:42 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
Message-ID: <3A015A3A.4EC7DBC6@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com>:
> 
> > The problem is that with nested scoping, a function defined
> > within another function will suddenly reference the variables
> > of the enclosing function
> 
> This could be avoided by requiring that variables which are
> to be visible in an inner scope be marked somehow in the
> scope where they are defined.
> 
> I don't think it's a serious enough problem to be worth
> fixing that way, though.

It may not look serious, but changing the Python lookup scheme
is, since many inspection tools rely and reimplement exactly
that scheme. With nested scopes, there would be next to no
way to emulate the lookups using these tools.

To be honest, I don't think static nested scopes buy us all that
much. You can do the same now, by using keyword arguments which
isn't all that nice, but works great and makes the scope clearly
visible.

Dynamic nested scopes is another topic... those are *very*
useful; especially when it comes to avoiding global variables
and implementing programs which work using control objects
instead of global function calls.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Thu Nov  2 15:30:26 2000
From: barry at wooz.org (barry at wooz.org)
Date: Thu, 2 Nov 2000 09:30:26 -0500 (EST)
Subject: [Python-Dev] Re: Move idle PEPs to status deferred
References: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <14849.31362.805142.553781@anthem.concentric.net>

I was away from email for most of the day yesterday.  I'll do a swing
through all the outstanding PEP updates later today -- after I finish
catching up on email. :(

-Barry




From moshez at math.huji.ac.il  Thu Nov  2 15:49:37 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 16:49:37 +0200 (IST)
Subject: [Python-Dev] PEP-0217
Message-ID: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>

I need some help:

1) BDFL pronouncement
2) someone to see about the Jython issue.

Thank you for your co-operation.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From jeremy at alum.mit.edu  Thu Nov  2 16:18:47 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 10:18:47 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
References: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
	<Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
Message-ID: <14849.34263.310260.404940@bitdiddle.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

  GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
  >> This is a tool problem, and should be solved with good tools.  Of
  >> course, installing the corret tools in people's minds will
  >> require some technological discoveries.

  GS> Bleck. Those tools are a crutch to deal with a poor language
  GS> design / feature. And are those tools portable? Are they part of
  GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
  GS> all have those capabilities?

Are you saying that compilers are a crutch and we should get rid of
them?  I don't think you intend that, but this is a completely
straightforward tool to build.  It is needed only for backwards
compatibility -- to identify scripts that depend on the changed
behavior.  There is no need for vi, emacs, or devstudio to understand
what's going on.

Jeremy



From moshez at math.huji.ac.il  Thu Nov  2 16:16:41 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:16:41 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.34263.310260.404940@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> >>>>> "GS" == Greg Stein <gstein at lyra.org> writes:
> 
>   GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
>   >> This is a tool problem, and should be solved with good tools.  Of
>   >> course, installing the corret tools in people's minds will
>   >> require some technological discoveries.
> 
>   GS> Bleck. Those tools are a crutch to deal with a poor language
>   GS> design / feature. And are those tools portable? Are they part of
>   GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
>   GS> all have those capabilities?
> 
> Are you saying that compilers are a crutch and we should get rid of
> them?  I don't think you intend that, but this is a completely
> straightforward tool to build.  It is needed only for backwards
> compatibility -- to identify scripts that depend on the changed
> behavior.  There is no need for vi, emacs, or devstudio to understand
> what's going on.

you guys are talking about different things.
Jeremy is talking about a tool to warn against incompatible changes
Greg is talking about a tool to identify, for each variable, what scope
it belongs to.

as-usual-the-answer-is-"you're-both-right"-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  2 04:26:14 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:26:14 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
             <3A015A3A.4EC7DBC6@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
            <3A015A3A.4EC7DBC6@lemburg.com> 
Message-ID: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>

[MAL]
> Dynamic nested scopes is another topic... those are *very*
> useful; especially when it comes to avoiding global variables
> and implementing programs which work using control objects
> instead of global function calls.

Marc-Andre, what are Dynamic nested scopes?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at math.huji.ac.il  Thu Nov  2 16:26:20 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:26:20 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> [MAL]
> > Dynamic nested scopes is another topic... those are *very*
> > useful; especially when it comes to avoiding global variables
> > and implementing programs which work using control objects
> > instead of global function calls.
> 
> Marc-Andre, what are Dynamic nested scopes?

If MAL means dynamic scoping (which I understood he does), then this 
simply means:

when looking for a variable "foo", you first search for it in the local
namespace. If not there, the *caller's* namespace, and so on. In the
end, the caller is the __main__ module, and if not found there, it is
a NameError.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  2 04:29:03 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:29:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
             <3A015A3A.4EC7DBC6@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
            <3A015A3A.4EC7DBC6@lemburg.com> 
Message-ID: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>

> It may not look serious, but changing the Python lookup scheme
> is, since many inspection tools rely and reimplement exactly
> that scheme. With nested scopes, there would be next to no
> way to emulate the lookups using these tools.

So fix the tools.

> To be honest, I don't think static nested scopes buy us all that
> much. You can do the same now, by using keyword arguments which
> isn't all that nice, but works great and makes the scope clearly
> visible.

Yes.  It's a hack that gets employed over and over.  And it has
certain problems.  We added 'import as' to get rid of a common
practice that was perceived unclean.  Maybe we should support nested
scopes to get rid of another unclean common practice?

I'm not saying that we definitely should add this to 2.1 (there's
enough on our plate already) but we should at least consider it, and
now that we have cycle GC, the major argument against it (that it
causes cycles) is gone...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at math.huji.ac.il  Thu Nov  2 16:29:55 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:29:55 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

This is the perfect moment to ask: what do we have on our plates for 2.1?
Shouldn't we have a list of goals for it or something? As a first-order
approximation, what PEPs are expected to be included? And, most the
conspiracy-theory question, what are Digital Creations' goals for Python?

We now return you to our regularily scheduled bug fixing.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Thu Nov  2 16:34:42 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 10:34:42 -0500
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 02, 2000 at 05:29:55PM +0200
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>
Message-ID: <20001102103442.B5027@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
>Shouldn't we have a list of goals for [Python 2.1] or something? As a 
>first-order approximation, what PEPs are expected to be included? And, 

Stuff I personally want to get done:
  * Finish PEP 222, "Web Programming Improvements" and implement whatever
    emerges from it.

  * Write a PEP on using Distutils to build the modules that come with
    Python, and implement it if accepted.

  * Work on something CPAN-like.  This may or may not have repercussions for
    the core; I don't know.

--amk



From thomas at xs4all.net  Thu Nov  2 16:42:39 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 16:42:39 +0100
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102103442.B5027@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Thu, Nov 02, 2000 at 10:34:42AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <20001102164239.R12812@xs4all.nl>

On Thu, Nov 02, 2000 at 10:34:42AM -0500, Andrew Kuchling wrote:

>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Probably not, though perhaps a new module would be nice. As for the
CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
Apachecon, and I think it would be suited extremely well as the transmission
protocol for SPAM (or however you want to call the Python CPAN ;). You can
do the uploading, downloading and searching for modules using WebDAV without
too much pain, and there's excellent WebDAV support for Apache ;)

Is anyone working on something like this, or even thinking about it ? I'm
not deep enough into distutils to join that SIG, but I definately would join
a CPyAN SIG ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From bckfnn at worldonline.dk  Thu Nov  2 16:37:43 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Thu, 02 Nov 2000 15:37:43 GMT
Subject: [Python-Dev] PEP-0217
In-Reply-To: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
Message-ID: <3a0186d5.25979536@smtp.worldonline.dk>

[Moshe Zadka]

>I need some help:
>
>1) BDFL pronouncement
>2) someone to see about the Jython issue.

I don't see any problems with this. This is already handled by a method
in the jython runtime (Py.printResult).

However, I think your example implementation should be:

    def displayhook(o):
        if o is None:
            return
        __builtin__._ = None
        print `o`
        __builtin__._ = o

I don't why, but that is the implementation currently used by Jython
(and I think CPython too).

regards,
finn



From moshez at math.huji.ac.il  Thu Nov  2 17:05:20 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:05:20 +0200 (IST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <Pine.GSO.4.10.10011021804210.1290-100000@sundial>

On Thu, 2 Nov 2000, Finn Bock wrote:

> However, I think your example implementation should be:
> 
>     def displayhook(o):
>         if o is None:
>             return
>         __builtin__._ = None
>         print `o`
>         __builtin__._ = o
> 

You're right.
I'll just add the necessary Jython changes to PEP-0217.
Thanks a lot.

I don't like this either, but the good news is that in Py2.1, you'll
be able to change this in site.py <wink>

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Thu Nov  2 17:05:14 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:05:14 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <3A0190BA.940FFADA@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 1 Nov 2000, Guido van Rossum wrote:
> 
> > [MAL]
> > > Dynamic nested scopes is another topic... those are *very*
> > > useful; especially when it comes to avoiding global variables
> > > and implementing programs which work using control objects
> > > instead of global function calls.
> >
> > Marc-Andre, what are Dynamic nested scopes?
> 
> If MAL means dynamic scoping (which I understood he does), then this
> simply means:
> 
> when looking for a variable "foo", you first search for it in the local
> namespace. If not there, the *caller's* namespace, and so on. In the
> end, the caller is the __main__ module, and if not found there, it is
> a NameError.

That would be one application, yes.

With dynamic scoping I meant that the context of a lookup is
defined at run-time and by explicitely or implicitely
hooking together objects which then define the nesting.

Environment acquisition is an example of such dynamic scoping:
attribute lookups are passed on to the outer scope in case they
don't resolve on the inner scope, e.g. say you have
object a with a.x = 1; all other objects don't define .x.
Then a.b.c.d.x will result in lookups
1. a.b.c.d.x
2. a.b.c.x
3. a.b.x
4. a.x -> 1

This example uses attribute lookup -- the same can be done for
other nested objects by explicitely specifying the nesting
relationship.

Jim's ExtensionClasses allow the above by using a lot of
wrappers around objects -- would be nice if we could come
up with a more general scheme which then also works for
explicit nesting relationships (e.g. dictionaries which
get hooked together -- Jim's MultiMapping does this).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Thu Nov  2 17:06:38 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:06:38 +0100
Subject: [Python-Dev] Python 2.1 tasks
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <3A01910E.A19FFBB2@lemburg.com>

Andrew Kuchling wrote:
> 
> On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
> >Shouldn't we have a list of goals for [Python 2.1] or something? As a
> >first-order approximation, what PEPs are expected to be included? And,
> 
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.
> 
>   * Write a PEP on using Distutils to build the modules that come with
>     Python, and implement it if accepted.
> 
>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Most important for 2.1 are probably:

1. new C level coercion scheme
2. rich comparisons
3. making the std lib Unicode compatible

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jeremy at alum.mit.edu  Thu Nov  2 17:11:30 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:11:30 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <14849.37426.860007.989619@bitdiddle.concentric.net>

>>>>> "TP" == Tim Peters <tim_one at email.msn.com> writes:

  TP> [Jeremy Hylton]
  >> ...  Guido once explained that his original reservation about
  >> nested scopes was a reaction to their overuse in Pascal.  In
  >> large Pascal programs he was familiar with, block structure was
  >> overused as an organizing principle for the program, leading to
  >> hard-to-read code.

  TP> Note that this problem will be much worse in Python: in Pascal,
  TP> you could always "look up" for the closest-containing func/proc
  TP> that explicitly declares a referenced vrbl.  In Python, you have
  TP> to indirectly *deduce* which vrbls are local to a def, by
  TP> searching the entire body for an appearance as a binding target.
  TP> So you have to "look up" and "look down" from the reference
  TP> point, and it's easy to miss a binding target.

I agree that visual inspection is a tad harder, but I contend that
existing programs that use the same name for a global variable and a
local variable -- and intend for the global to be visible within a
function nested in the local variable's region -- are confusing.  
It's too hard for a first-time reader of the code to figure out what
is going on.

Incidentally, I have yet to see an example of this problem occurring
in anyone's code.  All the examples seem a bit contrived.  I wonder if
anyone has an example in existing code.

  [My SICP example omitted]

  TP> Unfortunately for proponents, this is exactly the kind of SICP
  TP> example that is much better done via a class.

Indeed, the PEP says exactly that: This kind of program is better
done via a class.  My intent was not to show a compelling use of
mutable state.  Instead it was to show that with read-only access,
people could still modify values on enclosing scopes.  The issue is
whether the language allows the programmer to express this intent
clearly or if she has to jump through some hoops to accomplish it.

  TP>                                                Not only is the
  TP> closure version strained by comparison, but as is usual it
  TP> manages to create a bank account with a write-only balance <0.9
  TP> wink>.

  TP>     def deposit(amount):
  TP>         global bank_account.balance balance += amount

  TP> is one old suggested way to explicitly declare non-local names
  TP> and the enclosing block to which they are local (and in analogy
  TP> with current "global", mandatory if you want to rebind the
  TP> non-local name, optional if you only want to reference it).
  TP> There are subtleties here, but explicit is better than implicit,
  TP> and the subtleties are only subtler if you refuse (like Scheme)
  TP> to make the intent explicit.

I'm still not sure I like it, because it mixes local variables of a
function with attribute access on objects.  I'll add it to the
discussion in the PEP (if Barry approves the PEP <wink>), though.

Do you have any opinion on the subtleties?  The two that immediately
come to mind are: 1) whether the function's local are available as
attributes anywhere or only in nested scopes and 2) whether you can
create new local variable using this notation.

Jeremy



From moshez at math.huji.ac.il  Thu Nov  2 17:09:08 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:09:08 +0200 (IST)
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102164239.R12812@xs4all.nl>
Message-ID: <Pine.GSO.4.10.10011021807060.1290-100000@sundial>

On Thu, 2 Nov 2000, Thomas Wouters wrote:

> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

Cries for this sig have been already made in c.l.py.
I'm moving this discussion to meta-sig. Please discuss it there.
I'm willing to champion it, but I'll defer if Andrew or Greg want
to do it.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez





From jeremy at alum.mit.edu  Thu Nov  2 17:13:52 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:13:52 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A015A3A.4EC7DBC6@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
	<3A015A3A.4EC7DBC6@lemburg.com>
Message-ID: <14849.37568.510427.834971@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> It may not look serious, but changing the Python lookup scheme
  MAL> is, since many inspection tools rely and reimplement exactly
  MAL> that scheme. With nested scopes, there would be next to no way
  MAL> to emulate the lookups using these tools.

Can you say more about this issue?  It sounds like it is worth
discussing in the PEP, but I can't get a handle on exactly what the
problem is.  Any tool needs to implement or model Python's name
resolution algorithm, call it algorithm A.  If we change name
resolution to use algorithm B, then the tools need to implement or
model a new algorithm.  I don't see where the impossibility of
emulation comes in.

Jeremy




From guido at python.org  Thu Nov  2 05:24:29 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:24:29 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "01 Nov 2000 18:13:07 GMT."
             <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>  
            <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> 
Message-ID: <200011020424.XAA07810@cj20424-a.reston1.va.home.com>

> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).

No, this is a bad idea.  Long, long ago, all calls requird building a
tuple for the arguments first.  This tuple creation turned out to be a
major bottleneck.  That's why the current call opcode exists.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From barry at wooz.org  Thu Nov  2 17:22:32 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:22:32 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38088.903104.936944@anthem.concentric.net>

If we get lexical scoping, there should be a fast (built-in) way to
get at all the accessible names from Python.  I.e. currently I can do

    d = globals().copy()
    d.update(locals())

and know that `d' contains a dictionary of available names, with the
right overloading semantics.  (PEP 42 now includes a feature request
to make vars() do this by default.)

-Barry



From barry at wooz.org  Thu Nov  2 17:23:56 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:23:56 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38172.908316.107381@anthem.concentric.net>

This has been added as PEP 227.



From guido at python.org  Thu Nov  2 05:28:00 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:28:00 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "Wed, 01 Nov 2000 14:06:46 EST."
             <14848.27078.923932.758419@bitdiddle.concentric.net> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>  
            <14848.27078.923932.758419@bitdiddle.concentric.net> 
Message-ID: <200011020428.XAA07841@cj20424-a.reston1.va.home.com>

> It certainly wouldn't hurt to implement this, as it would provide some
> practical implementation experience that would inform a PEP on the
> subject.

If it solves the mess with supporting extended call syntax, adding
these opcodes might be a good idea.

But as I said, for the normal (not extended) case, the existing
CALL_FUNCTION opcode is the right thing to use unless you want things
to slow down significantly.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Thu Nov  2 17:31:35 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:31:35 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <14849.38631.997377.600214@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

    MZ> If MAL means dynamic scoping (which I understood he does),
    MZ> then this simply means:

    MZ> when looking for a variable "foo", you first search for it in
    MZ> the local namespace. If not there, the *caller's* namespace,
    MZ> and so on. In the end, the caller is the __main__ module, and
    MZ> if not found there, it is a NameError.

This is how Emacs Lisp behaves, and it's used all the time in ELisp
programs.  On the one hand it's quite convenient for customizing the
behavior of functions.  On the other hand, it can make documenting the
interface of functions quite difficult because all those dynamically
scoped variables are now part of the function's API.

It's interesting to note that many ELispers really hate dynamic
scoping and pine for a move toward lexical scoping.  I'm not one of
them.

I'm not as concerned about "fixing" nested functions because I hardly
ever use them, and rarely see them much in Python code.  Fixing
lambdas would be nice, but since Guido considers lambdas themselves a
mistake, and given that lamda use /can/ be a performance hit in some
situations, does it make sense to change something as fundamental as
Python's scoping rules to fix this eddy of the language?

-Barry



From jeremy at alum.mit.edu  Thu Nov  2 17:36:45 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:36:45 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <3A0190BA.940FFADA@lemburg.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
	<3A0190BA.940FFADA@lemburg.com>
Message-ID: <14849.38941.59576.682495@bitdiddle.concentric.net>

Moshe's explanation of "dynamic scope" is the definition I've seen in
every programming language text I've ever read.  The essence of the
defintion, I believe, is that a free variable is resolved in the
environment created by the current procedure call stack.

I think it muddles the discussion to use "dynamic scope" to describe
acquistion, though it is a dynamic feature.

Python using dynamic scope for exceptions.  If any exception is
raised, the exception handler that is triggered is determined by the
environment in which the procedure was called.

There are few languages that use dynamic scoping for normal name
resolution.  Many early Lisp implementations did, but I think all the
modern ones use lexical scoping instead.  It is hard to write modular
code using dynamic scope, because the behavior of a function with free
variables can not be determined by the module that defines it.  Not
saying it isn't useful, just that it makes it much harder to reason
about how a particular modular or function works in isolation from the
rest of the system.

Jeremy



From jeremy at alum.mit.edu  Thu Nov  2 17:38:04 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:38:04 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.39020.532855.494649@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

  MZ> you guys are talking about different things.  Jeremy is talking
  MZ> about a tool to warn against incompatible changes Greg is
  MZ> talking about a tool to identify, for each variable, what scope
  MZ> it belongs to.

Not sure we're talking about different things.  The compiler will need
to determine the scope of each variable.  It's a tool.  If it
implements the specifiction for name binding, other tools can too.

Jeremy



From barry at wooz.org  Thu Nov  2 17:36:11 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:36:11 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.38907.808752.186414@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

    MZ> you guys are talking about different things.  Jeremy is
    MZ> talking about a tool to warn against incompatible changes Greg
    MZ> is talking about a tool to identify, for each variable, what
    MZ> scope it belongs to.

And Greg's point is well taken, because it /will/ be harder to tell at
a glance where a name is coming from, so programming tools will have
to find ways to help with this.

-Barry



From barry at wooz.org  Thu Nov  2 17:44:06 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:44:06 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
	<20001101234134.O12812@xs4all.nl>
Message-ID: <14849.39382.960359.909365@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:

    >> Update this PEP to current, harsh, reality. It's been rejected
    >> :) If at all possible, the reasoning should be extended to
    >> include the real reasons it was rejected -- this is just
    >> guesswork from my side. (This means you, Guido, or anyone who
    >> can channel Guido enough to call himself Guido.)

    TW> In addition to that, PEP 0 also needs to be updated. Shall I
    TW> do that myself, now that Barry is apparently away ?

I've just done it.
    
    TW> While I was at it, I also noticed PEP 0200 still says
    TW> 'Incomplete', though that might be by design.

I've updated both these too, thanks.

-Barry



From mal at lemburg.com  Thu Nov  2 17:45:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:45:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
		<3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>
Message-ID: <3A019A18.20D12FE1@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> It may not look serious, but changing the Python lookup scheme
>   MAL> is, since many inspection tools rely and reimplement exactly
>   MAL> that scheme. With nested scopes, there would be next to no way
>   MAL> to emulate the lookups using these tools.
> 
> Can you say more about this issue?  It sounds like it is worth
> discussing in the PEP, but I can't get a handle on exactly what the
> problem is.  Any tool needs to implement or model Python's name
> resolution algorithm, call it algorithm A.  If we change name
> resolution to use algorithm B, then the tools need to implement or
> model a new algorithm.  I don't see where the impossibility of
> emulation comes in.

Well first you'd have to change all tools to use the new
scheme (this includes debuggers, inspection tools, reflection
kits, etc.). This certainly is not a smart thing to do since
Python IDEs are just starting to appear -- you wouldn't want
to break all those.

What get's harder with the nested scheme is that
you can no longer be certain that globals() reaches out to
the module namespace. But this is needed by some lazy evaluation
tools. Writing to globals() would not be defined anymore --
where should you bind the new variable ?

Another problem is that there probably won't be a way to access
all the different nesting levels on a per-level basis (could be
that I'm missing something here, but debugging tools would need
some sort of scope() builtin to access the different scopes).
I'm not sure whether this is possible to do without some sort
of link between the scopes. We currently don't need such links.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Thu Nov  2 17:52:50 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:52:50 -0500 (EST)
Subject: [Python-Dev] PEP-0217
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
	<3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <14849.39906.374949.489679@anthem.concentric.net>

>>>>> "FB" == Finn Bock <bckfnn at worldonline.dk> writes:

    FB> I don't see any problems with this. This is already handled by
    FB> a method in the jython runtime (Py.printResult).

Yep, should be easy to add to Jython.
-Barry



From mwh21 at cam.ac.uk  Thu Nov  2 18:00:37 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:00:37 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Wed, 01 Nov 2000 23:24:29 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com>
Message-ID: <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> > two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> > and NCALL_FUNC_KW.
> > 
> > NCALL_FUNC would pop a function object and a tuple off the stack and
> > apply the function to the tuple.
> > 
> > NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> > the moral equivalent of f(*args,**kw).
> 
> No, this is a bad idea.  Long, long ago, all calls requird building a
> tuple for the arguments first.  This tuple creation turned out to be a
> major bottleneck.  That's why the current call opcode exists.

Yes, I realize this now.  I made my suggestion having not actually
looked at the code or thought about it very much.

I still think there is (or at least might be) value in rewriting the
code for the more complex cases, and moving the dictionary creation
out of the implementation of CALL_FUNCTION.

Then the following could be made essentially equivalent:

f(a,b,c=d,e=f)

dict = {c:d,e:f}
f(a,b,**dict)

(modulo evaluation order).

(I've made some changes to get BUILD_MAP using it's argument and
construct literal dictionaries using it, which I'll whack up onto sf
just as soon as I get round to it... ah, it's at

https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102227&group_id=5470

).

Cheers,
M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp




From gstein at lyra.org  Thu Nov  2 18:07:29 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 09:07:29 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <14849.39382.960359.909365@anthem.concentric.net>; from barry@wooz.org on Thu, Nov 02, 2000 at 11:44:06AM -0500
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>
Message-ID: <20001102090729.A1874@lyra.org>

On Thu, Nov 02, 2000 at 11:44:06AM -0500, Barry A. Warsaw wrote:
> 
> >>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:
> 
>     >> Update this PEP to current, harsh, reality. It's been rejected
>     >> :) If at all possible, the reasoning should be extended to
>     >> include the real reasons it was rejected -- this is just
>     >> guesswork from my side. (This means you, Guido, or anyone who
>     >> can channel Guido enough to call himself Guido.)
> 
>     TW> In addition to that, PEP 0 also needs to be updated. Shall I
>     TW> do that myself, now that Barry is apparently away ?
> 
> I've just done it.

Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
Barry's administrative overload.

I mean, geez... this is what source control is about. Let a lot of people in
there, but be able to back up in case somebody totally goofs it.

This goes for adding new PEPs, too. I'm not as convinced here, since some
level of "good enough for a PEP" filtering is probably desirable, but then
again, it would seem that the people with commit access probably have that
filter in their head anyways.

Just a thought... how can we grease things up a bit more...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From pf at artcom-gmbh.de  Thu Nov  2 17:59:21 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 2 Nov 2000 17:59:21 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011021618.IAA15298@slayer.i.sourceforge.net> from Barry Warsaw at "Nov 2, 2000  8:18:27 am"
Message-ID: <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>

Barry Warsaw wrote:
> Update of /cvsroot/python/python/nondist/peps
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv15290
> 
> Added Files:
> 	pep-0227.txt 
> Log Message:
> PEP 227, Statically Nested Scopes, Jeremy Hylton
> 
> 
> ***** Error reading new file: (2, 'No such file or directory')

It was obviously not intended to be mailed out that way again.  
Problem with pathname and/or current directory?  Barry got this 
right once, now it is broken again.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Thu Nov  2 18:07:15 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 18:07:15 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
	            <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <3A019F43.E13604BA@lemburg.com>

Guido van Rossum wrote:
> 
> > It may not look serious, but changing the Python lookup scheme
> > is, since many inspection tools rely and reimplement exactly
> > that scheme. With nested scopes, there would be next to no
> > way to emulate the lookups using these tools.
> 
> So fix the tools.

Eek. Are you proposing to break all the Python IDE that are
just appearing out there ?

> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?

I think the common practice mainly comes from the fact,
that by making globals locals which can benefit from LOAD_FAST
you get a noticable performance boost.

So the "right" solution to these weird looking hacks would
be to come up with a smart way by which the Python compiler
itself can do the localizing.

Nested scopes won't help eliminating the current keyword 
practice.
 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

Hmm, so far the only argument for changing Python lookups
was to allow writing lambdas without keyword hacks. Does this
really warrant breaking code ?

What other advantages would statically nested scopes have ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Thu Nov  2 06:16:43 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:16:43 -0500
Subject: [Python-Dev] PEP-0217
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:49:37 +0200."
             <Pine.GSO.4.10.10011021648390.1290-100000@sundial> 
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial> 
Message-ID: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>

> 1) BDFL pronouncement

I believe Ping has also proposed such a display hook.  I'm not against
the idea, but I'm also not much in favor -- so I'm kind of +/- 0...

I've always thought that the interpreter mainloop should be rewritten
in Python.  That would be another PEP, and this would be a good place
to add a display hook as a feature.

Note that the example curreltly in the PEP has a fatal flaw: it's got
a recursive reference to print.  This is one of the things to consider
when proposing such a feature.

Sorry I can't be of more help...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 18:20:12 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:20:12 -0500 (EST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
	<200011020516.AAA08400@cj20424-a.reston1.va.home.com>
Message-ID: <14849.41548.308591.119778@bitdiddle.concentric.net>

I think the current draft of PEP 217 is far too thin and vague to be
acceptable.  You ought to re-read the PEP guidlines and make sure
you've covered all the points.  You must address at least:

- motivation
- specification of feature independent of implementation

Jeremy



From claird at starbase.neosoft.com  Thu Nov  2 18:18:20 2000
From: claird at starbase.neosoft.com (Cameron Laird)
Date: Thu, 2 Nov 2000 11:18:20 -0600 (CST)
Subject: [Python-Dev] Tk news you'll want to read
Message-ID: <200011021718.LAA70431@starbase.neosoft.com>

<URL:http://deja.com/=dnc/getdoc.xp?AN=688816476>

Note the clear intention to co-operate with Perl and Python,
the ambition to be considerably more portable and superior in
performance to GTK+, and so on.

I encourage you to e-mail this on to others it might interest.
A lot of the Perl people don't receive anything from me because
my ISP is ORBS-blacklisted (don't ask).



From guido at python.org  Thu Nov  2 06:29:27 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:29:27 -0500
Subject: [Python-Dev] CPyAN
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:42:39 +0100."
             <20001102164239.R12812@xs4all.nl> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>  
            <20001102164239.R12812@xs4all.nl> 
Message-ID: <200011020529.AAA08549@cj20424-a.reston1.va.home.com>

[Andrew]
> > * Work on something CPAN-like.  This may or may not have repercussions for
> >   the core; I don't know.

[Thomas]
> Probably not, though perhaps a new module would be nice. As for the
> CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
> Apachecon, and I think it would be suited extremely well as the transmission
> protocol for SPAM (or however you want to call the Python CPAN ;). You can
> do the uploading, downloading and searching for modules using WebDAV without
> too much pain, and there's excellent WebDAV support for Apache ;)
> 
> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

This is a nice thing to have, but I don't see why it should be tied to
the 2.1 effort.  Let's not couple projects that can be carried out
independently!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mwh21 at cam.ac.uk  Thu Nov  2 18:29:52 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:29:52 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Michael Hudson's message of "02 Nov 2000 17:00:37 +0000"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21 at cam.ac.uk> writes:

> Guido van Rossum <guido at python.org> writes:
> 
> Then the following could be made essentially equivalent:
> 
> f(a,b,c=d,e=f)
> 
> dict = {c:d,e:f}
> f(a,b,**dict)

... except when you have stuff like

f(a,b=c,**kw)

Gruntle.  Maybe Python function calls are just complicated!  (Has
anyone looked at bytecodehacks.xapply? - and that doesn't even handle
*-ed and **-ed arguments...).

Hmm - the interaction of "a=b" style args & *-ed args is a bit
counter-intuitive, particularly as the "a=b" args syntatically have to
come before the *-ed args:

>>> def f(a,b,c,d):
...  return a,b,c,d
... 
>>> f(1,c=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'c' redefined in call to f()
>>> f(1,b=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'b' redefined in call to f()
>>> f(1,d=4,*(2,3))
(1, 2, 3, 4)

I humbly submit that This Is Wrong.  I haven't seen anybody complain
about it, which suggests to me that noone is using this combination,
and I propose either:

1) banning it
2) demanding that the *-ed arg precede the "a=b" args

Of course, if noone is using this "feature", then maybe this dusty
little corner of the language should be left undisturbed.

And I haven't even thought about default arguments yet...

about-to-make-an-opcode-called-BODGE_KEYWORD_ARGUMENTS-ly y'rs
M.

-- 
  nonono,  while we're making wild  conjectures about the behavior 
  of completely  irrelevant tasks,  we must not  also make serious 
  mistakes, or the data might suddenly become statistically valid.
                                        -- Erik Naggum, comp.lang.lisp




From barry at wooz.org  Thu Nov  2 18:32:13 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:32:13 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
	<20001101234134.O12812@xs4all.nl>
	<14849.39382.960359.909365@anthem.concentric.net>
	<20001102090729.A1874@lyra.org>
Message-ID: <14849.42269.297360.615404@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

    GS> Shouldn't we allow other people to tweak PEP 0? It would
    GS> certainly lighten Barry's administrative overload.

I certainly don't mind at the very least, people modifying PEP 0 when
the status of their own peps change.

    GS> I mean, geez... this is what source control is about. Let a
    GS> lot of people in there, but be able to back up in case
    GS> somebody totally goofs it.

    GS> This goes for adding new PEPs, too. I'm not as convinced here,
    GS> since some level of "good enough for a PEP" filtering is
    GS> probably desirable, but then again, it would seem that the
    GS> people with commit access probably have that filter in their
    GS> head anyways.

    GS> Just a thought... how can we grease things up a bit more...

I do like to make a sanity pass through the text before approving it,
just to make sure we've got consistent format throughout the peps.
Also, I know we're all on "internet time" here <wink>, but one day
isn't too much time to let pass before taking action on things. :)
I'd also prefer it if there's /some/ limited editorial review before
these things get added.

That having been said, I'm very happy if someone wants to co-edit the
peps.  The pep 0 re-organization and slacker scolding would definitely
benefit from more than one watchdog.

Volunteers? :)

-Barry



From barry at wooz.org  Thu Nov  2 18:33:11 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:33:11 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
References: <200011021618.IAA15298@slayer.i.sourceforge.net>
	<m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>
Message-ID: <14849.42327.23020.553510@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf at artcom-gmbh.de> writes:

    PF> It was obviously not intended to be mailed out that way again.
    PF> Problem with pathname and/or current directory?  Barry got
    PF> this right once, now it is broken again.

Except that AFAIK, I didn't do anything to fix it, or to break it
again.

-Barry



From guido at python.org  Thu Nov  2 06:45:09 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:45:09 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:05:14 +0100."
             <3A0190BA.940FFADA@lemburg.com> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>  
            <3A0190BA.940FFADA@lemburg.com> 
Message-ID: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>

> > > [MAL]
> > > > Dynamic nested scopes is another topic... those are *very*
> > > > useful; especially when it comes to avoiding global variables
> > > > and implementing programs which work using control objects
> > > > instead of global function calls.
> > >
> > > Marc-Andre, what are Dynamic nested scopes?

[Moshe]
> > If MAL means dynamic scoping (which I understood he does), then this
> > simply means:
> > 
> > when looking for a variable "foo", you first search for it in the local
> > namespace. If not there, the *caller's* namespace, and so on. In the
> > end, the caller is the __main__ module, and if not found there, it is
> > a NameError.

Ah, yuck.  For variable namespace, this is a really bad idea.  For
certain other things (e.g. try/except blocks, Jeremy's example) it is
of course OK.

[MAL]
> That would be one application, yes.
> 
> With dynamic scoping I meant that the context of a lookup is
> defined at run-time and by explicitely or implicitely
> hooking together objects which then define the nesting.

But certainly you're not thinking of doing this to something as basic
to Python's semantics as local/global variable lookup!

> Environment acquisition is an example of such dynamic scoping:
> attribute lookups are passed on to the outer scope in case they
> don't resolve on the inner scope, e.g. say you have
> object a with a.x = 1; all other objects don't define .x.
> Then a.b.c.d.x will result in lookups
> 1. a.b.c.d.x
> 2. a.b.c.x
> 3. a.b.x
> 4. a.x -> 1

This seems only remotely related to dynamic scopes.  There are
namespaces here, but not "scopes" as I think of them: a scope defines
the validity for an unadorned variable.  Static scopes mean that this
is determined by the program source structure.  Dynamic scopes means
that that this is determined by run-time mechanisms.  Python uses
static scoping for local variables, but a dynamic scoping mechanism
for non-local variable references: first it does a module-global
lookup, then it looks for a built-in name.

When we're not talking about simple name lookup, we should speak of
namespaces (which can also have lifetimes).

> This example uses attribute lookup -- the same can be done for
> other nested objects by explicitely specifying the nesting
> relationship.
> 
> Jim's ExtensionClasses allow the above by using a lot of
> wrappers around objects -- would be nice if we could come
> up with a more general scheme which then also works for
> explicit nesting relationships (e.g. dictionaries which
> get hooked together -- Jim's MultiMapping does this).

I think we're getting way off track here.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 18:43:03 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 12:43:03 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDIHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.

I believe all early Lisps were dynamically scoped.  Scheme changed that, and
Common Lisp followed.  Many interpreted languages *start* life with dynamic
scoping because it's easy to hack together, but that never lasts.  REBOL
went thru this a couple years ago, switching entirely from dynamic to
lexical before its first public release, and breaking most existing programs
in the process.  Perl also started with dynamic scoping, but, in Perl-like
fashion, Perl5 *added* lexical scoping on top of dynamic ("local" vars use
dynamic scoping; "my" vars lexical; and all Perl5 gotcha guides stridently
recommend never using "local" anymore).  Here's some Perl5:

$i = 1;

sub f {
    local($i) = 2;
    &g();
}

sub g {
    return $i;
}

print "i at start is $i\n";
print "i in g called directly is ", &g(), "\n";
print "i in g called indirectly via f is ", &f(), "\n";
print "i at end is $i\n";

Here's what it prints:

i at start is 1
i in g called directly is 1
i in g called indirectly via f is 2
i at end is 1

> It is hard to write modular code using dynamic scope, because the
> behavior of a function with free variables can not be determined by
> the module that defines it.

As shown above; dynamic scoping is a nightmare even in the absence of nested
functions.

> Not saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

People who spend too much time writing meta-systems <wink> overestimate its
usefulness.  Most programmers who need this kind of effect would be much
better off explicitly passing a dict of name->value mappings explicitly
manipulated, or simply passing the values of interest (if I want a function
that sucks the value of "i" out of its caller, what clearer way than for the
caller to pass i in the arglist?!  dynamic scoping is much subtler, of
course -- it sucks the value of "i" out of *whatever* function up the call
chain happened to define an i "most recently").

Lexical scoping is much tamer, and, indeed, Python is one of the few modern
languages that doesn't support it.  Last time Guido and I hand-wrung over
this, that was the chief reason to implement it:  newcomers are endlessly
surprised that when they write functions that visually nest, their scopes
nevertheless don't nest.  There's certainly nothing novel or unexpected
about lexical scoping anymore.

The problem unique to Python is its rule for determining which vrbls are
local; in Lisps, REBOL and Perl, you have to explicitly name all vrbls local
to a given scope, which renders "how do you rebind a non-local name?" a
non-issue.

has-the-feel-of-inevitability-ly y'rs  - tim





From guido at python.org  Thu Nov  2 06:56:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:56:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:11:30 EST."
             <14849.37426.860007.989619@bitdiddle.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>  
            <14849.37426.860007.989619@bitdiddle.concentric.net> 
Message-ID: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>

[Jeremy and Tim argue about what to do about write access for
variables at intermediate levels of nesting, neither local nor
module-global.]

I'll risk doing a pronouncement, even though I know that Jeremy (and
maybe also Tim?) disagree.

You don't need "write access" (in the sense of being able to assign)
for variables at the intermediate scopes, so there is no need for a
syntax to express this.  Assignments are to local variables (normally)
or to module-globals (when 'global' is used).  Use references search
for a local, then for a local of the containing function definition,
then for a local in its container, and so forth, until it hits the
module globals, and then finally it looks for a builtin.

We can argue over which part of this is done statically and which part
is done dynamically: currently, locals are done dynamically and
everything else is done statically.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 18:57:09 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:57:09 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
	<3A0190BA.940FFADA@lemburg.com>
	<200011020545.AAA08743@cj20424-a.reston1.va.home.com>
Message-ID: <14849.43765.419161.105395@bitdiddle.concentric.net>

I don't think I buy your explanation that Python uses dynamic scope
for resolving globals.  As I understand the mechanism, the module
namespace and builtins namespace are those for the module in which the
function was defined.  If so, this is still static scope.

Here's a quick example that illustrates the difference:

module foo:
-----------------------
a = 12

def add(b):
    return a + b
-----------------------

module bar:
-----------------------
from foo import add

a = -1
print add(1)
-----------------------

If Python used static scope, "python bar.py" should print 13 (it
does).  If it used dynamic scope, I would expect the answer to be 0.

Jeremy



From fdrake at acm.org  Thu Nov  2 18:55:11 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 12:55:11 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14848.39586.832800.139182@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
Message-ID: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I just did a clean configure and make from the latest CVS tree.  It
 > seems to get stuck in a loop calling makesetup over and over again.

  Please do an update and run autoconf, then try again.
  I didn't check in the autoconf output; Guido thinks that we want to
stick with autoconf 2.13, and I've got 2.14.1.  The reason is that
there are enough differences in the m4 expansions that seeing the real
effects of the configure.in changes is hard, just because there are so
many changed lines which are not related to the actual change.
  (If you do have autoconf 2.13, please check in the new configure.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov  2 07:02:10 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:02:10 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:22:32 EST."
             <14849.38088.903104.936944@anthem.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net>  
            <14849.38088.903104.936944@anthem.concentric.net> 
Message-ID: <200011020602.BAA08974@cj20424-a.reston1.va.home.com>

> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
> 
>     d = globals().copy()
>     d.update(locals())
> 
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.  (PEP 42 now includes a feature request
> to make vars() do this by default.)

Note that I just deleted that feature request from PEP 42 -- vars() or
locals() returns the dictionary containing the variables, and you
can't just change the semantics to return a newly copied dictionary
(which could be quite expensive too!).

I don't think you need to have a mechanism to find all accessible
names; I don't see a common use for that.  It's sufficient to have a
mechanism to look up any specific name according to whatever mechanism
we decide upon.  This is needed for internal use of course; it can
also be useful for e.g. variable substitution mechanisms like the one
you recently proposed or Ping's Itmpl.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From jeremy at alum.mit.edu  Thu Nov  2 19:04:10 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:04:10 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44186.494602.297323@bitdiddle.concentric.net>

I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
placed three files in Modules: Makefile.pre, Setup, and Setup.config.

I ran "make" and it immediately reported an error:
"../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
Setup.local: No such file or directory"

Jeremy



From guido at python.org  Thu Nov  2 07:07:04 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:07:04 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:31:35 EST."
             <14849.38631.997377.600214@anthem.concentric.net> 
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021724340.1290-100000@sundial>  
            <14849.38631.997377.600214@anthem.concentric.net> 
Message-ID: <200011020607.BAA09013@cj20424-a.reston1.va.home.com>

[Barry, on dynamic scopes]
> This is how Emacs Lisp behaves, and it's used all the time in ELisp
> programs.  On the one hand it's quite convenient for customizing the
> behavior of functions.  On the other hand, it can make documenting the
> interface of functions quite difficult because all those dynamically
> scoped variables are now part of the function's API.
> 
> It's interesting to note that many ELispers really hate dynamic
> scoping and pine for a move toward lexical scoping.  I'm not one of
> them.

I don't care what you pine for in ELisp, but for Python this would be
a bad idea.

> I'm not as concerned about "fixing" nested functions because I hardly
> ever use them, and rarely see them much in Python code.  Fixing
> lambdas would be nice, but since Guido considers lambdas themselves a
> mistake, and given that lamda use /can/ be a performance hit in some
> situations, does it make sense to change something as fundamental as
> Python's scoping rules to fix this eddy of the language?

I referred to this in our group meeting as "fixing lambda" because
that's where others seem to need it most often.  But it is a real
problem that exists for all nested functions.  So let me rephrase
that: "fixing nested function definitions" is useful, if it can be
done without leaking memory, without breaking too much existing code,
and without slowing down code that doesn't use the feature.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 19:06:12 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:06:12 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
	<14849.37426.860007.989619@bitdiddle.concentric.net>
	<200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <14849.44308.333212.37111@bitdiddle.concentric.net>

Don't know if you saw the discussion in the PEP or not.  I made two
arguments for being able to assign to variables bound in enclosing
scopes.

1. Every other language that supports nested lexical scoping allows
   this.  To the extent that programmers have seen these other
   languages, they will expect it to work.

2. It is possible to work around this limitation by using containers.
   If you want to have an integer that can be updated by nested
   functions, you wrap the interger in a list and make all assignments
   and references refer to list[0].  It would be unfortunate if
   programmers used this style, because it is obscure.  I'd rather see
   the language provide a way to support this style of programming
   directly. 

Jeremy



From esr at thyrsus.com  Thu Nov  2 23:10:16 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:10:16 -0800
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>; from guido@python.org on Wed, Nov 01, 2000 at 10:29:03PM -0500
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <20001102141016.B1838@thyrsus.com>

Guido van Rossum <guido at python.org>:
> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?
> 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

For whatever it's worth, I agree with both these arguments.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"The power to tax involves the power to destroy;...the power to
destroy may defeat and render useless the power to create...."
	-- Chief Justice John Marshall, 1819.



From fdrake at acm.org  Thu Nov  2 19:04:34 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:04:34 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44186.494602.297323@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
Message-ID: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
 > placed three files in Modules: Makefile.pre, Setup, and Setup.config.
 > 
 > I ran "make" and it immediately reported an error:
 > "../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
 > Setup.local: No such file or directory"

  Very interesting!  The first thing I get is this:

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/fdrake/projects/python/temp/Modules'
echo "# Edit this file for local setup changes" >Setup.local
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
make[1]: Leaving directory `/home/fdrake/projects/python/temp/Modules'

  Can you capture stout & stderr from a clean configure & make and
mail it to me?
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov  2 07:10:57 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:10:57 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:36:45 EST."
             <14849.38941.59576.682495@bitdiddle.concentric.net> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com>  
            <14849.38941.59576.682495@bitdiddle.concentric.net> 
Message-ID: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>

> Moshe's explanation of "dynamic scope" is the definition I've seen in
> every programming language text I've ever read.  The essence of the
> defintion, I believe, is that a free variable is resolved in the
> environment created by the current procedure call stack.

Ah.  The term "free variable" makes sense here.

> I think it muddles the discussion to use "dynamic scope" to describe
> acquistion, though it is a dynamic feature.
> 
> Python using dynamic scope for exceptions.  If any exception is
> raised, the exception handler that is triggered is determined by the
> environment in which the procedure was called.

Then I think this is also muddles the discussion, since the look for
exception handlers has nothing to do with free variable lookup.

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

I think Python 3000 ought to use totally static scoping.  That will
make it possible to do optimize code using built-in names!

--Guido van Rossum (home page: http://www.python.org/~guido/)




From esr at thyrsus.com  Thu Nov  2 23:12:57 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:12:57 -0800
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <3A01910E.A19FFBB2@lemburg.com>; from mal@lemburg.com on Thu, Nov 02, 2000 at 05:06:38PM +0100
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <3A01910E.A19FFBB2@lemburg.com>
Message-ID: <20001102141257.C1838@thyrsus.com>

M.-A. Lemburg <mal at lemburg.com>:
> Most important for 2.1 are probably:
> 
> 1. new C level coercion scheme
> 2. rich comparisons
> 3. making the std lib Unicode compatible

I'd certainly like to see rich comparisons go in.  I have a "Set" class 
all ready for addition to the standard library except that it's waiting
on this feature in order to do partial ordering properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Every election is a sort of advance auction sale of stolen goods. 
	-- H.L. Mencken 



From esr at thyrsus.com  Thu Nov  2 23:14:24 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:14:24 -0800
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 11:36:45AM -0500
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <20001102141424.D1838@thyrsus.com>

Jeremy Hylton <jeremy at alum.mit.edu>:
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.

Correct.  Based on my LISP experience, I would be strongly opposed to
dymamic scoping.  That path has been tried and found wanting.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"Guard with jealous attention the public liberty.  Suspect every one
who approaches that jewel.  Unfortunately, nothing will preserve it
but downright force.  Whenever you give up that force, you are
inevitably ruined."
	-- Patrick Henry, speech of June 5 1788



From jeremy at alum.mit.edu  Thu Nov  2 19:14:08 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:14:08 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44784.479281.162333@bitdiddle.concentric.net>

Despite the error, I did get a successful build this time.  I guess
your configure change worked.

Jeremy



From jeremy at alum.mit.edu  Thu Nov  2 19:16:25 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:16:25 -0500 (EST)
Subject: [Python-Dev] Larry Wall talk on Perl 6
Message-ID: <14849.44921.765397.205628@bitdiddle.concentric.net>

http://dev.perl.org/~ask/als/

Interesting reading.

Jeremy



From fdrake at acm.org  Thu Nov  2 19:12:38 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:12:38 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44784.479281.162333@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
	<14849.44784.479281.162333@bitdiddle.concentric.net>
Message-ID: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Despite the error, I did get a successful build this time.  I guess
 > your configure change worked.

  I don't think the bug I fixed and the bug you reported were tightly
related.  I don't understand why you got the error after before or
after my change.
  I'f you're running autoconf 2.13, though, please do check in the new
configure script!
  (If anyone else is, you can check it in too!  The winner gets one
free documentation download!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From barry at wooz.org  Thu Nov  2 19:20:06 2000
From: barry at wooz.org (barry at wooz.org)
Date: Thu, 2 Nov 2000 13:20:06 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<14849.38088.903104.936944@anthem.concentric.net>
	<200011020602.BAA08974@cj20424-a.reston1.va.home.com>
Message-ID: <14849.45142.860806.358489@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    >> If we get lexical scoping, there should be a fast (built-in)
    >> way to get at all the accessible names from Python.
    >> I.e. currently I can do d = globals().copy() d.update(locals())
    >> and know that `d' contains a dictionary of available names,
    >> with the right overloading semantics.  (PEP 42 now includes a
    >> feature request to make vars() do this by default.)

    GvR> Note that I just deleted that feature request from PEP 42 --
    GvR> vars() or locals() returns the dictionary containing the
    GvR> variables, and you can't just change the semantics to return
    GvR> a newly copied dictionary (which could be quite expensive
    GvR> too!).

Saw that.  I was just thinking that locals() already does what
vars()-no-args does, so why have two ways to do the same thing?

    GvR> I don't think you need to have a mechanism to find all
    GvR> accessible names; I don't see a common use for that.  It's
    GvR> sufficient to have a mechanism to look up any specific name
    GvR> according to whatever mechanism we decide upon.  This is
    GvR> needed for internal use of course; it can also be useful for
    GvR> e.g. variable substitution mechanisms like the one you
    GvR> recently proposed or Ping's Itmpl.

Ah, something like this then:

-------------------- snip snip --------------------
import sys
from UserDict import UserDict

class NamesDict(UserDict):
    def __init__(self, frame):
        self.__frame = frame
        UserDict.__init__(self)

    def __getitem__(self, key):
        if self.data.has_key(key):
            return self.data[key]
        locals = self.__frame.f_locals
        if locals.has_key(key):
            return locals[key]
        globals = self.__frame.f_globals
        if globals.has_key(key):
            return globals[key]
        raise KeyError, key

def _(s):
    try: raise 'oops'
    except: frame = sys.exc_info()[2].tb_frame.f_back
    return s % NamesDict(frame)

theirs = 'theirs'
def give(mine, yours):
    print _('mine=%(mine)s, yours=%(yours)s, theirs=%(theirs)s')
-------------------- snip snip --------------------
Python 2.0 (#128, Oct 18 2000, 04:48:44) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import dict
>>> dict.give('mine', 'yours')
mine=mine, yours=yours, theirs=theirs
>>> 

-Barry



From guido at python.org  Thu Nov  2 07:28:26 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:28:26 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: Your message of "Thu, 02 Nov 2000 09:07:29 PST."
             <20001102090729.A1874@lyra.org> 
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>  
            <20001102090729.A1874@lyra.org> 
Message-ID: <200011020628.BAA09282@cj20424-a.reston1.va.home.com>

> Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
> Barry's administrative overload.
> 
> I mean, geez... this is what source control is about. Let a lot of people in
> there, but be able to back up in case somebody totally goofs it.

Agreed.

> This goes for adding new PEPs, too. I'm not as convinced here, since some
> level of "good enough for a PEP" filtering is probably desirable, but then
> again, it would seem that the people with commit access probably have that
> filter in their head anyways.

Here, common sense and good judgement should be applied.  If there
seems to be consensus that a PEP is needed, there's no need to wait
for Barry.  The update to PEP-0000 commits the assignment of the new
PEP number.  But the new PEP should follow all the rules for a new
PEP!

Having Barry in the loop makes sense for those who aren't sure they
can comply with all the rules, and for those outside the python-dev
community.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 19:29:41 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 13:29:41 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com>

[Guido]
> [Jeremy and Tim argue about what to do about write access for
> variables at intermediate levels of nesting, neither local nor
> module-global.]
>
> I'll risk doing a pronouncement, even though I know that Jeremy (and
> maybe also Tim?) disagree.
>
> You don't need "write access" (in the sense of being able to assign)
> for variables at the intermediate scopes, so there is no need for a
> syntax to express this.

I can live with that!  Reference-only access to intermediate scopes would
address 99% of current gripes.  Of course, future gripes will shift to that
there's no rebinding access.  If we have to support that someday too, my
preferred way of spelling it in Python requires explicit new syntax, so
adding that later would not break anything.

> Assignments are to local variables (normally) or to module-globals (when
> 'global' is used).  Use references search for a local, then for a local of
> the containing function definition, then for a local in its container,

The Pascal standard coined "closest-containing scope" to describe this
succinctly, and I recommend it for clarity and brevity.

> and so forth, until it hits the module globals, and then finally it looks
> for a builtin.
>
> We can argue over which part of this is done statically and which part
> is done dynamically: currently, locals are done dynamically and
> everything else is done statically.

I'm not sure what you're trying to say there, but to the extent that I think
I grasp it, I believe it's backwards:  locals are static today but
everything else is dynamic (and not in the sense of "dynamic scoping", but
in the operational sense of "requires runtime search to resolve non-local
names", while local names are fully resolved at compile-time today (but in
the absence of "exec" and "import *")).

what's-in-a-name?-a-rose-in-any-other-scope-may-not-smell-as-
     sweet-ly y'rs  - tim





From guido at python.org  Thu Nov  2 07:35:00 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:35:00 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 18:07:15 +0100."
             <3A019F43.E13604BA@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>  
            <3A019F43.E13604BA@lemburg.com> 
Message-ID: <200011020635.BAA09321@cj20424-a.reston1.va.home.com>

[MAL]
> > > It may not look serious, but changing the Python lookup scheme
> > > is, since many inspection tools rely and reimplement exactly
> > > that scheme. With nested scopes, there would be next to no
> > > way to emulate the lookups using these tools.

[GvR]
> > So fix the tools.

[MAL]
> Eek. Are you proposing to break all the Python IDE that are
> just appearing out there ?

Yes.  If a tool does variable scope analysis, it should be prepared
for changes in the rules.  Otherwise we might as well have refused the
syntax changes in 2.0 because they required changes to tools!

> > > To be honest, I don't think static nested scopes buy us all that
> > > much. You can do the same now, by using keyword arguments which
> > > isn't all that nice, but works great and makes the scope clearly
> > > visible.
> > 
> > Yes.  It's a hack that gets employed over and over.  And it has
> > certain problems.  We added 'import as' to get rid of a common
> > practice that was perceived unclean.  Maybe we should support nested
> > scopes to get rid of another unclean common practice?
> 
> I think the common practice mainly comes from the fact,
> that by making globals locals which can benefit from LOAD_FAST
> you get a noticable performance boost.
> 
> So the "right" solution to these weird looking hacks would
> be to come up with a smart way by which the Python compiler
> itself can do the localizing.

Can you elaborate?  I dun't understand what you are proposing here.

> Nested scopes won't help eliminating the current keyword 
> practice.

Why not?  I'd say that

    def create_adder(n):
        def adder(x, n=n): return x+n
	return adder

is a hack and that nested scopes can fix this by allowing you to write


    def create_adder(n):
        def adder(x): return x+n
	return adder
    
like one would expect.  (Don't tell me that it isn't a FAQ why this
doesn't work!)

> > I'm not saying that we definitely should add this to 2.1 (there's
> > enough on our plate already) but we should at least consider it, and
> > now that we have cycle GC, the major argument against it (that it
> > causes cycles) is gone...
> 
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?
> 
> What other advantages would statically nested scopes have ?

Doing what's proper.  Nested scopes are not a bad idea.  They weren't
implemented because they were hard to get right (perhaps impossible
without creating cycles), and I was okay with that because I didn't
like them; but I've been convinced that examples like the second
create_adder() above should reall work.  Just like float+int works
(in Python 0.1, this was a type-error -- you had to case the int arg
to a float to get a float result).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 07:40:19 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:40:19 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:45:12 +0100."
             <3A019A18.20D12FE1@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>  
            <3A019A18.20D12FE1@lemburg.com> 
Message-ID: <200011020640.BAA09370@cj20424-a.reston1.va.home.com>

> Well first you'd have to change all tools to use the new
> scheme (this includes debuggers, inspection tools, reflection
> kits, etc.). This certainly is not a smart thing to do since
> Python IDEs are just starting to appear -- you wouldn't want
> to break all those.

I've seen a Komodo demo.  Yes, it does this.  But it's soooooooo far
from being done that adding this wouldn't really slow them down much,
I think.  More likely, the toolmakers will have fun competing with
each other to be the first to support this! :-)

> What get's harder with the nested scheme is that
> you can no longer be certain that globals() reaches out to
> the module namespace. But this is needed by some lazy evaluation
> tools. Writing to globals() would not be defined anymore --
> where should you bind the new variable ?

I think globals() should return the module's __dict__, and the global
statement should cause the variable to reach directly into there.

We'll need to introduce a new builtin to do a name *lookup* in the
nested scopes.

> Another problem is that there probably won't be a way to access
> all the different nesting levels on a per-level basis (could be
> that I'm missing something here, but debugging tools would need
> some sort of scope() builtin to access the different scopes).
> I'm not sure whether this is possible to do without some sort
> of link between the scopes. We currently don't need such links.

Correct.  That link may have to be added to the frame object.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From fredrik at effbot.org  Thu Nov  2 19:56:40 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Thu, 2 Nov 2000 19:56:40 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>              <3A019F43.E13604BA@lemburg.com>  <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <000f01c044ff$47b72cb0$3c6340d5@hagrid>

Guido van Rossum wrote:> [MAL]
> > Eek. Are you proposing to break all the Python IDE that are
> > just appearing out there ?
> 
> Yes.  If a tool does variable scope analysis, it should be prepared
> for changes in the rules.

we can live with that...

</F>




From guido at python.org  Thu Nov  2 07:50:45 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:50:45 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "02 Nov 2000 17:29:52 GMT."
             <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>  
            <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011020650.BAA09505@cj20424-a.reston1.va.home.com>

> Hmm - the interaction of "a=b" style args & *-ed args is a bit
> counter-intuitive, particularly as the "a=b" args syntatically have to
> come before the *-ed args:
> 
> >>> def f(a,b,c,d):
> ...  return a,b,c,d
> ... 
> >>> f(1,c=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'c' redefined in call to f()
> >>> f(1,b=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'b' redefined in call to f()
> >>> f(1,d=4,*(2,3))
> (1, 2, 3, 4)
> 
> I humbly submit that This Is Wrong.  I haven't seen anybody complain
> about it, which suggests to me that noone is using this combination,
> and I propose either:
> 
> 1) banning it
> 2) demanding that the *-ed arg precede the "a=b" args

Interesting.  The *-ed arg cannot precede the a=b args currently, but
I agree that it would have made more sense to do it that way.

I'm not sure that it's worth allowing the combination of kw args and
*tuple, but I'l also not sure that it *isn't* worth it.

> Of course, if noone is using this "feature", then maybe this dusty
> little corner of the language should be left undisturbed.

I think it may be best to leave it alone.

> And I haven't even thought about default arguments yet...

That's unrelated.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Thu Nov  2 19:56:55 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Thu, 2 Nov 2000 12:56:55 -0600 (CST)
Subject: [Python-Dev] PythonLabs -> Digital Creations
Message-ID: <14849.47351.835291.365708@buffalo.fnal.gov>

So, when are the relevant websites going to be updated to reflect the
new reality?  Pythonlabs.com and BeOpen.com still reflect the old
regime...  I'd think the Digital Creations folks would be eager to
announce the changing of the guard...



From gstein at lyra.org  Thu Nov  2 19:57:07 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 10:57:07 -0800
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Thu, Nov 02, 2000 at 01:12:38PM -0500
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>
Message-ID: <20001102105707.H2037@lyra.org>

On Thu, Nov 02, 2000 at 01:12:38PM -0500, Fred L. Drake, Jr. wrote:
> 
> Jeremy Hylton writes:
>  > Despite the error, I did get a successful build this time.  I guess
>  > your configure change worked.
> 
>   I don't think the bug I fixed and the bug you reported were tightly
> related.  I don't understand why you got the error after before or
> after my change.
>   I'f you're running autoconf 2.13, though, please do check in the new
> configure script!

Wouldn't it make sense to do the upgrade to take advantages of the autoconf
fixes? If there are any problems with the change, then now is the time to do
it and to sort them out!

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From tim_one at email.msn.com  Thu Nov  2 20:07:18 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:07:18 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.37426.860007.989619@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>

[Jeremy]
> I agree that visual inspection is a tad harder, but I contend that
> existing programs that use the same name for a global variable and a
> local variable -- and intend for the global to be visible within a
> function nested in the local variable's region -- are confusing.
> It's too hard for a first-time reader of the code to figure out what
> is going on.
>
> Incidentally, I have yet to see an example of this problem occurring
> in anyone's code.  All the examples seem a bit contrived.  I wonder if
> anyone has an example in existing code.

I wasn't the one making the "will break code" argument (I'm sure it will,
but very little, and not at all for most people since most people never nest
functions in Python today apart from default-abusing lambdas).  Visual
inspection stands on its own as a potential problem.

>   [My SICP example omitted]
>
>   TP> Unfortunately for proponents, this is exactly the kind of SICP
>   TP> example that is much better done via a class.
>
> Indeed, the PEP says exactly that: This kind of program is better
> done via a class.  My intent was not to show a compelling use of
> mutable state.  Instead it was to show that with read-only access,
> people could still modify values on enclosing scopes.  The issue is
> whether the language allows the programmer to express this intent
> clearly or if she has to jump through some hoops to accomplish it.

Guido said "jump through some hoops", so I'm dropping it, but first noting
that "the container" in *idiomatic* Python will most often be "self":

    def yadda(self, ...):
        def whatever(amount):
            self.balance += amount
        return whatever

will work to rebind self.balance.  I don't think Guido will ever be
interested in supporting idiomatic Scheme.

>   TP>     def deposit(amount):
>   TP>         global bank_account.balance
>   TP>         balance += amount

> I'm still not sure I like it, because it mixes local variables of a
> function with attribute access on objects.  I'll add it to the
> discussion in the PEP (if Barry approves the PEP <wink>), though.

Actually, no connection to attribute access was intended there:  it was just
a backward-compatible way to spell the pair (name of containing scope, name
of vrbl in that scope).

    global back_account:balance
or
    global balance from bank_account

would do as well (and maybe better as they don't imply attribute access; but
maybe worse as they don't bring to mind attribute access <wink>).

> Do you have any opinion on the subtleties?  The two that immediately
> come to mind are: 1) whether the function's local are available as
> attributes anywhere or only in nested scopes

I didn't intend attribute-like access at all (although we had earlier talked
about that wrt JavaScript, I didn't have that in mind here).

> and 2) whether you can create new local variable using this notation.

Yes, that's the primary one I was thinking about.





From tim_one at email.msn.com  Thu Nov  2 20:10:52 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:10:52 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.38088.903104.936944@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>

[Barry A. Warsaw]
> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
>
>     d = globals().copy()
>     d.update(locals())
>
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.

It was long ago agreed (don't you love how I pull this stuff out of thin
historical air <wink>?) that if nested lexical scoping was added, we would
need also to supply a new mapping object that mimicked the full Python
lookup rules (including builtins).  Not necessarily a dictionary, though.





From guido at python.org  Thu Nov  2 08:15:50 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:15:50 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:33:11 EST."
             <14849.42327.23020.553510@anthem.concentric.net> 
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>  
            <14849.42327.23020.553510@anthem.concentric.net> 
Message-ID: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>

>     PF> It was obviously not intended to be mailed out that way again.
>     PF> Problem with pathname and/or current directory?  Barry got
>     PF> this right once, now it is broken again.
> 
> Except that AFAIK, I didn't do anything to fix it, or to break it
> again.

We have too much to do to try and fix this now.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 20:14:50 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:14:50 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A019F43.E13604BA@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEPHOAA.tim_one@email.msn.com>

[MAL]
> ...
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?

*Some* amount of code, sure.  Hard to quantify, but hard to believe there's
much code at risk.

> What other advantages would statically nested scopes have ?

Pythonic obviousness.  Virtually everyone coming to Python from other
languages *expects* visually nested functions to work this way.  That they
don't today is a very frequent source of surprises and complaints.





From barry at wooz.org  Thu Nov  2 20:31:37 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 14:31:37 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.38088.903104.936944@anthem.concentric.net>
	<LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>
Message-ID: <14849.49433.197319.973848@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim_one at email.msn.com> writes:

    TP> It was long ago agreed (don't you love how I pull this stuff
    TP> out of thin historical air <wink>?) that if nested lexical
    TP> scoping was added, we would need also to supply a new mapping
    TP> object that mimicked the full Python lookup rules (including
    TP> builtins).  Not necessarily a dictionary, though.

Oh yeah, I vaguely remember now <0.5 scratch-head>.

Works for me, although I'll point out that we needn't wait for lexical
scoping to provide such an object!

-Barry



From guido at python.org  Thu Nov  2 08:34:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:34:53 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:57:09 EST."
             <14849.43765.419161.105395@bitdiddle.concentric.net> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <200011020545.AAA08743@cj20424-a.reston1.va.home.com>  
            <14849.43765.419161.105395@bitdiddle.concentric.net> 
Message-ID: <200011020734.CAA09751@cj20424-a.reston1.va.home.com>

> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

That's not what I meant, but I expressed it clumsily.  Perhaps the
terminology is just inadequate.  I simply meant that builtins can be
overridden by module-globals.  But only in the module whose globals
are searched for globals -- that part is still static.

Let's drop this thread...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Thu Nov  2 20:31:59 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 14:31:59 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.49013.363302.823824@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
	<14849.45483.304866.556547@bitdiddle.concentric.net>
	<14849.45714.207143.196945@cj42289-a.reston1.va.home.com>
	<14849.47007.862290.364694@bitdiddle.concentric.net>
	<14849.48464.183713.819894@cj42289-a.reston1.va.home.com>
	<14849.49013.363302.823824@bitdiddle.concentric.net>
Message-ID: <14849.49455.97877.379601@cj42289-a.reston1.va.home.com>

[We've uncovered the root of Jeremy's problem here -- this is a
 problem/limitation of make with VPATH.]

Jeremy Hylton writes:
 > There is a Setup.local in the source directory.

  That's why it isn't being built.  Since it's on the VPATH, make
thinks it already exists in a usable form.  Since the command that
uses it doesn't look for the "most local" version but assumes it
exists in the right place, the command fails.
  This is a problem that exists with VPATH; the built files cannot
already exist in the source directories (hence, the build directories
can't refer to source dirs which have been used as build dirs; "make
clobber" isn't good enough for the Setup* files).
  Remove (or move) the existing file, and it should work fine.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From tim_one at email.msn.com  Thu Nov  2 20:41:12 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:41:12 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: <14849.47351.835291.365708@buffalo.fnal.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFBHOAA.tim_one@email.msn.com>

[Charles G Waldman]
> So, when are the relevant websites going to be updated to reflect the
> new reality?

Sorry, we have no idea.

> Pythonlabs.com and BeOpen.com still reflect the old regime...

And they're running on BeOpen.com machines, which we can no longer fiddle
with.  The people remaining at BeOpen.com probably don't view updating those
sites as a priority anymore (you all saw how long it took them to get
pythonlabs.com and Starship merely running again, although at the time we
couldn't tell you *why* we weren't able to fix them ourselves).

> I'd think the Digital Creations folks would be eager to announce the
> changing of the guard...

They're getting the word out in their own way.  Hope you saw the cover story
on LWN today!

    http://www.lwn.net/

There's a lot of good info there, including useful interviews with Paul
Everitt and Guido.





From mwh21 at cam.ac.uk  Thu Nov  2 20:45:47 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 19:45:47 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Thu, 02 Nov 2000 01:50:45 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk> <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> <200011020650.BAA09505@cj20424-a.reston1.va.home.com>
Message-ID: <m3vgu6duok.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > Hmm - the interaction of "a=b" style args & *-ed args is a bit
> > counter-intuitive, particularly as the "a=b" args syntatically have to
> > come before the *-ed args:
> > 
> > >>> def f(a,b,c,d):
> > ...  return a,b,c,d
> > ... 
> > >>> f(1,c=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'c' redefined in call to f()
> > >>> f(1,b=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'b' redefined in call to f()
> > >>> f(1,d=4,*(2,3))
> > (1, 2, 3, 4)
> > 
> > I humbly submit that This Is Wrong.  I haven't seen anybody complain
> > about it, which suggests to me that noone is using this combination,
> > and I propose either:
> > 
> > 1) banning it
> > 2) demanding that the *-ed arg precede the "a=b" args
> 
> Interesting.  The *-ed arg cannot precede the a=b args currently, but
> I agree that it would have made more sense to do it that way.
> 
> I'm not sure that it's worth allowing the combination of kw args and
> *tuple, but I'l also not sure that it *isn't* worth it.
> 
> > Of course, if noone is using this "feature", then maybe this dusty
> > little corner of the language should be left undisturbed.
> 
> I think it may be best to leave it alone.

Oh, I mostly agree and am not going to put any effort into this area -
but if I or someone else manages to make striking simplifications to
the function call code that has the side effect of banning/changing
the syntax of this combination, then this probably wouldn't be a good
enough reason for its rejection.

> > And I haven't even thought about default arguments yet...
> 
> That's unrelated.

I know, it was just something that occurred to me as adding more
complexity.  Hypothetical excercise: Write a description of Python's
current function call behaviour in sufficient detail to allow
reimplementation of it without playing with an existing interpreter in
less than 1000 words.  It's just hairy, and it amazes me that it feels
so natural most of the time...

Cheers,
M.

-- 
  Not only does the English Language borrow words from other
  languages, it sometimes chases them down dark alleys, hits 
  them over the head, and goes through their pockets.   -- Eddy Peters




From guido at python.org  Thu Nov  2 08:51:15 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:51:15 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:20:06 EST."
             <14849.45142.860806.358489@anthem.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <14849.38088.903104.936944@anthem.concentric.net> <200011020602.BAA08974@cj20424-a.reston1.va.home.com>  
            <14849.45142.860806.358489@anthem.concentric.net> 
Message-ID: <200011020751.CAA12539@cj20424-a.reston1.va.home.com>

[Barry]
> Saw that.  I was just thinking that locals() already does what
> vars()-no-args does, so why have two ways to do the same thing?

Not clear -- maybe one of them needs to be made obsolete.

>     GvR> I don't think you need to have a mechanism to find all
>     GvR> accessible names; I don't see a common use for that.  It's
>     GvR> sufficient to have a mechanism to look up any specific name
>     GvR> according to whatever mechanism we decide upon.  This is
>     GvR> needed for internal use of course; it can also be useful for
>     GvR> e.g. variable substitution mechanisms like the one you
>     GvR> recently proposed or Ping's Itmpl.
> 
> Ah, something like this then:
[Example deleted]

I'm not sure that the mechanism provided should follow the mapping
API.  *All* it needs to do it provide lookup capability.  Having
.keys(), .items(), .has_key() etc. just slows it down.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 20:54:05 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:54:05 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.43765.419161.105395@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFCHOAA.tim_one@email.msn.com>

[Jeremy]
> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

It's dynamic in the shallow (but real!) sense that it can't be determined to
which of {module scope, builtin scope} a global name resolves today until
runtime.  Indeed, in

def f():
    print len

the resolving scope for "len" may even change with each invocation of f(),
depending on who's playing games with the containing module's __dict__.
That's not "dynamic scoping" in the proper sense of the term, but it's sure
dynamic!

words-lead-to-more-words-so-become-one-with-the-essence-instead<wink>-ly
    y'rs  - tim





From guido at python.org  Thu Nov  2 08:59:47 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:29:41 EST."
             <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com> 
Message-ID: <200011020759.CAA12600@cj20424-a.reston1.va.home.com>

> [Guido]
> > [Jeremy and Tim argue about what to do about write access for
> > variables at intermediate levels of nesting, neither local nor
> > module-global.]
> >
> > I'll risk doing a pronouncement, even though I know that Jeremy (and
> > maybe also Tim?) disagree.
> >
> > You don't need "write access" (in the sense of being able to assign)
> > for variables at the intermediate scopes, so there is no need for a
> > syntax to express this.

[Tim]
> I can live with that!  Reference-only access to intermediate scopes would
> address 99% of current gripes.  Of course, future gripes will shift to that
> there's no rebinding access.  If we have to support that someday too, my
> preferred way of spelling it in Python requires explicit new syntax, so
> adding that later would not break anything.

Exactly my point.

> > Assignments are to local variables (normally) or to module-globals (when
> > 'global' is used).  Use references search for a local, then for a local of
> > the containing function definition, then for a local in its container,
> 
> The Pascal standard coined "closest-containing scope" to describe this
> succinctly, and I recommend it for clarity and brevity.

Thanks, that's a good term.  And the principle is totally
uncontroversial.

> > and so forth, until it hits the module globals, and then finally it looks
> > for a builtin.
> >
> > We can argue over which part of this is done statically and which part
> > is done dynamically: currently, locals are done dynamically and
> > everything else is done statically.
> 
> I'm not sure what you're trying to say there, but to the extent that I think
> I grasp it, I believe it's backwards:  locals are static today but
> everything else is dynamic (and not in the sense of "dynamic scoping", but
> in the operational sense of "requires runtime search to resolve non-local
> names", while local names are fully resolved at compile-time today (but in
> the absence of "exec" and "import *")).

Oops, yes, I had it backwards.

As I said elsewhere, in Python 3000 I'd like to do it all more
statically.  So perhaps we should look up nested locals based on
static information too.  Thus:

  x = "global-x"

  def foo():
      if 0:
          x = "x-in-foo"
      def bar():
	  return x
      return bar

  print foo()()

should raise UnboundLocalError, not print "global-x".

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 08:59:55 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:55 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:06:12 EST."
             <14849.44308.333212.37111@bitdiddle.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com> <14849.37426.860007.989619@bitdiddle.concentric.net> <200011020556.AAA08847@cj20424-a.reston1.va.home.com>  
            <14849.44308.333212.37111@bitdiddle.concentric.net> 
Message-ID: <200011020759.CAA12609@cj20424-a.reston1.va.home.com>

> Don't know if you saw the discussion in the PEP or not.

Sorry, I had no time.  I have read it now, but it doesn't change my
point of view.

> I made two
> arguments for being able to assign to variables bound in enclosing
> scopes.
> 
> 1. Every other language that supports nested lexical scoping allows
>    this.  To the extent that programmers have seen these other
>    languages, they will expect it to work.

But we have a unique way of declaring variables, which makes the
issues different.

Your PEP wonders why I am against allowing assignment to intermediate
levels.  Here's my answer: all the syntaxes that have been proposed
to spell this have problems.  So let's not provide a way to spell it.
I predict that it won't be a problem.  If it becomes a problem, we can
add a way to spell it later.  I expect that the mechanism that will be
used to find variables at intermediate levels can also be used to set
them, so it won't affect that part of the implementation much.

> 2. It is possible to work around this limitation by using containers.
>    If you want to have an integer that can be updated by nested
>    functions, you wrap the interger in a list and make all assignments
>    and references refer to list[0].  It would be unfortunate if
>    programmers used this style, because it is obscure.  I'd rather see
>    the language provide a way to support this style of programming
>    directly. 

I don't expect that programmers will use this style.  When they have
this need, they will more likely use a class.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 09:08:14 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:08:14 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:56:55 CST."
             <14849.47351.835291.365708@buffalo.fnal.gov> 
References: <14849.47351.835291.365708@buffalo.fnal.gov> 
Message-ID: <200011020808.DAA16162@cj20424-a.reston1.va.home.com>

> So, when are the relevant websites going to be updated to reflect the
> new reality?  Pythonlabs.com and BeOpen.com still reflect the old
> regime...  I'd think the Digital Creations folks would be eager to
> announce the changing of the guard...

While technically I have the capability to update the pythonlabs.com
website, ethically I feel I cannot do this.  The site belongs to
BeOpen.  I'll ask them to make some kind of announcement.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 09:16:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:16:53 -0500
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:57:07 PST."
             <20001102105707.H2037@lyra.org> 
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>  
            <20001102105707.H2037@lyra.org> 
Message-ID: <200011020816.DAA16227@cj20424-a.reston1.va.home.com>

> Wouldn't it make sense to do the upgrade to take advantages of the autoconf
> fixes? If there are any problems with the change, then now is the time to do
> it and to sort them out!

Yeah, but I have too much to do to get to this any time soon -- it
just isn't important enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From skip at mojam.com  Thu Nov  2 22:19:11 2000
From: skip at mojam.com (Skip Montanaro)
Date: Thu, 2 Nov 2000 15:19:11 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
Message-ID: <14849.55887.283432.505910@beluga.mojam.com>

>>>>> "Tim" == Tim Peters <tim_one at email.msn.com> writes:

    Tim> [Jeremy]
    >> All the examples seem a bit contrived.  I wonder if anyone has an
    >> example in existing code.

    Tim> I wasn't the one making the "will break code" argument ...

Nor was I.  Jeremy (I think) asked MAL how it could break code.  I posted a
(simple, but obviously contrived) example.  I have no particular opinion on
this subject.  I was just trying to answer Jeremy's question.

Skip



From jeremy at alum.mit.edu  Thu Nov  2 21:25:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 15:25:36 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.55887.283432.505910@beluga.mojam.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
	<14849.55887.283432.505910@beluga.mojam.com>
Message-ID: <14849.52672.943439.978785@bitdiddle.concentric.net>

Looks like we need to rehash this thread at least enough to determine
who is responsible for causing us to rehash it.

MAL said it would break code.  I asked how.  Skip and Tim obliged with
examples.  I said their examples exhibited bad style; neither of them
claimed they were good style.

In the end, I observed that while it could break code in theory, I
doubted it really would break much code.  Furthermore, I believe that
the code it will break is already obscure so we needn't worry about
it.

Jeremy



From tim_one at email.msn.com  Thu Nov  2 22:13:47 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 16:13:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFKHOAA.tim_one@email.msn.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
>
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.

Ah!  I wasn't trying to give an example of code that would break, but, ya,
now that you mention it, it would.  I was just giving an example of why
visual inspection will be harder in Python than in Pascal.

I expect that the kind of code I showed *will* be common: putting all the
nested "helper functions" at the top of a function, just as was also done in
Pascal.  The killer difference is that in Pascal, the containing function's
locals referenced by the nested helpers can be found declared at the top of
the containing function; but in Python you'll have to search all over the
place, and if you don't find a non-local var at once, you'll be left uneasy,
wondering whether you missed a binding target, or whether the var is truly
global, or what.





From guido at python.org  Thu Nov  2 10:41:01 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 04:41:01 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 15:25:36 EST."
             <14849.52672.943439.978785@bitdiddle.concentric.net> 
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com>  
            <14849.52672.943439.978785@bitdiddle.concentric.net> 
Message-ID: <200011020941.EAA18905@cj20424-a.reston1.va.home.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's quite enough rehashing.  I don't think we'll have to worry
about breakiung much code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Thu Nov  2 22:47:35 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 22:47:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 02:15:50AM -0500
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de> <14849.42327.23020.553510@anthem.concentric.net> <200011020715.CAA09616@cj20424-a.reston1.va.home.com>
Message-ID: <20001102224734.U12812@xs4all.nl>

On Thu, Nov 02, 2000 at 02:15:50AM -0500, Guido van Rossum wrote:

> >     PF> It was obviously not intended to be mailed out that way again.
> >     PF> Problem with pathname and/or current directory?  Barry got
> >     PF> this right once, now it is broken again.

> We have too much to do to try and fix this now.

But I haven't ;) I've made syncmail give some more information if it can't
find the file, to compensate for the fact that cvs.python.sourceforge.net
runs a Python before 1.5.2, and thus doesn't show the file it couldn't find.
I won't bother with creating files just for the hell of it, so we'll just
wait until Barry triggers it again.

(Not that I'm bored (not much, anyway), but this was a no-brainer.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From MarkH at ActiveState.com  Thu Nov  2 22:55:07 2000
From: MarkH at ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:55:07 +1100
Subject: [Python-Dev] please subscribe me ;-)
Message-ID: <LCEPIIGDJPKCOIHOBJEPAENNCFAA.MarkH@ActiveState.com>

Hi all,
	My skippinet.com.au address is temporarily bouncing, and thus I am no
longer getting python-dev mail.  I re-subscribed on Monday, but am still
waiting moderator approval.

Can someone approve me please?

Thanks,

Mark.




From MarkH at ActiveState.com  Thu Nov  2 22:57:58 2000
From: MarkH at ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:57:58 +1100
Subject: [Python-Dev] Dont bother subscribing me!
Message-ID: <LCEPIIGDJPKCOIHOBJEPEENNCFAA.MarkH@ActiveState.com>

All done already - sorry about the noise.

Thanks,

Mark.



From guido at python.org  Thu Nov  2 11:06:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 05:06:16 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:34:42 EST."
             <20001102103442.B5027@kronos.cnri.reston.va.us> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>  
            <20001102103442.B5027@kronos.cnri.reston.va.us> 
Message-ID: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>

[Andrew]
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.

I just skimmed PEP 222.  I agree that the classes defined by cgi.py
are unnecessarily arcane.  I wonder if it isn't better to just start
over rather than trying to add yet another new class to the already
top-heavy CGI module???

Regarding file uploads: you *seem* to be proposing that uploaded files
should be loaded into memory only.  I've got complaints from people
who are uploading 10 Mb files and don't appreciate their process
growing by that much.  Perhaps there should be more options, so that
the caller can control the disposition of uploads more carefully?
What's wrong with subclassing?  Maybe two example subclasses should be
provided?

Regarding templating -- what's wrong with HTMLgen as a starting point?
Just that it's too big?  I've never used it myself, but I've always
been impressed with its appearance. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Thu Nov  2 23:12:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 17:12:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 05:06:16AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>
Message-ID: <20001102171226.A21364@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:06:16AM -0500, Guido van Rossum wrote:
>I just skimmed PEP 222.  I agree that the classes defined by cgi.py
>are unnecessarily arcane.  I wonder if it isn't better to just start
>over rather than trying to add yet another new class to the already
>top-heavy CGI module???

I've wondered about that, too; writing a neat request class that wraps
up field values, cookies, and environment variables, and provides
convenience functions for re-creating the current URL, Something like
the request classes in Zope and Webware.

>Regarding file uploads: you *seem* to be proposing that uploaded files
>should be loaded into memory only.  I've got complaints from people

Mrr...?  The only file upload reference simply says you shouldn't have
to subclass in order to use them; it's not implying files have to be
read into memory.  (We have to deal with whackingly large mask layout
files at work, after all.)

>Regarding templating -- what's wrong with HTMLgen as a starting point?
>Just that it's too big?  I've never used it myself, but I've always
>been impressed with its appearance. :-)

I, personally, am against including templating, but it was suggested.
I'm against it because there are too many solutions with different
tradeoffs.  Do you want a simple regex search-and-replace,
constructing HTML pages as Python objects, or a full-blown
minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
Python-compatible syntax?  Much better just to move templating into
the "Rejected" category and give the above rationale.

--amk




From jeremy at alum.mit.edu  Thu Nov  2 23:07:37 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 17:07:37 -0500 (EST)
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <20001102171226.A21364@kronos.cnri.reston.va.us>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011021727341.1290-100000@sundial>
	<20001102103442.B5027@kronos.cnri.reston.va.us>
	<200011021006.FAA19152@cj20424-a.reston1.va.home.com>
	<20001102171226.A21364@kronos.cnri.reston.va.us>
Message-ID: <14849.58793.132399.810370@bitdiddle.concentric.net>

Since today has been busy on the meta-sig, I wonder if we should
create a web-sig to thrash out these issues.

Jeremy



From thomas at xs4all.net  Thu Nov  2 23:51:52 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 23:51:52 +0100
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: <200011020500.VAA11872@sf-web2.i.sourceforge.net>; from noreply@sourceforge.net on Wed, Nov 01, 2000 at 09:00:43PM -0800
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>
Message-ID: <20001102235152.N12776@xs4all.nl>

On Wed, Nov 01, 2000 at 09:00:43PM -0800, noreply at sourceforge.net wrote:

> Comment:
> Accepted and assigned back to Thomas.

> Guido approved of this "in theory" before, so go for it!

Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
prefix, etc) and I was about to check it in, when I got cold feet. This
change is going to break programs embedding or extending python, that rely
on getopt() (the C function) being available, one way or another. After this
move, that is no longer necessarily true. I got cold feet because of
Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
optind and optarg, and doesn't bother with #include <getopt.h> or some such.

Before, that would probably always work, depending on which library/archive
was searched first, for the getopt() function. It would only break if
system-libraries were searched first, for some reason, and the system
libraries provide a broken getopt. In all other cases, like there not being
a getopt, getopt needing seperate includes or defines, or separate link or
compile arguments, the Python getopt would be used, and it would do what was
expected (probably, anyway ;)

After the change, people need to use the system getopt(), and do their own
hoop-jumping to find out if it's there, how to call it, etc. Now I myself
have never written anything that would depend on Python providing getopt(),
but that doesn't say anything at all. Is this acceptable breakage ? Or
should we go the whole nine yards, make it an official API, document it, and
provide backwards-compatible symbols on platforms where getopt used to be
used ?

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Thu Nov  2 14:41:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:41:53 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:12:26 EST."
             <20001102171226.A21364@kronos.cnri.reston.va.us> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>  
            <20001102171226.A21364@kronos.cnri.reston.va.us> 
Message-ID: <200011021341.IAA19469@cj20424-a.reston1.va.home.com>

> >Regarding file uploads: you *seem* to be proposing that uploaded files
> >should be loaded into memory only.  I've got complaints from people
> 
> Mrr...?  The only file upload reference simply says you shouldn't have
> to subclass in order to use them; it's not implying files have to be
> read into memory.  (We have to deal with whackingly large mask layout
> files at work, after all.)

Mrrauw...?  Do you really have to subclass?  I thought it just says
that you can subclass if you're not happy with the given make_file()
implementation?

> >Regarding templating -- what's wrong with HTMLgen as a starting point?
> >Just that it's too big?  I've never used it myself, but I've always
> >been impressed with its appearance. :-)
> 
> I, personally, am against including templating, but it was suggested.
> I'm against it because there are too many solutions with different
> tradeoffs.  Do you want a simple regex search-and-replace,
> constructing HTML pages as Python objects, or a full-blown
> minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
> Python-compatible syntax?  Much better just to move templating into
> the "Rejected" category and give the above rationale.

Sure -- I'm perfectly happy with ad-hoc templating solutions myself
(see my FAQ wizard).  I've also heard Tom Christiansen complain that
Perl is slower to start up than Python for CGI work -- because the
templating classes are so big, and are all loaded at startup!

I do see a use for a helper or helpers to creates tables though --
tables are notoriously tag-intensive and hard to get right.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 14:51:14 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:51:14 -0500
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: Your message of "Thu, 02 Nov 2000 23:51:52 +0100."
             <20001102235152.N12776@xs4all.nl> 
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>  
            <20001102235152.N12776@xs4all.nl> 
Message-ID: <200011021351.IAA19544@cj20424-a.reston1.va.home.com>

> Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
> prefix, etc) and I was about to check it in, when I got cold feet. This
> change is going to break programs embedding or extending python, that rely
> on getopt() (the C function) being available, one way or another. After this
> move, that is no longer necessarily true. I got cold feet because of
> Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
> optind and optarg, and doesn't bother with #include <getopt.h> or some such.
> 
> Before, that would probably always work, depending on which library/archive
> was searched first, for the getopt() function. It would only break if
> system-libraries were searched first, for some reason, and the system
> libraries provide a broken getopt. In all other cases, like there not being
> a getopt, getopt needing seperate includes or defines, or separate link or
> compile arguments, the Python getopt would be used, and it would do what was
> expected (probably, anyway ;)
> 
> After the change, people need to use the system getopt(), and do their own
> hoop-jumping to find out if it's there, how to call it, etc. Now I myself
> have never written anything that would depend on Python providing getopt(),
> but that doesn't say anything at all. Is this acceptable breakage ? Or
> should we go the whole nine yards, make it an official API, document it, and
> provide backwards-compatible symbols on platforms where getopt used to be
> used ?

Don't worry -- getopt has never been something offered by Python.
It's something assumed to be in the system library, and if it isn't we
provide our own -- but we don't provide it for the benefit of those
who embed us.  (In fact, if Python is embedded, it will not *use*
getopt.)  We can't even guarantee to provide it, because it's likely
already in the system library.  (That's why I like that the prefix
begins with an underscore -- those are not provided for use by others,
only for internal use.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Fri Nov  3 04:57:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 22:57:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <14849.58793.132399.810370@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 05:07:37PM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com> <20001102171226.A21364@kronos.cnri.reston.va.us> <14849.58793.132399.810370@bitdiddle.concentric.net>
Message-ID: <20001102225726.A21509@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:07:37PM -0500, Jeremy Hylton wrote:
>Since today has been busy on the meta-sig, I wonder if we should
>create a web-sig to thrash out these issues.

There's already a python-web-modules at egroups.com list, for authors 
of Python Web frameworks; fairly on-topic for that list, I should think.

--amk



From moshez at math.huji.ac.il  Fri Nov  3 11:18:26 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Fri, 3 Nov 2000 12:18:26 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>

On Thu, 2 Nov 2000, Guido van Rossum wrote:

> I think Python 3000 ought to use totally static scoping.  That will
> make it possible to do optimize code using built-in names!

Isn't that another way of saying you want the builtin names to be
part of the language definition? Part of today's method advantages
is that new builtins can be added without any problems.

no-clear-win-either-way-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Fri Nov  3 11:02:13 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:02:13 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
		<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
		<14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <3A028D25.B6A227F2@lemburg.com>

Jeremy Hylton wrote:
> 
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's just what I was trying to say all along: statically
nested scopes don't buy you anything except maybe for lambdas
and nested functions (which is bad style programming, IMHO too).

The only true argument for changing scoping I see is that
of gained purity in language design... without much practical
use.  

Other issues that need sorting out:

x = 2
class C:
  x = 1
  C = 'some string'
  def a(self):
     print x
  def b(self):
     global x
     x = 3

class D(C):
  C = 'some string'
  def a(self):
     C.a(self)
     print C

o = C()
o.a()
o.b()
o.a()

o = D()
o.a()

What would the output look like under your proposal ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 11:46:59 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:46:59 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
Message-ID: <3A0297A3.A778F4FC@lemburg.com>

Moshe Zadka wrote:
> 
> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!
> 
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

+1.

Wouldn't it be more Python-like to provide the compiler with
a set of known-to-be-static global name bindings ?

A simple way of avoiding optimizations like these:

def f(x, str=str):
   return str(x) + '!'

would then be to have the compiler lookup "str" in the globals()
passed to it and assign the found value to the constants of the
function, provided that "str" appears in the list of
known-to-be-static global name bindings (perhaps as optional
addition parameter to compile() with some reasonable default
in sys.staticsymbols).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 11:58:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:58:10 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>  
	            <3A019F43.E13604BA@lemburg.com> <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <3A029A42.3C11C774@lemburg.com>

Guido van Rossum wrote:
> 
> > > > To be honest, I don't think static nested scopes buy us all that
> > > > much. You can do the same now, by using keyword arguments which
> > > > isn't all that nice, but works great and makes the scope clearly
> > > > visible.
> > >
> > > Yes.  It's a hack that gets employed over and over.  And it has
> > > certain problems.  We added 'import as' to get rid of a common
> > > practice that was perceived unclean.  Maybe we should support nested
> > > scopes to get rid of another unclean common practice?
> >
> > I think the common practice mainly comes from the fact,
> > that by making globals locals which can benefit from LOAD_FAST
> > you get a noticable performance boost.
> >
> > So the "right" solution to these weird looking hacks would
> > be to come up with a smart way by which the Python compiler
> > itself can do the localizing.
> 
> Can you elaborate?  I dun't understand what you are proposing here.

See my other post... I would like to have the compiler do the
localization for me in case it sees a global which has been
"defined" static.

> > Nested scopes won't help eliminating the current keyword
> > practice.
> 
> Why not?  I'd say that
> 
>     def create_adder(n):
>         def adder(x, n=n): return x+n
>         return adder
> 
> is a hack and that nested scopes can fix this by allowing you to write
> 
>     def create_adder(n):
>         def adder(x): return x+n
>         return adder
> 
> like one would expect.  (Don't tell me that it isn't a FAQ why this
> doesn't work!)

I know... still, I consider function definitions within a
function bad style. Maybe just me, though ;-)
 
> > > I'm not saying that we definitely should add this to 2.1 (there's
> > > enough on our plate already) but we should at least consider it, and
> > > now that we have cycle GC, the major argument against it (that it
> > > causes cycles) is gone...
> >
> > Hmm, so far the only argument for changing Python lookups
> > was to allow writing lambdas without keyword hacks. Does this
> > really warrant breaking code ?
> >
> > What other advantages would statically nested scopes have ?
> 
> Doing what's proper.  Nested scopes are not a bad idea.  They weren't
> implemented because they were hard to get right (perhaps impossible
> without creating cycles), and I was okay with that because I didn't
> like them; but I've been convinced that examples like the second
> create_adder() above should reall work.  Just like float+int works
> (in Python 0.1, this was a type-error -- you had to case the int arg
> to a float to get a float result).

Ok, but how does nested scoping mix with class definitions
and global lookups then ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 13:50:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 13:50:17 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
			<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
			<14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net> <3A028D25.B6A227F2@lemburg.com>
Message-ID: <3A02B489.89EF108C@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Jeremy Hylton wrote:
> >
> > Looks like we need to rehash this thread at least enough to determine
> > who is responsible for causing us to rehash it.
> >
> > MAL said it would break code.  I asked how.  Skip and Tim obliged with
> > examples.  I said their examples exhibited bad style; neither of them
> > claimed they were good style.
> >
> > In the end, I observed that while it could break code in theory, I
> > doubted it really would break much code.  Furthermore, I believe that
> > the code it will break is already obscure so we needn't worry about
> > it.
> 
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions (which is bad style programming, IMHO too).
> 
> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.
> 
> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

[Moshe pointed out to me in private mail that the above would
continue to work as it does now due to a difference being made
between class and function scoping categories]

More questions:

How are you going to explain the different
scoping categories to a newbie ? What if you define a class 
within a method ? How can you explicitely attach a dynamically
defined class to a certain scope ?

More problems (?!):

Nested scopes will introduce cycles in all frame objects.
This means that with GC turned off, frame objects will live
forever -- Python will eat up memory at a very fast pace.

BTW, Python's GC only works for a few builtin types (frames
are not among the supported types): what if a frame finds its
way into a user defined type ? Will GC still be able to cleanup
the cycles ?

Perhaps I'm just being silly, but I still don't see the
benefits of breaking todays easy-to-grasp three level
scoping rules... 

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Fri Nov  3 14:05:08 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:05:08 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 12:18:26 +0200."
             <Pine.GSO.4.10.10011031216280.8123-100000@sundial> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> 
Message-ID: <200011031305.IAA22103@cj20424-a.reston1.va.home.com>

> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!

[Moshe]
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

The built-in names have always been part of the language definition in
my mind.  The way they are implemented doesn't reflect this, but
that's just an implementation detail.  How would you like it if
something claimed to be Python but didn't support len()?  Or map()?

That doesn't mean you can't add new built-ins, and I don't think that
the new implementation will prevent that -- but it *will* assume that
you don't mess with the definitions of the existing built-ins.

Of course you still will be able to define functions whose name
overrides a built-in -- in that case the compiler can see that you're
doing that (because it knows the scope rules and can see what you are
doing).  But you won't be able to confuse someone else's module by
secretly sticking a replacement built-in into their module's __dict__.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Fri Nov  3 14:12:11 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:12:11 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 11:02:13 +0100."
             <3A028D25.B6A227F2@lemburg.com> 
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>  
            <3A028D25.B6A227F2@lemburg.com> 
Message-ID: <200011031312.IAA22141@cj20424-a.reston1.va.home.com>

[MAL]
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions

That's a tautology -- that's what nested scopes are FOR!

> (which is bad style programming, IMHO too).

Not always.  The keyword argument hack is so common that it must serve
a purpose, and that's what we're trying to fix -- for lambda's *and*
nested functions (which are semantically equivalent anyway).

> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.

But there is practical use: get rid of the unintuitive, unobvious and
fragile keyword argument hack.

> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

This is a good point!  If we considered the class as a nested scope
here, I think it might break too much code, plus it would allow a new
coding style where you could reference class variables without a self
or <classname> prefix.  I don't like that prospect, so I'm in favor
for ruling this out.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Fri Nov  3 15:16:20 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 09:16:20 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
Message-ID: <14850.51380.7327.977719@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> The built-in names have always been part of the language
    GvR> definition in my mind.  The way they are implemented doesn't
    GvR> reflect this, but that's just an implementation detail.  How
    GvR> would you like it if something claimed to be Python but
    GvR> didn't support len()?  Or map()?

    GvR> That doesn't mean you can't add new built-ins, and I don't
    GvR> think that the new implementation will prevent that -- but it
    GvR> *will* assume that you don't mess with the definitions of the
    GvR> existing built-ins.

    GvR> Of course you still will be able to define functions whose
    GvR> name overrides a built-in -- in that case the compiler can
    GvR> see that you're doing that (because it knows the scope rules
    GvR> and can see what you are doing).  But you won't be able to
    GvR> confuse someone else's module by secretly sticking a
    GvR> replacement built-in into their module's __dict__.

I'm a little confused.  I've occasionally done the following within an
application:

----------driver.py
# need to override built-in open() to do extra debugging
def debuggin_open(filename, mode, bufsize):
    # ...
if EXTRA_DEBUGGING:
    import __builtin__.__dict__['open'] = debugging_open
-------------------- snip snip --------------------

Would this be illegal?  Would other modules in my application (even if
imported from the standard library!) automatically get
debugging_open() for open() like they do now?

-Barry



From gward at mems-exchange.org  Fri Nov  3 15:33:03 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 09:33:03 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sat, Oct 28, 2000 at 12:51:20PM +0200
References: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>
Message-ID: <20001103093303.A2683@ludwig.cnri.reston.va.us>

On 28 October 2000, Martin v. Loewis said:
> > I don't have access to a Solaris machine, so I can't do anything to
> > help these users.
> 
> The patch in 117606 looks right to me: gcc on Solaris (and on any
> other platform) needs -shared to build shared library; configure
> currently passes -G. I haven't actually tried the patch, since it is a
> pain to extract it from the SF bug report page. What happens is that
> gcc passes -G to the linker, which correctly produces a shared
> library.  However, gcc also links crt1/crti into the library, which
> causes the reference to main.

Well, I do have access to a Solaris machine -- I try not to use it if I
don't have to, and about the only purpose it has these days is
occasionally building Python to make sure it still works.

Incidentally, I'm the one who changed "ld -G" to "$(CC) -G" -- see
revision 1.124 of configure.in:

  revision 1.124
  date: 2000/05/26 12:22:54;  author: gward;  state: Exp;  lines: +6 -2
  When building on Solaris and the compiler is GCC, use '$(CC) -G' to
  create shared extensions rather than 'ld -G'.  This ensures that shared
  extensions link against libgcc.a, in case there are any functions in the
  GCC runtime not already in the Python core.

I think the checkin message there is fairly clear; the context was in
using Robin Dunn's extension for BSDDB 2.x, which does some 64-bit
arithmetic deep down inside.  Turned out that GCC compiled a 64-bit
divide into a function call, and that function is in GCC's own runtime
library.  Using "ld -G" -- that's Sun's linker, which knows nothing
about GCC's runtime library -- the function in question wasn't
available, so loading the extension failed.  I assume that if *Python*
did a 64-bit divide somewhere in *its* guts, that function would have
been available (linked into the python binary), which is why this
problem doesn't come up very often -- Python probably does use most of
GCC's runtime.  ;-)

Anyways, I thought the patch in bug #117606 looked fine, so I tried it
out.  Not so good; here's what happens when I try to build
arraymodule.so (the first extension, alphabetically) with "gcc -shared":

  Text relocation remains                         referenced
      against symbol                  offset      in file
  _PyObject_NewVar                    0x654       arraymodule.o
  <unknown>                           0x26cc      arraymodule.o
  <unknown>                           0x26c8      arraymodule.o
  [...many many <unknown> symbols...]
  PyErr_Occurred                      0x1274      arraymodule.o
  PyErr_Occurred                      0x4a0       arraymodule.o
  PyErr_Occurred                      0x22d8      arraymodule.o
  PyErr_Occurred                      0x115c      arraymodule.o
  PyErr_Occurred                      0x368       arraymodule.o
  PyErr_Occurred                      0x1074      arraymodule.o
  PyErr_Occurred                      0x1f50      arraymodule.o
  PyInt_FromLong                      0x3f4       arraymodule.o
  [...]
  _Py_NoneStruct                      0x19d4      arraymodule.o
  Py_InitModule4                      0x26b8      arraymodule.o
  ld: fatal: relocations remain against allocatable but non-writable sections
  collect2: ld returned 1 exit status

All told, there were 500+ relocation errors in that one extension.
About half where "Py" or "_Py" symbols; a bunch were "<unknown>", and
the rest were 'malloc', 'memcpy', 'sprintf', and other standard library
functions.  So apparently "-shared" tells GCC to forget everything it
knows about linking C code and be as stupid as it can.  Hmmm.

I tried adding "../libpython2.0.a", and "-L.. -lpython2.0", and instead
got 20,000 relocation errors, since of course libpython2.0.a needs a lot
more symbols than arraymodule.o.  Hmmm.

I have no idea what's going on here.  I've updated the bug report, and I
am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless, of
course, there are other linker options that make it work right...

        Greg



From gmcm at hypernet.com  Fri Nov  3 15:48:29 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 09:48:29 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14850.51380.7327.977719@anthem.concentric.net>
Message-ID: <3A0289ED.6247.EB4DD0E@localhost>

> 
> >>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

>     GvR> Of course you still will be able to define functions whose
>     GvR> name overrides a built-in -- in that case the compiler can
>     GvR> see that you're doing that (because it knows the scope rules
>     GvR> and can see what you are doing).  But you won't be able to
>     GvR> confuse someone else's module by secretly sticking a
>     GvR> replacement built-in into their module's __dict__.

[A Craven Dog overrides builtin open...]

> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

I have always found it very elegant & powerful that keyword 
"import" calls builtin __import__, and that well-behaved C goes 
through the same hook. In a similar vein, I have for quite 
awhile wanted to experiment with mountable virtual file 
systems so that I can "mount" a (for example) MetaKit 
database at /cheesewhiz and other modules in my app, when 
navigating into /cheesewhiz will, unbeknownst to them, be 
reading from the MetaKit DB (these ideas leaked from tcl-land 
by Jean-Claude).

I most definitely appreciate that these facilities are dangerous 
and this type of stuff tends to be abused gratuitously (eg, 
most import hacks), but disallowing them might be considered 
a gratuitous limitation (heck - Barry knows how to bypass the 
governor on every cheezewhizzer ever manufactured).

- Gordon



From guido at python.org  Fri Nov  3 16:36:22 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 10:36:22 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 09:16:20 EST."
             <14850.51380.7327.977719@anthem.concentric.net> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com>  
            <14850.51380.7327.977719@anthem.concentric.net> 
Message-ID: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>

[Barry]
> I'm a little confused.  I've occasionally done the following within an
> application:
> 
> ----------driver.py
> # need to override built-in open() to do extra debugging
> def debuggin_open(filename, mode, bufsize):
>     # ...
> if EXTRA_DEBUGGING:
>     import __builtin__.__dict__['open'] = debugging_open
> -------------------- snip snip --------------------
> 
> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

That's up for discussion.  Note that the open() function is special in
this respect -- I don't see you doing the same to range() or hash().

If this is deemed a useful feature (for open()), we can make a rule
about which built-ins you cannot override like this and which ones you
can.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Fri Nov  3 16:45:05 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 10:45:05 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
	<14850.51380.7327.977719@anthem.concentric.net>
	<200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14850.56705.120643.110016@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> That's up for discussion.  Note that the open() function is
    GvR> special in this respect -- I don't see you doing the same to
    GvR> range() or hash().

Me neither, but special casing overridability seems like a fragile
hack.

    GvR> If this is deemed a useful feature (for open()), we can make
    GvR> a rule about which built-ins you cannot override like this
    GvR> and which ones you can.

Hmm, maybe we need __open__() and an open-hook? ;)

-Barry



From guido at python.org  Fri Nov  3 17:07:13 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 11:07:13 -0500
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
Message-ID: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>

At our PythonLabs group meeting last Tuesday we decided that we needed
to set up a Patches page and a FAQ for Python 2.0.  (This because we
don't see a reason yet to issue a bugfix release, but we do need to
answer some common questions and point people to some patches.)

I figured that we could do ourselves a favor by making this a set of
dynamic pages maintained wiki-style, rather than static pages
(especially since I've managed to personally become the bottleneck for
editing the static pages, until the move of python.org to a DC hosted
machine is complete :-).

So, when I saw the announcement of MoinMoin, a Wiki clone written in
Python as a single CGI script, I decided to try it -- instead of
ZWiki, which might be the obvious choice given my new employer.  This
is not because I don't like Zope or ZWiki, but because we can learn
from using different implementations of the same idea.

So, I humbly present the Python 2.0 Info Area: 

    http://www.python.org/cgi-bin/moinmoin

I've added one critical patch, three non-critical (misc) patches,
answers to two frequent bug reports to the FAQ page, and a bunch of
links to the front page.

Note that to generate patches, I use SourceForge cvsweb's diff
feature.  The URLs are ugly, but only the page editors see these, and
it saves having to store copies of the patches.

I'd like to get some feedback from the python-dev group before I link
it into the 2.0 home page.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Fri Nov  3 17:12:04 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:12:04 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 10:36:22AM -0500
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <20001103171203.G28658@xs4all.nl>

On Fri, Nov 03, 2000 at 10:36:22AM -0500, Guido van Rossum wrote:
> [Barry]

> > Would this [replacing builtins] be illegal?

> That's up for discussion.  Note that the open() function is special in
> this respect -- I don't see you doing the same to range() or hash().

Eep. I don't care much about scoping, so I'm not going to say much about
that, but I certainly do care about Python's flexibility. One of the great
things is that there are so little special cases, that (nearly) everything
is delightfully consistent. Being able to override open() but not hash() or
range() sounds directly contrary to that, at least to me. Being able to
change __builtins__ *just like any other dict* strikes me as terribly
Pythonic, though I realize this is probably contrary to Guido's view of
Python :-) It also makes for instance 'exec' and 'eval' terribly obvious.

I understand where the wish to restrict replacement and shadowing of
builtins comes from, but I'm hoping here that this is going to be
'optional', like Perl's -w and 'use strict'. Defaulting to maximum
simplicity and maximum flexibility (in that order:) but with optional
warnings (when shadowing/modifying a builtin) and optimizations (using
constants for builtins, when not modifying or shadowing them, for instance.)

Just-my-fl.0,04-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Fri Nov  3 17:31:07 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:31:07 +0100
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 11:07:13AM -0500
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <20001103173107.H28658@xs4all.nl>

On Fri, Nov 03, 2000 at 11:07:13AM -0500, Guido van Rossum wrote:

> So, I humbly present the Python 2.0 Info Area: 

>     http://www.python.org/cgi-bin/moinmoin

> I've added one critical patch, three non-critical (misc) patches,

Looks good. I'd never seen a Wiki thing before, and the naming requirements
takes a bit getting used to, but I think it looks great, regardless :)

It also reminds me that we still need to fix the erroneous conclusion by
configure that BSDI has large file support, just because it has an off_t
type that is 64 bit. (It does have that, it just doesn't use that in any of
the seek/tell functions available.) Trent, you were making noises about
looking at it, when I left for ApacheCon. Did you ever get to look at it ?
If not, I'll see if I can figure it out ;P Once it's fixed, I think it
should be added to CriticalPatches, but it might not be as straightforward
as pointing to a cvsweb URL ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From loewis at informatik.hu-berlin.de  Fri Nov  3 19:51:49 2000
From: loewis at informatik.hu-berlin.de (Martin von Loewis)
Date: Fri, 3 Nov 2000 19:51:49 +0100 (MET)
Subject: [Python-Dev] build problems under Solaris
Message-ID: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>

>  Text relocation remains                         referenced
>      against symbol                  offset      in file
[...]

> I have no idea what's going on here.  I've updated the bug report,
> and I am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless,
> of course, there are other linker options that make it work right...

That happens only when using the system linker (/usr/ccs/bin/ld). The
GNU linker won't complain, and the resulting executables will run
fine.

To make the system linker happy, you also need to compile the modules
with -fPIC - which, according to Sun, we should have done all the time
when building shared libraries. The linker complains about relocations
in the text segment, which shouldn't be there - shared libraries
should be position independent.

Alternatively, you can use 'gcc -shared -mimpure-text'; that does not
request that remaining relocations cause errors.

Regards,
Martin



From mal at lemburg.com  Fri Nov  3 20:08:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 20:08:17 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
Message-ID: <3A030D21.413B8C8A@lemburg.com>

I'm having trouble opening ZIP files created using InfoZIP's
zip utility (which uses zlib) with zipfile.py:

>>> x = z.read('README')
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "/home/lemburg/lib/zipfile.py", line 242, in read
    bytes = dc.decompress(bytes)
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Is this due to the installed zlib on my system being incompatible,
or is this a bug in zipfile.py ? I have libz version 1.1.3 and
zip version 2.2.

Also, I wonder why zipfile forces the mode flag to be 'r',
'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
to the mode flag instead ?!

The ZipFile is also missing some kind of method which
extracts files in the ZIP archive to a file-like object. This
would be very useful for extracting large files from a ZIP
archive without having to first read in the whole file into
memory.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From skip at mojam.com  Fri Nov  3 22:04:44 2000
From: skip at mojam.com (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:04:44 -0600 (CST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
	<14850.51380.7327.977719@anthem.concentric.net>
	<200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14851.10348.715861.488845@beluga.mojam.com>

    Guido> If this is deemed a useful feature (for open()), we can make a
    Guido> rule about which built-ins you cannot override like this and
    Guido> which ones you can.

I thought we were all adults...

For Py3k I think it should be sufficient to define the semantics of the
builtin functions so that if people want to override them they can, but that
overriding them in incompatible ways is likely to create some problems.
(They might have to run with a "no optimize" flag to keep the compiler from
assuming semantics, for instance.)  I see no particular reason to remove the
current behavior unless there are clear instances where something important
is not going to work properly.

Modifying builtins seems to me to be akin to linking a C program with a
different version of malloc.  As long as the semantics of the new functions
remain the same as the definition, everyone's happy.  You can have malloc
leave a logfile behind or keep histograms of allocation sizes.  If someone
links in a malloc library that only returns a pointer to a region that's
only half the requested size though, you're likely to run into problems.

Skip




From skip at mojam.com  Fri Nov  3 22:17:12 2000
From: skip at mojam.com (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:17:12 -0600 (CST)
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <14851.11096.337480.434853@beluga.mojam.com>

    Guido> I'd like to get some feedback from the python-dev group before I
    Guido> link it into the 2.0 home page.

Looks good to me.

Skip



From mal at lemburg.com  Fri Nov  3 21:14:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:14:17 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
		<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
		<14850.51380.7327.977719@anthem.concentric.net>
		<200011031536.KAA22620@cj20424-a.reston1.va.home.com> <14851.10348.715861.488845@beluga.mojam.com>
Message-ID: <3A031C99.7F7DD8E2@lemburg.com>

[Making builtins static...]

What about the idea to have the compiler make the decision
whether a global symbol may be considered static based on a 
dictionary ?

In non-optimizing mode this dictionary would be empty, with -O
it would include all builtins which should never be overloaded
and with -OO even ones which can be overloaded such as open()
in addition to some standard modules which are known to only
contain static symbols.

Perhaps this needs some additional help of the "define" statement
we planned as dynamic compiler interface ?! ...

define static_symbols = *            # all globals in this module
define static_symbols = func1, func2 # only these two
etc.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Fri Nov  3 21:31:51 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 15:31:51 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:04:44 CST."
             <14851.10348.715861.488845@beluga.mojam.com> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>  
            <14851.10348.715861.488845@beluga.mojam.com> 
Message-ID: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>

>     Guido> If this is deemed a useful feature (for open()), we can make a
>     Guido> rule about which built-ins you cannot override like this and
>     Guido> which ones you can.

[Skip]
> I thought we were all adults...

And consenting as well... :-)

> For Py3k I think it should be sufficient to define the semantics of the
> builtin functions so that if people want to override them they can, but that
> overriding them in incompatible ways is likely to create some problems.
> (They might have to run with a "no optimize" flag to keep the compiler from
> assuming semantics, for instance.)  I see no particular reason to remove the
> current behavior unless there are clear instances where something important
> is not going to work properly.
> 
> Modifying builtins seems to me to be akin to linking a C program with a
> different version of malloc.  As long as the semantics of the new functions
> remain the same as the definition, everyone's happy.  You can have malloc
> leave a logfile behind or keep histograms of allocation sizes.  If someone
> links in a malloc library that only returns a pointer to a region that's
> only half the requested size though, you're likely to run into problems.

Actually, the C standard specifically says you are *not* allowed to
override standard library functions like malloc().

I'm thinking of the example of the rules in Fortran for intrinsic
functions (Fortran's name for built-ins).  Based on what Tim has told
me, I believe that Fortran by default assumes that you're not doing
anything funky with intrinsics (like sin, cos, tan) it can use a
shortcut, e.g. inline them.  But there are also real functions by
these names in the Fortran standard library, and you can call those by
declaring e.g. "external function sin".  (There may also be an
explicit way to say that you're happy with the intrinsic one.)  I
believe that when you use the external variant, they may be overridden
by the user.

I'm thinking of something similar here for Python.  If the bytecode
compiler knows that the builtins are vanilla, it can generate better
(== more efficient) code for e.g.

   for i in range(10):
       ...

Ditto for expressions like len(x) -- the len() operation is typically
so fast that the cost is dominated by the two dict lookup operations
(first in globals(), then in __builtins__).

Why am I interested in this?  People interested in speed routinely use
hacks that copy a built-in function into a local variable so that they
don't have dictlookups in their inner loop; it's really silly to have
to do this, and if certain built-ins were recognized by the compiler
it wouldn't be necessary.  There are other cases where this is not so
easy without much more analysis; but the built-ins (to me) seem
low-hanging fruit.  (Search the archives for that term, I've used it
before in this context.)

I assume that it's *really* unlikely that there are people patching
the __builtin__ module to replace the functions that are good inline
candidates (range, len, id, hash and so on).  So I'm interesting in
complicating the rules here.  I'd be happy to make an explicit list of
those builtins that should not be messed with, as part of the language
definition.  Program that *do* mess with these have undefined
semantics.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Fri Nov  3 21:50:48 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:50:48 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test/output
 test_unicodedata,1.3,1.4
References: <200011032024.MAA26242@slayer.i.sourceforge.net>
Message-ID: <3A032528.D743FC0A@lemburg.com>

Fredrik Lundh wrote:
> 
> Update of /cvsroot/python/python/dist/src/Lib/test/output
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv25791/lib/test/output
> 
> Modified Files:
>         test_unicodedata
> Log Message:
> 
> Added 38,642 missing characters to the Unicode database (first-last
> ranges) -- but thanks to the 2.0 compression scheme, this doesn't add
> a single byte to the resulting binaries (!)
> 
> Closes bug #117524

Cool :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gmcm at hypernet.com  Fri Nov  3 21:56:56 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 15:56:56 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 03 Nov 2000 15:04:44 CST."             <14851.10348.715861.488845@beluga.mojam.com> 
Message-ID: <3A02E048.17812.10062D77@localhost>

> >     Guido> If this is deemed a useful feature (for open()), we can make a
> >     Guido> rule about which built-ins you cannot override like this and
> >     Guido> which ones you can.

I think I would be satisfied with just those builtins which involve 
interfaces to the external world. Where Java allows such 
deviance, they tend to provide an API whereby you can supply 
or register a factory to override or extend the default behavior.

In principle this seems less hackish than stomping on builtins; 
in practice I doubt it makes much difference ;-).
 
> [Skip]
> > I thought we were all adults...

Snort. One of my kids will be voting on Tuesday, but I *still* 
don't know what I want to be when I grow up.

- Gordon



From trentm at ActiveState.com  Fri Nov  3 21:51:41 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 3 Nov 2000 12:51:41 -0800
Subject: 64-bit stuff on BSDI (was: Re: [Python-Dev] New Wiki-based Python 2.0 Info Area)
In-Reply-To: <20001103173107.H28658@xs4all.nl>; from thomas@xs4all.net on Fri, Nov 03, 2000 at 05:31:07PM +0100
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com> <20001103173107.H28658@xs4all.nl>
Message-ID: <20001103125141.I20329@ActiveState.com>

On Fri, Nov 03, 2000 at 05:31:07PM +0100, Thomas Wouters wrote:
> It also reminds me that we still need to fix the erroneous conclusion by
> configure that BSDI has large file support, just because it has an off_t
> type that is 64 bit. (It does have that, it just doesn't use that in any of
> the seek/tell functions available.) Trent, you were making noises about
> looking at it, when I left for ApacheCon. Did you ever get to look at it ?

No. I was in Washington D.C. for the past week at SD 2000 (and getting
occassionally verbally abused by Tim about 64-bit stuff).

> If not, I'll see if I can figure it out ;P 

I can answer questions about fixing it but you can test it, so better for you
to submit the patch.

> Once it's fixed, I think it
> should be added to CriticalPatches, but it might not be as straightforward
> as pointing to a cvsweb URL ;P
>

Sure. Yes, it will probably be more than one file (configure.in, configure,
and fileobject.c).  Whatever, I don't think it should be a problem to put up
the actually checkin patch rather than a link.

Thanks!
Trent


-- 
Trent Mick
TrentM at ActiveState.com



From gward at mems-exchange.org  Fri Nov  3 23:45:00 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 17:45:00 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>; from loewis@informatik.hu-berlin.de on Fri, Nov 03, 2000 at 07:51:49PM +0100
References: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>
Message-ID: <20001103174500.B6683@ludwig.cnri.reston.va.us>

On 03 November 2000, Martin von Loewis said:
> That happens only when using the system linker (/usr/ccs/bin/ld). The
> GNU linker won't complain, and the resulting executables will run
> fine.

I'm not sure which linker my GCC on Solaris is using.  (Even though,
umm, I built that GCC.  Errr...)  But ISTR that the choice of linker is
made when you build GCC, so it's not really an option here.  Bad enough
to tell people that they need to reinstall Python because they need some
extension; requiring them to rebuild GCC -- !

But it's not necessary at all, because...

> To make the system linker happy, you also need to compile the modules
> with -fPIC - which, according to Sun, we should have done all the time
> when building shared libraries. The linker complains about relocations
> in the text segment, which shouldn't be there - shared libraries
> should be position independent.

...compiling everything with -fPIC is exactly what the doctor ordered.
I added "-fPIC" to OPT in the Makefile and rebuilt, and everything went
smoothly when linking the extensions with "gcc -shared".  No problems
compiling or linking, and the tests are running right now without a
hitch.  Yaayyyyh!

So here's an update of the patch: this changes LDSHARED to "$(CC)
-shared" when using GCC under Solaris, and it adds "-fPIC" to OPT when
using GCC *anywhere*.  This seems like a good thing to me when building
shared objects, but if anyone is aware of a platform where "gcc
... -fPIC" is a bad idea, speak up now!

*** configure.in	2000/11/03 08:18:36	1.177
--- configure.in	2000/11/03 22:43:50
***************
*** 308,315 ****
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes";;
  		esac
  		;;
  	*)	OPT="-O";;
--- 308,315 ----
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes -fPIC";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes -fPIC";;
  		esac
  		;;
  	*)	OPT="-O";;
***************
*** 564,570 ****
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -G'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;
--- 564,570 ----
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -shared'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;


I'll go update the bug report now.  Thanks, Martin!

        Greg



From guido at python.org  Sat Nov  4 00:01:27 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 18:01:27 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
Message-ID: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>

Moshe and Andrew (the impatient youth :-) have asked what's on our
plate for Python 2.1.


Schedule
--------

The current plan for the 2.1 release is in PEP 226:

    http://python.sourceforge.net/peps/pep-0226.html

According to that PEP, the tentative schedule is:

    16-Dec-2000: 2.1 PEPs ready for review
    01-Feb-2001: First 2.1 beta release
    16-Mar-2001: 2.1 final release

But the PEP doesn't really say much about which PEPs we're going to
work on.


Overview of the PEPs
--------------------

In my recent reworking of PEP 0 (the PEP index) I created a category
of Active PEPs -- these are PEPs that I definitely want to consider in
Python 2.1:

 I    42  pep-0042.txt  Small Feature Requests                 Hylton
 S   207  pep-0207.txt  Rich Comparisons                       Ascher
 S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
 S   217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 S   222  pep-0222.txt  Web Library Enhancements               Kuchling
 I   226  pep-0226.txt  Python 2.1 Release Schedule            Hylton
 S   227  pep-0227.txt  Statically Nested Scopes               Hylton

Note that I said *consider*: I'm sure that not all feature requests
from PEP 42 (a grab-bag of stuff that may or may not be feasible to
implement) will be implemented, and I'm not sure that all the other
PEPs on the list will be (especially nested scopes still seems iffy).

Two of these (207, 208) haven't been written yet -- but I know roughly
what they will contain, so they are included in this list, and not in
the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
PEPs).  These may be reconsidered for Python 2.1 if their authors care
to follow the PEP guidelines.

There are a bunch of other PEPs that I moved to the Pie-in-the-sky
category -- these are form-complete PEPs, but they are controversial
(e.g. there are two competing matrix op PEPs); some others I think are
not important; for yet others I think it's unrealistic to expect them
to be implemented by Python 2.1.  (There are also other things I'd
like to do that fit in the pie-in-the-sky category, like breaking up
the interpreter in several replaceable pieces.)


Other issues to work on
-----------------------

These aren't PEPs yet, but I think they need to become PEPs soon --
I'd like to see work on them go into Python 2.1:

- The buffer interface needs a revamping or simplification; this was
  discussed here previously.

- A warning framework.  I've got some ideas already.

- Integer division.  If we want to play by Paul Prescod's Language
  Evolution rules (PEP 5), we better get started on the first stage.
  E.g. we could introduce a // operator in 2.1 for integer division,
  and issue a warning when / is used for integers.  Then a year later
  (i.e., March 2002!) we could change / so that it returns a floating
  point number.

- Case sensitivity???

- Class/type dichotomy???

- Weak references.  This *is* a PEP, but there's no contents yet.  We
  could also try to implement (just) weak dictionaries.

- Internationalization.  Barry knows what he wants here; I bet Martin
  von Loewis and Marc-Andre Lemburg have ideas too.

- Arbitrart attributes on functions?  This would be a generalization
  of docstrings; with the intent that you don't have to put semantics
  in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
  This could possibly lead to implementing private, protected, public
  attributes too.

- Whatever you want to work on.  If you have an idea for a PEP that
  you think should be implemented in Python 2.1, or if you want to
  revive a PEP that's currently listed in one of the "unattainable"
  categories, now's the time to make a plea!


--Guido van Rossum (home page: http://www.python.org/~guido/)



From gstein at lyra.org  Sat Nov  4 00:13:50 2000
From: gstein at lyra.org (Greg Stein)
Date: Fri, 3 Nov 2000 15:13:50 -0800
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103151350.C12266@lyra.org>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>...
> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'd like to write up a free-threading PEP, but with all of my work on
Subversion right now, I'm not sure when that writing will occur or the
coding. I'll certainly do the PEP first because it could also be used as a
development roadmap for somebody do some of the work.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From gward at mems-exchange.org  Sat Nov  4 00:17:35 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 18:17:35 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
Message-ID: <20001103181734.A6809@ludwig.cnri.reston.va.us>

Hi all --

since "-Wall -Wstrict-prototypes" were added to OPT by default, a bunch
of warnings are showing up with GCC on Solaris.  (I haven't seen them on
Linux, so I assume this is OS-specific.)  See PR#121479 for a complete
list:
   https://sourceforge.net/bugs/?func=detailbug&bug_id=121479&group_id=5470)

Is anyone else looking into these?  Some of them are easy to fix,
eg. cast char to int when using 'isalnum()' and friends.  Some of them
are easy but unnecessary -- eg. sham variable initializations to shut up
the "might be used uninitialized" warning.  Others seem to require
#define'ing XOPEN_SOURCE or __EXTENSIONS__, which was a bit
controversial when it was done on Linux... but may be needed on Solaris
too.  I'm not sure yet; I've just started looking at it.  And I have to
run now.

Anyways, if anyone has guidelines/hints/suggestions for the best way to
fix these warnings, I'm all ears.  Also, if there are warnings we're not
going to worry about (eg. incorrect "might be used uninitialized"),
lemme know.

Thanks --

        Greg



From DavidA at ActiveState.com  Sat Nov  4 00:38:13 2000
From: DavidA at ActiveState.com (David Ascher)
Date: Fri, 3 Nov 2000 15:38:13 -0800 (Pacific Standard Time)
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>

>  S   207  pep-0207.txt  Rich Comparisons                       Ascher
>  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

> Two of these (207, 208) haven't been written yet -- but I know roughly
> what they will contain, so they are included in this list, and not in
> the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> PEPs).  These may be reconsidered for Python 2.1 if their authors care
> to follow the PEP guidelines.

I would _love_ to find someone to take over PEP ownership of these.  I
feel terrible that I haven't been able to find the time to do those right
(although I only feel moral ownership of 207, not 208, where others have
much more in-depth knowledge).

If someone wants to take over, please speak up and grab them.  I'll try to
find the time to share the information I have, would gladly give the very
early and now obsolete patches I wrote up.

--david
 




From akuchlin at mems-exchange.org  Sat Nov  4 02:31:05 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 3 Nov 2000 20:31:05 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103203105.A25097@kronos.cnri.reston.va.us>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>- Whatever you want to work on.  If you have an idea for a PEP that
>  you think should be implemented in Python 2.1, or if you want to

Using Distutils to compile most of the extension modules, taking the
first step to eliminating the Setup/Makefile.pre.in scheme.  I'll
begin drafting a PEP.

--amk




From guido at python.org  Sat Nov  4 04:38:17 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 22:38:17 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:38:13 PST."
             <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a> 
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a> 
Message-ID: <200011040338.WAA26703@cj20424-a.reston1.va.home.com>

> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

[David]
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).

I will take care of these myself if noone else picks them up.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Sat Nov  4 08:07:43 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Sat, 4 Nov 2000 02:07:43 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <20001103181734.A6809@ludwig.cnri.reston.va.us>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>

[Greg Ward]
> ...
> Also, if there are warnings we're not going to worry about (eg.
> incorrect "might be used uninitialized"), lemme know.

If a compiler is afraid something might be uninitialized, the code is too
clever for people to be sure it's correct at a glance too.  Note that right
before 2.0 was released, a bogus "uninitalized" msg got repaired, which
turned up a *legitimate* "uninitialized" msg that the bogus msg was covering
up.  The effort needed to fix one of these is too minor to be worth even
considering not fixing.

I'm not sure what gcc is complaining about in many of the cases; others are
quite clear (e.g., "confstr" apparently has no prototype in scope by the
time it's called in posixmodule.c, and that is indeed not good).

unix-should-be-shot-not-that-windows-shouldn't-ly y'rs  - tim





From py-dev at zadka.site.co.il  Sat Nov  4 19:00:49 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:00:49 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>

Here's a draft PEP.
Have I already mentioned how much it irks me to cater for other
editors in the PEP text itself?

PEP: Unassigned
Title: Reworking Python's Numeric Model
Version: $Revision$
Author: pep at zadka.site.co.il (Moshe Zadka)
Status: Draft
Type: Standards Track
Created: 4-Nov-2000
Post-History:

Abstract

    Today, Python's numerical model is similar to the C numeric model: 
    there are several unrelated numerical types, and when operations 
    between numerical types are requested, coercions happen. While the C 
    rational for the numerical model is that it is very similar to what
    happens on the hardware level, that rational does not apply to Python.
    So, while it is acceptable to C programmers that 2/3 == 0, it is very
    surprising to Python programmers.

Rationale

    In usability studies, one of Python hardest to learn features was
    the fact integer division returns the floor of the division. This
    makes it hard to program correctly, requiring casts to float() in
    various parts through the code. Python numerical model stems from
    C, while an easier numerical model would stem from the mathematical
    understanding of numbers.

Other Numerical Models

    Perl's numerical model is that there is one type of numbers -- floating
    point numbers. While it is consistent and superficially non-suprising,
    it tends to have subtle gotchas. One of these is that printing numbers
    is very tricky, and requires correct rounding. In Perl, there is also
    a mode where all numbers are integers. This mode also has its share of
    problems, which arise from the fact that there is not even an approximate
    way of dividing numbers and getting meaningful answers.

Suggested Interface For Python Numerical Model

    While coercion rules will remain for add-on types and classes, the built
    in type system will have exactly one Python type -- a number. There
    are several things which can be considered "number methods":

    1. isnatural()
    2. isintegral()
    3. isrational()
    4. isreal()
    5. iscomplex()

    a. isexact()

    Obviously, a number which answers m as true, also answers m+k as true.
    If "isexact()" is not true, then any answer might be wrong. (But not
    horribly wrong: it's close the truth).

    Now, there is two thing the models promises for the field operations
    (+, -, /, *): 

        If both operands satisfy isexact(), the result satisfies isexact()

        All field rules are true, except that for not-isexact() numbers,
        they might be only approximately true.      

    There is one important operation, inexact() which takes a number
    and returns an inexact number which is a good approximation.

    Several of the classical Python operations will return exact numbers
    when given inexact numbers: e.g, int().

Inexact Operations

    The functions in the "math" module will be allowed to return inexact
    results for exact values. However, they will never return a non-real
    number. The functions in the "cmath" module will return the correct
    mathematicl result.

Numerical Python Issues

    People using Numerical Python do that for high-performance
    vector operations. Therefore, NumPy should keep it's hardware
    based numeric model.

Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 10:50:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:50:10 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
Message-ID: <3A03DBD2.7E024978@lemburg.com>

David Ascher wrote:
> 
> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
> 
> > Two of these (207, 208) haven't been written yet -- but I know roughly
> > what they will contain, so they are included in this list, and not in
> > the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> > PEPs).  These may be reconsidered for Python 2.1 if their authors care
> > to follow the PEP guidelines.
> 
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).
> 
> If someone wants to take over, please speak up and grab them.  I'll try to
> find the time to share the information I have, would gladly give the very
> early and now obsolete patches I wrote up.

I can take over the coercion PEP: I've been working
on this before (see the proposal on my Python Pages).

I would also like to know whether the PEP-0224 will be considered
for 2.1 if I update the patch to make it a little more robust
w/r to the problems mentioned in that PEP -- I'd really like
to see this in Python soon, since it makes documenting Python
programs so much easier.

Note that I won't get around to do much work on these before
January... way too busy at the moment :-/

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From py-dev at zadka.site.co.il  Sat Nov  4 18:42:48 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 19:42:48 +0200
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Fri, 03 Nov 2000 18:01:27 EST." <200011032301.SAA26346@cj20424-a.reston1.va.home.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> 
Message-ID: <E13s7Lz-0002m1-00@darjeeling.zadka.site.co.il>

> Moshe and Andrew (the impatient youth :-) have asked what's on our
> plate for Python 2.1.

Yeah, sure. They're not making enough fun of my age where I work, <wink>.

> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

I'm working on that one, and will submit a PEP draft ASAP. 
I do want to know what kind of changes are acceptable: it seems you have
no problem with just patching over the numerical model, while I think
that solving this problem without recreating a host of others needs
total reworking of the numerical model.

> - Case sensitivity???

Writing a PEP for this seems trivial, but I thought the consensus was
that this should be solved by tools, not the language.

> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'm not sure I want to work on it, but in the past, we through around
ideas for pluggable nanny architecture. This is related to both
the case-sensitivity discussion above, and we also had lots of FAQ-stopping
nannies. (self-nanny, e.g., was designed to be a FAQ stopper)
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 10:58:49 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:58:49 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03DDD9.8527F7BE@lemburg.com>

Moshe Zadka wrote:
> 
> Here's a draft PEP.
> Have I already mentioned how much it irks me to cater for other
> editors in the PEP text itself?
> 
> PEP: Unassigned
> Title: Reworking Python's Numeric Model
> Version: $Revision$
> Author: pep at zadka.site.co.il (Moshe Zadka)
> Status: Draft
> Type: Standards Track
> Created: 4-Nov-2000
> Post-History:
> 
> Abstract
> 
>     Today, Python's numerical model is similar to the C numeric model:
>     there are several unrelated numerical types, and when operations
>     between numerical types are requested, coercions happen. While the C
>     rational for the numerical model is that it is very similar to what
>     happens on the hardware level, that rational does not apply to Python.
>     So, while it is acceptable to C programmers that 2/3 == 0, it is very
>     surprising to Python programmers.
> 
> Rationale
> 
>     In usability studies, one of Python hardest to learn features was
>     the fact integer division returns the floor of the division. This
>     makes it hard to program correctly, requiring casts to float() in
>     various parts through the code. Python numerical model stems from
>     C, while an easier numerical model would stem from the mathematical
>     understanding of numbers.
> 
> Other Numerical Models
> 
>     Perl's numerical model is that there is one type of numbers -- floating
>     point numbers. While it is consistent and superficially non-suprising,
>     it tends to have subtle gotchas. One of these is that printing numbers
>     is very tricky, and requires correct rounding. In Perl, there is also
>     a mode where all numbers are integers. This mode also has its share of
>     problems, which arise from the fact that there is not even an approximate
>     way of dividing numbers and getting meaningful answers.
> 
> Suggested Interface For Python Numerical Model
> 
>     While coercion rules will remain for add-on types and classes, the built
>     in type system will have exactly one Python type -- a number. 

While I like the idea of having the numeric model in Python
based on a solid class hierarchy, I don't think that this model
is implementable in Python 2.x without giving away performance.

> There
>     are several things which can be considered "number methods":
> 
>     1. isnatural()
>     2. isintegral()
>     3. isrational()
>     4. isreal()
>     5. iscomplex()

+1.

I would like to see methods on Python numbers too (after having
made some really good experiences with methods on strings ;-).
There's one problem though: how would you call these on
numeric literals ? ... 1.2.isreal() ?!

>     a. isexact()
> 
>     Obviously, a number which answers m as true, also answers m+k as true.
>     If "isexact()" is not true, then any answer might be wrong. (But not
>     horribly wrong: it's close the truth).

Not sure what you mean here: perhaps .isexact() <=> can be
represented in IEEE ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From py-dev at zadka.site.co.il  Sat Nov  4 19:19:13 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:19:13 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Sat, 04 Nov 2000 10:58:49 +0100." <3A03DDD9.8527F7BE@lemburg.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com> 
Message-ID: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>

> While I like the idea of having the numeric model in Python
> based on a solid class hierarchy, I don't think that this model
> is implementable in Python 2.x without giving away performance.

I think they are, using a similar trick to Fred's automorphing dictionaries.

> +1.
> 
> I would like to see methods on Python numbers too (after having
> made some really good experiences with methods on strings ;-).
> There's one problem though: how would you call these on
> numeric literals ? ... 1.2.isreal() ?!

Ummmm....how would you say you want to add 3 and 4, and multiply the result
by 5? 3+4*5?

No, you use parens:

(3+4)*5
(1.2).isreal()

> >     a. isexact()
> > 
> >     Obviously, a number which answers m as true, also answers m+k as true.
> >     If "isexact()" is not true, then any answer might be wrong. (But not
> >     horribly wrong: it's close the truth).
> 
> Not sure what you mean here: perhaps .isexact() <=> can be
> represented in IEEE ?

No, I meant "not represented exactly". The real meaning for that (one
that we might or might not promise) is that it's a float. It's a place 
where the numeric model takes the easy way out <wink>.

-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 11:21:52 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:21:52 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <3A03E340.38F78FA2@lemburg.com>

Guido van Rossum wrote:
> 
> Other issues to work on
> -----------------------
> 
> These aren't PEPs yet, but I think they need to become PEPs soon --
> I'd like to see work on them go into Python 2.1:
> 
> - The buffer interface needs a revamping or simplification; this was
>   discussed here previously.
> 
> - A warning framework.  I've got some ideas already.

This would be a *cool* thing. I have a need for such a framework
in mx.ODBC where ODBC often issues warnings. Currently these
turn out as exceptions which is not really all that useful
because some warnings can safely be ignored.
 
> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

+0... and then only, if there will be a tool to check Python
source code for integer divides.
 
> - Case sensitivity???

Should be left to Py3k. It could then be implemented by using
a special dictionary subclass as instance dictionary.
 
> - Class/type dichotomy???

One thing that would probably be implementable is a way to
maintain "instance" dictionaries for types (which are created
on-demand whenever an assignment is made). This would enable
extending types with new methods and attributes. "Subclassing"
could then be emulated by using new contructors which add the
new or changed methods to each created type instance, e.g.

class myclose:

    def __init__(self, object, basemethod):
        self.object = object
        self.basemethod = basemethod

    def __call__(self):
        print 'Closed file %s' % self.object
        self.basemethod()

def myfile(filename):
    f = open(filename)
    # add/override attributes
    f.newattribute = 1
    # add/override methods
    f.close = myclose(f, f.close)
    return f

Types would have to be made aware of this possibility. Python
could provide some helping APIs to make life easier for the
programmer.

> - Weak references.  This *is* a PEP, but there's no contents yet.  We
>   could also try to implement (just) weak dictionaries.

These already exist... http://www.handshake.de/~dieter/weakdict.html

mx.Proxy also has an implementation which support weak references.

BTW, are these still needed now that we have GC ?
 
> - Internationalization.  Barry knows what he wants here; I bet Martin
>   von Loewis and Marc-Andre Lemburg have ideas too.

We'd need a few more codecs, support for the Unicode compression,
normalization and collation algorithms.

> - Arbitrart attributes on functions?  This would be a generalization
>   of docstrings; with the intent that you don't have to put semantics
>   in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
>   This could possibly lead to implementing private, protected, public
>   attributes too.

Perhaps the above "trick" could help with this ?!
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov  4 11:31:01 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:31:01 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03E565.9D23559D@lemburg.com>

Moshe Zadka wrote:
> 
> > While I like the idea of having the numeric model in Python
> > based on a solid class hierarchy, I don't think that this model
> > is implementable in Python 2.x without giving away performance.
> 
> I think they are, using a similar trick to Fred's automorphing dictionaries.

You mean numbers "morph" to become floats, complex numbers, etc. 
on demand ? E.g. math.sqrt(-1) would return 1j ?!

> > +1.
> >
> > I would like to see methods on Python numbers too (after having
> > made some really good experiences with methods on strings ;-).
> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!
> 
> Ummmm....how would you say you want to add 3 and 4, and multiply the result
> by 5? 3+4*5?
> 
> No, you use parens:
> 
> (3+4)*5
> (1.2).isreal()

Ah. Of course :-) Cool !
 
> > >     a. isexact()
> > >
> > >     Obviously, a number which answers m as true, also answers m+k as true.
> > >     If "isexact()" is not true, then any answer might be wrong. (But not
> > >     horribly wrong: it's close the truth).
> >
> > Not sure what you mean here: perhaps .isexact() <=> can be
> > represented in IEEE ?
> 
> No, I meant "not represented exactly". The real meaning for that (one
> that we might or might not promise) is that it's a float. It's a place
> where the numeric model takes the easy way out <wink>.

Uhm, that's what I meant. I don't see much use for this though:
the whole meaning of "exact" is void w/r to floats. It should
be replaced by "accurate to n digits".

Note that there is a whole mathematical theory that tries to
deal with this problem: interval calculus. A package to support
this would probably make sense... a nice side-effect of 
interval calculus is that it allows "automatic" optimization
of functions within certain bounds. Numbers are replaced with
intervals and calculus is then done on the interval bounds.

This is just about as close as you can get to floating point
values with computer machinery ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at zadka.site.co.il  Sat Nov  4 19:52:35 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:52:35 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Sat, 04 Nov 2000 11:31:01 +0100." <3A03E565.9D23559D@lemburg.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <3A03E565.9D23559D@lemburg.com> 
Message-ID: <E13s8Qf-0002ok-00@darjeeling.zadka.site.co.il>

[Moshe Zadka, about efficient and honest numbers]
> I think they are, using a similar trick to Fred's automorphing dictionaries

[MAL]
> You mean numbers "morph" to become floats, complex numbers, etc. 
> on demand ? E.g. math.sqrt(-1) would return 1j ?!

math.sqrt has been dealt elsewhere in the PEP. It has been suggested
that math.foo will accept and return real numbers, and that
cmath.foo will accept and return complex numbers. If you 
want to always deal with complex numbers, put this in your site.py

import cmath
import sys
sys.modules['math']=cmath

> > No, I meant "not represented exactly". The real meaning for that (one
> > that we might or might not promise) is that it's a float. It's a place
> > where the numeric model takes the easy way out <wink>.
> 
> Uhm, that's what I meant. I don't see much use for this though:
> the whole meaning of "exact" is void w/r to floats. It should
> be replaced by "accurate to n digits".

I'm just promising that floats are inexact. I don't see a need for
"accuracy to n digits" (interval mathematics, etc.) in core Python.
This should be a new module if anyone needs it. Since inexact numbers
will only come about via external modules, you can just use:

import imath # interval math module
import sys
sys.modules['math']=imath.

I'm not repeating myself.

> This is just about as close as you can get to floating point
> values with computer machinery ;-)

I thought floats are the way to get to floating point values with computer
machinery?
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sat Nov  4 12:12:22 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 12:12:22 +0100
Subject: [Python-Dev] 2.1 tasks (Was: statically nested scopes)
Message-ID: <200011041112.MAA01135@loewis.home.cs.tu-berlin.de>

>  * Work on something CPAN-like.  This may or may not have repercussions for
>    the core; I don't know.

At a minimum, I think we need to include somethingCPANlike.py into the
core (library) to make something CPAN-like useful.

Regards,
Martin




From thomas at xs4all.net  Sat Nov  4 14:54:17 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 14:54:17 +0100
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>; from tim_one@email.msn.com on Sat, Nov 04, 2000 at 02:07:43AM -0500
References: <20001103181734.A6809@ludwig.cnri.reston.va.us> <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>
Message-ID: <20001104145417.I28658@xs4all.nl>

On Sat, Nov 04, 2000 at 02:07:43AM -0500, Tim Peters wrote:
> [Greg Ward]
> > ...
> > Also, if there are warnings we're not going to worry about (eg.
> > incorrect "might be used uninitialized"), lemme know.

> If a compiler is afraid something might be uninitialized, the code is too
> clever for people to be sure it's correct at a glance too.

> I'm not sure what gcc is complaining about in many of the cases; others are
> quite clear (e.g., "confstr" apparently has no prototype in scope by the
> time it's called in posixmodule.c, and that is indeed not good).

There are a few messages that certainly should be looked at. The
'uninitialized usage' messages, for instances, might seriously be problems.
In this case, though, the "'ord' might be used uninitialized" warning isn't
a real problem. 'ord' is indeed only set when 'size == 1' is true, but it's
also only used if 'size == 1', and size isn't changed inbetween those
checks. Whether it should be fixed or not is another issue, but at least
it isn't causing problems.

The 'subscript has type char' message I'm not so sure about -- what is the
problem with those ? I assume it has something to do with char's signedness
being undefined, but I'm not sure. I also thought 'up'casting, such as in
functioncalls (function needs int, you give it char) was done automatically,
as part of the language, and thus shouldn't be a problem.

But the -Wstrict-prototypes seems to detect a lot more 'errors' in system
headerfiles than in Python. For instance, all the "function declaration
isn't a prototype" warnings in signalmodule.c and intrcheck.c seem to be
caused by the SIG_ERR, SIG_DFL and SIG_IGN #defines, which Python can do
nothing about. (those SIG_ #defines are apparently defined as function
declarations without prototypes.) I've seen the same effect on BSDI 4.0.1,
where a few system include files define or declare functions without
prototypes. We can't fix those errors, except by complaining to the OS
vendor. Maybe we should just disable -Wstrict-prototypes (but not -Wall) for
releases, to avoid confusion. (Developers should still use
-Wstrict-prototypes, to catch warnings Python *can* do something about, me
thinks.)

And the 'confstr' message, well, ewww ;) The manpage on Linux seems to
indicate you need to define either _POSIX_C_SOURCE as 2, or define
_XOPEN_SOURCE, for it to work. But I've never understood how those defines
are supposed to interact, so I'm hesitant to go there ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 15:08:43 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:08:43 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>; from py-dev@zadka.site.co.il on Sat, Nov 04, 2000 at 08:19:13PM +0200
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <20001104150842.J28658@xs4all.nl>

On Sat, Nov 04, 2000 at 08:19:13PM +0200, Moshe Zadka wrote:

> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!

> you use parens:
> (1.2).isreal()

Why ? There is exactly no problem with this example :)

>>> 1.2.isreal()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'float' object has no attribute 'isreal'

If float objects had attributes, it would already work. The real problem
isn't with floats, but with nonfloats:

>>> 1.isreal()
  File "<stdin>", line 1
    1.isreal()
           ^
SyntaxError: invalid syntax

And the limitation is just the parser, currently. Whether we want to allow
that syntax is something that will have to be figured out. And the parser
would have to be fixed (either rewritten into at least an LL(2) parser, or
the metagrammar hacked so that 'NUMBER' doesn't eat the . after a number if
it isn't followed by another number or whitespace.) (At least IMHO, this
isn't an insurmountable problem, or even a medium-sized problem. It was just
never necessary to fix it.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 15:46:00 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:46:00 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104150842.J28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:08:43PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl>
Message-ID: <20001104154559.L28658@xs4all.nl>

On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:

> And the parser would have to be fixed (either rewritten into at least an
> LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> after a number if it isn't followed by another number or whitespace.) (At
> least IMHO, this isn't an insurmountable problem, or even a medium-sized
> problem. It was just never necessary to fix it.)

Actually, no, it isn't easily fixable, if at all. The problem is mostly the
scientific notation:

1.e5

Even if it was parsed properly, it is definately going to confuse people.
They wouldn't be able to say, for instance, 

1.e()

;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From cgw at fnal.gov  Sat Nov  4 16:21:02 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:21:02 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
	<3A03DDD9.8527F7BE@lemburg.com>
	<E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <14852.10590.837218.382614@buffalo.fnal.gov>

Moshe Zadka writes:

 > MAL > Not sure what you mean here: perhaps .isexact() <=> can be
 > MAL > represented in IEEE ?
 > 
 > No, I meant "not represented exactly". The real meaning for that (one
 > that we might or might not promise) is that it's a float. It's a place 
 > where the numeric model takes the easy way out <wink>.

Hmm, I like almost everything about your proposal.  The above point
bothers me slightly.  Are you saying (1.0).isexact() == 0?

Also, how about long integers?  Will they, under your new proposal, be
indistinguisable from regular ints?  While this has some appeal to it
it would be problematic for C extension modules.

Finally, although I'm no Schemer, the hierarchy you suggest sounds
very Schemish to me - I know they have a similar hierarchy of numeric
types with some of these same predicates to test for integrality,
rationality, reality, exactness - maybe there is something to be
learned by studying the Scheme model?





From cgw at fnal.gov  Sat Nov  4 16:25:37 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:25:37 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
	<3A03DDD9.8527F7BE@lemburg.com>
	<mal@lemburg.com>
	<E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
	<20001104150842.J28658@xs4all.nl>
	<20001104154559.L28658@xs4all.nl>
Message-ID: <14852.10865.96590.740569@buffalo.fnal.gov>

Thomas Wouters writes:

 > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
 > scientific notation:
 > 
 > 1.e5

You could strongly encourage people to spell this 1.0e5



From bckfnn at worldonline.dk  Sat Nov  4 17:21:48 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 16:21:48 GMT
Subject: [Python-Dev] Three argument slices.
Message-ID: <3a04376a.28016074@smtp.worldonline.dk>

Hi,

While updating the difference page in the Jython documentation, I came
across this:

- JPython sequences support three argument slices. i.e. 
  range(3)[::-1] == [2,1,0]. 
  CPython should be fixed.

Is this actually true? Should (and will cpython) change in this respect?

regards,
finn



From py-dev at zadka.site.co.il  Sun Nov  5 01:57:12 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 02:57:12 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Charles G Waldman <cgw@fnal.gov> 
   of "Sat, 04 Nov 2000 09:21:02 CST." <14852.10590.837218.382614@buffalo.fnal.gov> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <14852.10590.837218.382614@buffalo.fnal.gov> 
Message-ID: <E13sE7V-0002v1-00@darjeeling.zadka.site.co.il>

> Hmm, I like almost everything about your proposal.  The above point
> bothers me slightly.  Are you saying (1.0).isexact() == 0?

Yes. 1.0 is not an exact number. What's wrong with that?
(Consider stuff like 0.333333333*3: this shouldn't be exact!)

> Also, how about long integers?  Will they, under your new proposal, be
> indistinguisable from regular ints?

Yes.

> While this has some appeal to it
> it would be problematic for C extension modules.

I haven't mentioned anything about implementation, so I haven't dealt
with the C level at all. Currently, a Python-level API is under 
consideration. I believe I can keep current day C API almost unchanged.

> Finally, although I'm no Schemer, the hierarchy you suggest sounds
> very Schemish to me

I shamelessly stole it from Scheme, with only minor changes -- most
of them about hardening some things Scheme left for implementations
to decide.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mwh21 at cam.ac.uk  Sat Nov  4 17:47:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 04 Nov 2000 16:47:31 +0000
Subject: [Python-Dev] Three argument slices.
In-Reply-To: bckfnn@worldonline.dk's message of "Sat, 04 Nov 2000 16:21:48 GMT"
References: <3a04376a.28016074@smtp.worldonline.dk>
Message-ID: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>

bckfnn at worldonline.dk (Finn Bock) writes:

> Hi,
> 
> While updating the difference page in the Jython documentation, I
> came across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this
> respect?

Well, there's a patch I wrote on sf to add this to CPython, but it was
too late for 2.0 and it got postponed.

Does J[P]ython allow

l = range(10)
l[::3] = range(4)

?  That's one of the odder bits of the behaviour of my patch.

awaiting-pronouncement-(or-even-consensus)-ly y'rs
m.

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From bckfnn at worldonline.dk  Sat Nov  4 19:41:59 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 18:41:59 GMT
Subject: [Python-Dev] Three argument slices.
In-Reply-To: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
References: <3a04376a.28016074@smtp.worldonline.dk> <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3a045808.36365811@smtp.worldonline.dk>

>> - JPython sequences support three argument slices. i.e. 
>>   range(3)[::-1] == [2,1,0]. 
>>   CPython should be fixed.
>> 
>> Is this actually true? Should (and will cpython) change in this
>> respect?
>
>Well, there's a patch I wrote on sf to add this to CPython, but it was
>too late for 2.0 and it got postponed.
>
>Does J[P]ython allow
>
>l = range(10)
>l[::3] = range(4)
>
>?  That's one of the odder bits of the behaviour of my patch.

No:

Jython 2.0 pre-alpha on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> l = range(10)
>>> l[::3] = range(4)
Traceback (innermost last):
  File "<console>", line 1, in ?
ValueError: step size must be 1 for setting list slice

regards,
finn



From gstein at lyra.org  Sat Nov  4 19:56:08 2000
From: gstein at lyra.org (Greg Stein)
Date: Sat, 4 Nov 2000 10:56:08 -0800
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:46:00PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl>
Message-ID: <20001104105608.A10135@lyra.org>

Oh, this is just simple:

  (1.2).isreal()
  1.2 .isreal()

As Thomas said, fixing the grammar/parser would be rather difficult, so just
expect people to use parens or an extra space if they want to use it on a
constant.

[ of course, it is very silly to make *any* changes to the grammar just to
  allow people to use these on a constant; that is quite a silly "usage"
  that we don't need to pander to; the above "workarounds", if you will, are
  sufficient for the bozos who use it on a constant. ]

Cheers,
-g

On Sat, Nov 04, 2000 at 03:46:00PM +0100, Thomas Wouters wrote:
> On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:
> 
> > And the parser would have to be fixed (either rewritten into at least an
> > LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> > after a number if it isn't followed by another number or whitespace.) (At
> > least IMHO, this isn't an insurmountable problem, or even a medium-sized
> > problem. It was just never necessary to fix it.)
> 
> Actually, no, it isn't easily fixable, if at all. The problem is mostly the
> scientific notation:
> 
> 1.e5
> 
> Even if it was parsed properly, it is definately going to confuse people.
> They wouldn't be able to say, for instance, 
> 
> 1.e()
> 
> ;P
> 
> -- 
> Thomas Wouters <thomas at xs4all.net>
> 
> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/



From thomas at xs4all.net  Sat Nov  4 22:29:28 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:29:28 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <14852.10865.96590.740569@buffalo.fnal.gov>; from cgw@fnal.gov on Sat, Nov 04, 2000 at 09:25:37AM -0600
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <14852.10865.96590.740569@buffalo.fnal.gov>
Message-ID: <20001104222928.M28658@xs4all.nl>

On Sat, Nov 04, 2000 at 09:25:37AM -0600, Charles G Waldman wrote:
> Thomas Wouters writes:

>  > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
>  > scientific notation:

>  > 1.e5

> You could strongly encourage people to spell this 1.0e5

Oh, sure, but that isn't going to solve anything, unless you are proposing
to break the common practice of not requiring zeros before or after decimal
points entirely, and thus breaking gobs of code. The problem is simply that
the syntax is truly ambiguous, and there is no way to tell whether the
statement

  x = 1.e5

is meant to assign 100000 (as a float) to 'x', or assign the 'e5' attribute
of the object '1' to 'x'. Not even full context-based parsing is going to
solve that. This is an edge case, and not likely to happen in real life, but
I don't think it's really worth the trouble, all in all. We'd have to
rewrite the parser into something other than a look-ahead parser to be able
to correctly parse the cases where the syntax isn't really ambiguous, such
as

x = 1.e42e

or some such, and that would still leave unparseable syntax. And all that
just to avoid forcing people to use parentheses around 'normal' integer
literals when directly following them with an attribute dereference. After
all,

 t = 1
 x = t.e5

is perfectly valid, and probably a lot more common. Who needs to run a
method on a literal anyway [don't mention the .join() war!]

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 22:32:39 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:32:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104105608.A10135@lyra.org>; from gstein@lyra.org on Sat, Nov 04, 2000 at 10:56:08AM -0800
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <20001104105608.A10135@lyra.org>
Message-ID: <20001104223239.N28658@xs4all.nl>

On Sat, Nov 04, 2000 at 10:56:08AM -0800, Greg Stein wrote:
> Oh, this is just simple:

>   (1.2).isreal()
>   1.2 .isreal()

Very observant, Greg ! I hadn't even realized that, but it makes perfect
sense if you think about it ;) '1.e5' is a single token, to the parser (a
NUMBER), so it can't have whitespace inbetween. But '1 .e5' is naturally
broken up into at least two tokens (three, in this case), and thus is
'correctly' parsed even in todays parser:

>>> 1 .e5
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'int' object has no attribute 'e5'

Y'all just need to quit thinking about floats as the problem... the problem
is ints, not floats ;)

I-shut-up-now-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From martin at loewis.home.cs.tu-berlin.de  Sat Nov  4 22:15:13 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 22:15:13 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>

> Yes. 1.0 is not an exact number. What's wrong with that?
> (Consider stuff like 0.333333333*3: this shouldn't be exact!)

It's not inherently wrong. It just points out an omission in the PEP:
it doesn't talk about the meaning of number literals. Since the new
model is going to follow algebraic principles more closely, I had
expected that

  0.333333333 == 333333333 / 1000000000

where, as I understand the proposal, the right-hand side is an exact
number (so 0.333333333*3 would be 999999999 / 1000000000).

One of the more-frequent questions on python-help is confusion about
floating-point numbers, e.g. why is the result of 1.99+4.99 printed
as 6.9800000000000004; users often report that as a bug.

Of course, spelling the number you had in mind as inexact(0.333333333)
is hardly acceptable, either.

Regards,
Martin



From skip at mojam.com  Sat Nov  4 23:55:44 2000
From: skip at mojam.com (Skip Montanaro)
Date: Sat, 4 Nov 2000 16:55:44 -0600 (CST)
Subject: [Python-Dev] Accessing RH 7.0 on SF?
Message-ID: <14852.37872.843069.514782@beluga.mojam.com>

Someone posted a bug about the bsddb config stuff related to RH7.0, which I
don't have direct access to.  I've seen others access different Linux
dialects on the SF site.  Can someone explain how I can access RH7.0 there?
All I really need to do at the moment is peruse the /usr/include/db3 stuff.

Thx,

Skip



From thomas at xs4all.net  Sat Nov  4 23:10:32 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 23:10:32 +0100
Subject: [Python-Dev] Accessing RH 7.0 on SF?
In-Reply-To: <14852.37872.843069.514782@beluga.mojam.com>; from skip@mojam.com on Sat, Nov 04, 2000 at 04:55:44PM -0600
References: <14852.37872.843069.514782@beluga.mojam.com>
Message-ID: <20001104231031.P28658@xs4all.nl>

On Sat, Nov 04, 2000 at 04:55:44PM -0600, Skip Montanaro wrote:
> Someone posted a bug about the bsddb config stuff related to RH7.0, which I
> don't have direct access to.  I've seen others access different Linux
> dialects on the SF site.  Can someone explain how I can access RH7.0 there?
> All I really need to do at the moment is peruse the /usr/include/db3 stuff.

I don't think they have it, yet. RH7 is pretty new after all. I can give you
an account on my home machine if you want, though, it's RH7. Only accessible
through ssh currently, but if you want I can turn on telnet. Just send me an
email with login/pw you want ;) Or if you'd rather I peruse the db3 stuff
for you, that's fine too.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From moshez at zadka.site.co.il  Sun Nov  5 09:20:45 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 10:20:45 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sat, 04 Nov 2000 22:15:13 +0100." <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> It's not inherently wrong. It just points out an omission in the PEP:
> it doesn't talk about the meaning of number literals.

That's right -- but I did mean that floating-point literals will be
inexact.

> Since the new
> model is going to follow algebraic principles more closely, I had
> expected that
> 
>   0.333333333 == 333333333 / 1000000000
> 
> where, as I understand the proposal, the right-hand side is an exact
> number (so 0.333333333*3 would be 999999999 / 1000000000).
> 
> One of the more-frequent questions on python-help is confusion about
> floating-point numbers, e.g. why is the result of 1.99+4.99 printed
> as 6.9800000000000004; users often report that as a bug.

That's one thing my PEP is not meant to help with -- floating point
numbers will remain hard. Hopefully, people will use them less often
when they'll have rational arithmetic.

-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 09:28:06 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 09:28:06 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 10:20:45 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
Message-ID: <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>

> > It's not inherently wrong. It just points out an omission in the PEP:
> > it doesn't talk about the meaning of number literals.
> 
> That's right -- but I did mean that floating-point literals will be
> inexact.

Remind you that your model has no notion of floating-point numbers -
then what the heck are floating-point literals? The numbers that you
can write in a base-10 notation are all rational numbers, and the
point doesn't really float in them...

Regards,
Martin



From py-dev at zadka.site.co.il  Sun Nov  5 18:25:33 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 19:25:33 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sun, 05 Nov 2000 09:28:06 +0100." <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> Remind you that your model has no notion of floating-point numbers -
> then what the heck are floating-point literals? The numbers that you
> can write in a base-10 notation are all rational numbers, and the
> point doesn't really float in them...

Well, of course they are rational numbers. The only question is whether 1.0
should be inexact or exact. While that is not specified in the PEP (which
was meant for Barry to assign me a PEP number primarily...), I think
the principle of least suprise would be to treat 1.0 as inexact.
(IOW, not to promise that (1.0/3.0)*3.0 == 1.0)
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 11:24:47 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 11:24:47 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 19:25:33 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>

> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

To long-term Python users, that would be the least surprise.

To new users, the entire notion of inexact numbers is surprising; more
so that something as simple as 1.0 is inexact.

To computer numerics fanatics, it is surprising that 1.0 is inexact,
since the common representations of floating point numbers are well
capable of representing it exactly.

Regards,
Martin



From py-dev at zadka.site.co.il  Sun Nov  5 20:04:10 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 21:04:10 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model 
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sun, 05 Nov 2000 11:24:47 +0100." <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>

[Moshe Zadka]
> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

[Martin v. Loewis]
> To long-term Python users, that would be the least surprise.

And to long term users of C/Perl/etc., once they map the numerical 
concepts correctly. But I hardly thing we should be arguing about this
at this stage: I'm willing to leave this as an open issue in the PEP,
if this is all you find wrong with it...

And a request to all Python-Devvers: please direct comments directly to
me, and I promise I'll summarize them all in the PEP. As soon as I
get a PEP number, I'll publish an updated version, with all objections and open
issues sumarized.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 12:21:14 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 12:21:14 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 21:04:10 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>

> And to long term users of C/Perl/etc., once they map the numerical 
> concepts correctly. But I hardly thing we should be arguing about this
> at this stage: I'm willing to leave this as an open issue in the PEP,
> if this is all you find wrong with it...

Sorry I didn't mention it: Overall, I like your proposal very well.
I'm missing the section on implementation strategies, though.

> And a request to all Python-Devvers: please direct comments directly
> to me, and I promise I'll summarize them all in the PEP. As soon as
> I get a PEP number, I'll publish an updated version, with all
> objections and open issues sumarized.

Yes, that is a tricky part of the PEP procedure: not commenting in the
public initially. I think PEP authors can contribute by not posting
the text of their PEP publically.

Regards,
Martin




From gstein at lyra.org  Sun Nov  5 12:40:44 2000
From: gstein at lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 03:40:44 -0800
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sun, Nov 05, 2000 at 12:21:14PM +0100
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <20001105034044.M10135@lyra.org>

On Sun, Nov 05, 2000 at 12:21:14PM +0100, Martin v. Loewis wrote:
>...
> > And a request to all Python-Devvers: please direct comments directly
> > to me, and I promise I'll summarize them all in the PEP. As soon as
> > I get a PEP number,

Euh... is process getting in the way of progress?

Allocate yourself a PEP number and publish the darn thing. If you don't feel
comfortable grabbing a PEP number, then just post it to the list or
something.

One of the worst things that can happen is to feel locked in by a cumbersome
process. Open Source works because people can flow at their own speed,
independent of what is happening with others. If Guido and company are too
busy to update syncmail... no problem! Thomas has time and inclination and
jumps in to fix it. Somebody too busy to revamp the headers for ANSI C? No
worries... we have a lot of volunteers just ready and able to do that.

But you throw in the gates? The locks? The process? It halts.

> > I'll publish an updated version, with all
> > objections and open issues sumarized.
> 
> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

I'm not sure what you're saying here. That a PEP author should develop the
PEP entirely in private? Only when it is fully-cooked, that it should be
published?

Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
a first draft. Revise. Revise. Revise.

Comments on PEPs are generated when people feel their are listened to. If a
PEP author develops a PEP entirely in secret, then the feedback is going to
be diminished because it is hard for people to really know if their comments
and ideas are being captured and considered. When you have that feedback
loop and the positive reinforcement, then you will engender more commentary.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From nhodgson at bigpond.net.au  Sun Nov  5 12:59:03 2000
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sun, 5 Nov 2000 22:59:03 +1100
Subject: [Python-Dev] What to choose to replace Tkinter?
Message-ID: <049d01c0471f$d7899450$8119fea9@neil>

Andrew Kuchling:

> I believe the GNOME (not GTk's, but GNOME's) canvas widget began as a
> fork of the Tk widget that was then substantially enhanced to be a
> general-purpose display engine, with antialiasing, alpha compositing,
> more attention to performance, and other fancy features.  I don't know
> if the corresponding text widget (which is Pango, www.pango.org, I
> think) is equally featureful.

   There is a port of the Tk text widget to GTK+ by Havoc Pennington which
doesn't require Pango. Its aims are a little muddled as a high quality Pango
based text widget is also under long term development.

   Pango isn't just a text widget but a layered set of capabilities allowing
development of internationalised layout and rendering (the equivalent of
Microsoft's Uniscribe). Scintilla for GTK+ will use Pango to implement
Unicode once Pango is released.

   Neil




From moshez at math.huji.ac.il  Sun Nov  5 13:12:09 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:12:09 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>

On Sun, 5 Nov 2000, Martin v. Loewis wrote:

> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

Perhaps, but that is part of the PEP flow: public post->number
Well, thanks a lot for the feedback.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Sun Nov  5 13:16:46 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:16:46 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
Message-ID: <Pine.GSO.4.10.10011051412530.28025-100000@sundial>

On Sun, 5 Nov 2000, Greg Stein wrote:

> Euh... is process getting in the way of progress?

Perhaps. Well, I won't let it get in the way more then a few more hours --
I'll allocate myself a PEP. It's  easier to get forgiveness then
permission <wink>

> I'm not sure what you're saying here. That a PEP author should develop the
> PEP entirely in private? Only when it is fully-cooked, that it should be
> published?

Far from it. Only tht the discussion tends to clutter up Python-Dev too
much, so I want to moderate it by way of private mail to me-> checking
to the PEP. I hope everyone here trust me to be honest enough not to
censor competing points.

> Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
> a first draft. Revise. Revise. Revise.

I intend to. 

> Comments on PEPs are generated when people feel their are listened to. If a
> PEP author develops a PEP entirely in secret

All future revisions will be in the Python CVS. Only security through
obscurity can keep me secret there <wink>
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Sun Nov  5 17:45:55 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 11:45:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
Message-ID: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Perhaps, but that is part of the PEP flow: public post->number
 > Well, thanks a lot for the feedback.

  Don't hesitate to publish an updated version just because Barry
hasn't assigned a number.  If you don't want to publish the full text
too often, assign a number to yourself in the PEP index and check that
in, then add the actual PEP with the right number.
  (Make sure you run "./pep2html.py -i 0 <num>" to update the index
and your PEP on python.sourceforge.net once you've made the checkins.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From thomas at xs4all.net  Sun Nov  5 20:18:51 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 20:18:51 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <200011051655.IAA13536@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Sun, Nov 05, 2000 at 08:55:27AM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
Message-ID: <20001105201851.C27208@xs4all.nl>

On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:

> Added Files:
> 	pep-0228.txt 
> Log Message:
> Added first revision of numerical model pep.

> ***** Error reading new file: (2, 'No such file or directory')
> ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461

Thank you, Moshe ;) Now to figure out why it does work for some people, or
in some cases ;P I'm guessing it's a race condition of some sort, but damned
if I know what triggers it.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From gstein at lyra.org  Sun Nov  5 22:54:39 2000
From: gstein at lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 13:54:39 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105201851.C27208@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 08:18:51PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl>
Message-ID: <20001105135439.V10135@lyra.org>

On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:
> On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:
> 
> > Added Files:
> > 	pep-0228.txt 
> > Log Message:
> > Added first revision of numerical model pep.
> 
> > ***** Error reading new file: (2, 'No such file or directory')
> > ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461
> 
> Thank you, Moshe ;) Now to figure out why it does work for some people, or
> in some cases ;P I'm guessing it's a race condition of some sort, but damned
> if I know what triggers it.

It certainly could be a race condition. Just look at blast_mail(). It forks
off the operation, and it might run before the file arrives in the
repository. (??)

Or is it possible that the script looks for pep-0228.txt rather than getting
a copy from pep-0228.txt,v ?? My script does "cvs -Qn update -p -r1.1 FILE"
for new files, and pipes that to the output.

Note that the log_accum.pl script that I use for my CVS repository (which
came from Apache, which came from BSD) does not fork. It seems to work quite
fine with added files.

[ and the point in blast_mail() about holding the CVS lock too long is
  rather silly given the horsepower of the SourceForge boxes ]

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From thomas at xs4all.net  Sun Nov  5 23:12:35 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:12:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105135439.V10135@lyra.org>; from gstein@lyra.org on Sun, Nov 05, 2000 at 01:54:39PM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org>
Message-ID: <20001105231235.X12776@xs4all.nl>

On Sun, Nov 05, 2000 at 01:54:39PM -0800, Greg Stein wrote:
> On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:

> It certainly could be a race condition. Just look at blast_mail(). It forks
> off the operation, and it might run before the file arrives in the
> repository. (??)

No... that's not it.

> Or is it possible that the script looks for pep-0228.txt rather than getting
> a copy from pep-0228.txt,v ??

Almost correct :) The problem is that the loginfo process is run in the
server-specific /tmp dir, and the filename is not prefixed by the path to
the CVSROOT or some such. I guess it's pure coincidence that the file is
still there when the syncmail script arrives at the 'open()' call.

> My script does "cvs -Qn update -p -r1.1 FILE" for new files, and pipes
> that to the output.

Yes... I just wrote a small patch to syncmail which does exactly that
(actually, it uses 'newrev' rather than a hardcoded '1.1', and it uses -f
and -n but not -Q -- if we add -Q, we should add it to the diff commands as
well, and it might confuse developers that are used to reading the debug
info ;)

The main reason I'm delaying the checkin is to test it on the only CVS
repository I can play with, which is over a slow link to an american
highschool. Now if only you had mailed an hour earlier, Greg, I wouldn't
have had to go through that trouble ;)

> [ and the point in blast_mail() about holding the CVS lock too long is
>   rather silly given the horsepower of the SourceForge boxes ]

Well, syncmail was written to manage the Python CVS tree on a slow Sun (I
believe) and did an rsync-over-ssh to another machine as well. That can
definately take long ;) If we just remove the fork, the rest of syncmail
might just work, even with new files. In the mean time, I'll check in my
change. It might be the best thing to do anyway, since it shouldn't
interfere unless the file isn't there.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jeremy at alum.mit.edu  Sun Nov  5 22:08:55 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:08:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
	<200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
	<E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
	<200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<20001105034044.M10135@lyra.org>
Message-ID: <14853.52327.355764.528782@bitdiddle.concentric.net>

[Comments from Greg and Moshe on the PEP process.]

I don't see the PEP process creating any impediments here.  There is
almost no process -- Barry assigns numbers to PEPs and Guido rules on
them.  We've got some rules about what must be in the PEP before it is
approved, but almost none about how the PEP is created.

There is nothing about the PEP process that prevents a healthy
discussion of issues, in private mail or on a mailing list (python-dev
or otherwise).  We had lots of comments on the statically nested
scopes PEP before Barry assigned it a number.  An entire PEP could be
created and discussed before it gets a number.

Someone may want to work on a PEP in privacy and wait to share it
until the entire document is complete; that's fine too, provided that
revisions are made based on feedback.

One goal we had when setting up the PEP process was to limit the
amount of repeated discussion on an issue.  It's not too uncommon for
email discussions to go in circles or to endlessly rehash the same few
issues.  We hoped that PEP authors would incorporate a discussion of
such issues in the PEP and reduce the amount of wasted bandwidth on
repetitive discussion.

Let's not waste time discussing how to create PEPs and instead
actually create them.  The clock is ticking for new features in 2.1;
the tentative deadline for new PEPs is mid-December.

Jeremy



From jeremy at alum.mit.edu  Sun Nov  5 22:14:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:14:36 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
Message-ID: <14853.52668.117844.28459@bitdiddle.concentric.net>

>>>>> "FLD" == Fred L Drake, <fdrake at acm.org> writes:

  FLD> Moshe Zadka writes:
  >> Perhaps, but that is part of the PEP flow: public post->number
  >> Well, thanks a lot for the feedback.

  FLD>   Don't hesitate to publish an updated version just because
  FLD>   Barry
  FLD> hasn't assigned a number.  If you don't want to publish the
  FLD> full text too often, assign a number to yourself in the PEP
  FLD> index and check that in, then add the actual PEP with the right
  FLD> number.

I thought we discussed this earlier and agreed that a little bit of
control over the process was healthy.  I would prefer to see all PEP
creation go through Barry.  We can circulate drafts in email before
that.

Jeremy



From thomas at xs4all.net  Sun Nov  5 23:29:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:29:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105231235.X12776@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 11:12:35PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org> <20001105231235.X12776@xs4all.nl>
Message-ID: <20001105232921.E27208@xs4all.nl>

On Sun, Nov 05, 2000 at 11:12:35PM +0100, Thomas Wouters wrote:

> In the mean time, I'll check in my change. It might be the best thing to
> do anyway, since it shouldn't interfere unless the file isn't there.

Since changes to files in CVSROOT go to python-checkins-admin rather than
python-checkins, here's the diff I just checked in:

Index: syncmail
===================================================================
RCS file: /cvsroot/python/CVSROOT/syncmail,v
retrieving revision 3.14
retrieving revision 3.15
diff -c -c -r3.14 -r3.15
*** syncmail    2000/11/02 21:44:32     3.14
--- syncmail    2000/11/05 22:24:29     3.15
***************
*** 85,91 ****
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             fp = open(file)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')
--- 85,95 ----
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             if os.path.exists(file):
!                 fp = open(file)
!             else:
!                 update_cmd = 'cvs -fn update -r %s -p %s' % (newrev, file)
!                 fp = os.popen(update_cmd)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')

See the manpage for 'cvs' for an explanation of the options ;) This should
fix 99.95% or so of the problem (there is still a tiny window for the file
being removed inbetween the os.path.exists and the actual open) and is
probably best even if we do remove the fork() from syncmail.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From greg at cosc.canterbury.ac.nz  Mon Nov  6 01:56:16 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 13:56:16 +1300 (NZDT)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
Message-ID: <200011060056.NAA29965@s454.cosc.canterbury.ac.nz>

"Martin v. Loewis" <martin at loewis.home.cs.tu-berlin.de>:

> To computer numerics fanatics, it is surprising that 1.0 is inexact,
> since the common representations of floating point numbers are well
> capable of representing it exactly.

I suppose in principle one could meticulously keep track
of which floating point numbers in a calculation were
exact and which weren't. But you'd lose the property that
any arithmetic operation on exact operands produces an
exact result.

Also, it would be very tedious and inefficient to have
to keep track of exactness so exactly!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 02:03:06 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:03:06 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
Message-ID: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz>

Guido:

> I'd be happy to make an explicit list of
> those builtins that should not be messed with

There's a precedent for this in Scheme, which has a notion
of "integrable procedures".

As for the rest, with static scoping it will be possible to
make access to builtins just as efficient as locals, while
still allowing them to be rebound, so there's no reason why
__builtin__.__dict__.open = foo can't continue to work,
if so desired.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 02:16:27 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:16:27 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A02B489.89EF108C@lemburg.com>
Message-ID: <200011060116.OAA29972@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> Nested scopes will introduce cycles in all frame objects.

It doesn't have to be that way. A static link is only
needed if a function actually uses any variables from
an outer scope. In the majority of cases, it won't.

And it's possible to do even better than that. You can
separate out variables referred to in an inner scope
and store them separately from the rests of the frame,
so you only keep what's really needed alive.

> This means that with GC turned off, frame objects will live
> forever

Don't allow GC to be turned off, then! (Presumably this
feature would only be considered once GC has become a
permanent feature of Python.)

> BTW, Python's GC only works for a few builtin types (frames
> are not among the supported types)

But they could become so if necessary, surely?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 02:19:02 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:19:02 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Sat, 04 Nov 2000 20:00:49 +0200."
             <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>

So I go offline for a couple of days to entertain guests and have my
body kicked around in a dance class, and I have 25 messages discussing
Python's numeric model waiting for me...

I was hoping that Tim would chime in, but he's apparently taken the
weekend off -- very much out of character. :-)

I like the idea of a PEP to rework the numeric model.  I think that
Moshe, being a mathematician by training, will make a good editor.  I
think that this has a snowball's chance in hell to make it into Python
2.1 however -- there are many deep issues, and it's a bad idea to
experiment too much in the released version of the language.

Some of the issues are more usability issues than of a mathematical
nature.  For example, Tim has conjectured that using binary floating
point will always be a problem for the "unwashed masses" -- the only
thing they might understand is decimal floating point, not rational
numbers.  There are at least two issues here:

(1) As long as the internal representation is not the same as what is
    commonly printed, there will be surprises -- with rationals just
    as much as with floating point.  There are issues with decimal
    floating point too, but they are only the issues having to do with
    loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
    1.0) and not with loss of precision in the printing, where most of
    the "bug reports" we get seem to concentrate.

(2) Rational numbers have the unpleasant property of growing
    unboundedly during many innocent calculations, thereby using up
    exorbitant amounts of memory and slowing down the calculation --
    often mysteriously, because what is displayed is truncated.

Another issue that I might bring up is that there are no inexact
numbers (each floating point number is perfectly exact and rational)
-- there are only inexact operations.  I'm not sure what to do with
this though.

If we take its meaning literally, the isreal() function should only
return true for numbers for which isrational() is also true:
mathematically speaking, real numbers that aren't also rational don't
have an easy finite representation, since they are numbers like
sqrt(2) or pi.

I'll leave it to Tim to explain why inexact results may not be close
to the truth.

Tim may also break a lance for IEEE 754.

Finally, the PEP doesn't talk about how the numeric model can be
extended, and how other numeric types can be blended in.  E.g. I've
heard of wild floating point representations that make multiplication
and division really cheap but addition a pain, rather than the other
way around; some people may want to have long ints implemented using
the GNU mp library, and so on.  Such custom types should be supported
as well as native types -- like they are now.

--Guido van Rossum (home page: http://www.python.org/~guido/)

PS.  The "1.isreal()" problem is a non-problem.  This question is only
interesting to ask about variables.



From guido at python.org  Mon Nov  6 02:25:08 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:25:08 -0500
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
In-Reply-To: Your message of "Sat, 04 Nov 2000 10:50:10 +0100."
             <3A03DBD2.7E024978@lemburg.com> 
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>  
            <3A03DBD2.7E024978@lemburg.com> 
Message-ID: <200011060125.UAA03986@cj20424-a.reston1.va.home.com>

Marc-Andre:
> I can take over the coercion PEP: I've been working
> on this before (see the proposal on my Python Pages).

Thanks, excellent (although I haven't seen your proposal yet).

> I would also like to know whether the PEP-0224 will be considered
> for 2.1 if I update the patch to make it a little more robust
> w/r to the problems mentioned in that PEP -- I'd really like
> to see this in Python soon, since it makes documenting Python
> programs so much easier.

I "kinda" like the idea of having attribute docstrings (meaning it's
not of great importance to me) but there are two things I don't like
in your current proposal:

1. The syntax you propose is too ambiguous: as you say, stand-alone
   string literal are used for other purposes and could suddenly
   become attribute docstrings.

2. I don't like the access method either (__doc_<attrname>__).

> Note that I won't get around to do much work on these before
> January... way too busy at the moment :-/

That's a problem -- we really want to have the PEPs ready for review
by mid December.  This will also be a problem for the coercion PEP --
if you think you won't be able to work on it before then, I'd prefer
to find another (co-)author.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:05:20 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:05:20 -0500
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:03:06 +1300."
             <200011060103.OAA29969@s454.cosc.canterbury.ac.nz> 
References: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>

> Guido:
> > I'd be happy to make an explicit list of
> > those builtins that should not be messed with

[Greg Ewing]
> There's a precedent for this in Scheme, which has a notion
> of "integrable procedures".

Good!

> As for the rest, with static scoping it will be possible to
> make access to builtins just as efficient as locals, while
> still allowing them to be rebound, so there's no reason why
> __builtin__.__dict__.open = foo can't continue to work,
> if so desired.

I'm not sure what you mean.  With integrable procedures (whatever they
may be :-) I believe this is possible.  Without them, the lookup in
globals() can be skipped for builtins, but a local is accessed with
*zero* dict lookups -- how would you do this while still supporting
__builtin__.__dict__.open = foo?  have "hookable" dictionaries?
(Those would solve a bunch of problems, but are not under
consideration at the moment.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gmcm at hypernet.com  Mon Nov  6 03:21:47 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Sun, 5 Nov 2000 21:21:47 -0500
Subject: [Python-Dev] Stackless pages
Message-ID: <3A05CF6B.32107.2CAF777@localhost>

I have put up 6 pages of information about stackless at

http://www.mcmillan-inc.com/stackless.html

The first page attempts to give a conceptual overview of 
stackless. Notice I said "conceptual" - I make no attempt to 
be technically accurate!

Next follow 4 pages of tutorial. Mainly this is a discussion of 
implementing generators and coroutines through the 
continuation module. It includes rewrites of 2 samples that 
Tim used to demonstrate his coroutines-implemented-via-
threads madness.

Finally, the last page is about SelectDispatcher, which is kind 
of Medusa using coroutines. Included as a demonstration is a 
full FTPServer that will run on Windows. This is not just demo 
quality code - it's at the core of a couple commercial apps I'm 
doing for clients, at least one of which will make something of 
a splash in its (large, prosperous) industry. SelectDispatcher 
and friends are released under the McMillan Enterprises 4 line 
license (do what thou wilt; maintain the copyright notice; no 
warranty).

While these are not the PEPs I owe on stackless, they are 
part of the background material for those PEPs, particularly in 
demonstrating why some of us are so interested in seeing 
these facilities within core Python.

I apologize in advance to Christian for any misunderstandings 
or misinformation these pages may contain.

Enjoy!

- Gordon



From guido at python.org  Mon Nov  6 03:34:40 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:34:40 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>

[Guido]
> > - Integer division.  If we want to play by Paul Prescod's Language
> >   Evolution rules (PEP 5), we better get started on the first stage.
> >   E.g. we could introduce a // operator in 2.1 for integer division,
> >   and issue a warning when / is used for integers.  Then a year later
> >   (i.e., March 2002!) we could change / so that it returns a floating
> >   point number.

[MAL]
> +0... and then only, if there will be a tool to check Python
> source code for integer divides.

Hm.  I don't believe it's possible to write a tool to check for
integer divides by inspection of the source code only -- you have to
actually execute the code (with specific input, etc.).

However, with the right warnings framework in place (I'll post some
ideas about this under a separate subject), the Python interpreter
itself can be the perfect tool to do the checking.  Given that it's
pretty uncontroversial that 1/2 in Py3K should equal 0.5, I'd rather
get this started sooner than later.

Let me state some requirements:

- We don't want to break code in 2.1 that works in 2.0.

- It's okay to issue warnings though (my warnings proposal will limit
  the warnings to once per source line).

- In Py3K, 1/2 will yield 0.5 and users must use a different way to
  spell floor(x/y).

- Starting in 2.1, we want to issue warnings that encourage users to
  make their code Py3K-ready.

- We want (almost) all users to have converted their code to using
  1//2 instead of 1/2 by the time 2.n (the last 2.x version before
  Py3K is released) comes out, because unchanged code will silently
  change its meaning at the Py3K transition.

- Code that is Py3K-ready (in this respect) should trigger no warnings
  in Python 2.1.

Note: it is open for debate whether the result of x/y for integer (or
long integer) arguments should yield an integer (or long integer) in
those cases where the result *is* representable as such (e.g. 4/2).
It is possible that the numeric tower will render this problem moot --
but that depends on what happens to Moshe's PEP 228, and that's a much
longer story.  However, I think we can decide on the transition path
from / to // independent from the outcome of that discussion, since in
all cases it is clear that 1/2 will change in meaning.

Here's a concrete proposal (could be PEPped pretty easily):

- In Py3K, there will be two division operators:

  - x/y will always yield the mathematically expected result (possibly
    inexact, depending on how the numeric model is changed).

  - x//y will always yield the floor of the mathematical result, even
    for float arguments.  For complex arguments, this should raise an
    exception, just as int(1j) does today.

- Starting with Python 2.1, x//y will do the right thing (1//2 == 0).

- In 2.1, using x/y for ints and longs will issue a warning.

- In Py3K, there *might* be a special "backward incompatibility
  warning mode" that issues warnings when x/y is used for ints; 1/2
  will still yield 0.5 in this mode, but a warning will be issued.

This is a minimal proposal.  If Python were to grow a syntax for
pragmas, it would be nice to have a pragma saying "I want int/int to
do float division" in code that is Py3K ready; otherwise, this always
has to be spelled as float(x)/y to ensure proper working in 2.x as
well as in Py3K.

David Scherer proposed to spell this pragma as a "magical import"
(http://www.python.org/pipermail/idle-dev/2000-April/000138.html). This
is an OK idea (+0) because it can be assumed to fail in pre-2.1
installations and doesn't require new syntax.  I don't give it a +1
because it's a hack -- "import" doesn't quite convey the intention.
(Perl's "use" is better for this purpose!)

Tim didn't seem to like this idea much
(http://www.python.org/pipermail/python-dev/2000-April/010029.html).
His dislike seems based on the assumption that such annotations would
mention specific language (or interpreter) version numbers, which
could be interpreted as resisting progress (one moment 1.7 is a
forward looking version, but the next moment it is backward looking).
However if we use directives (I don't want to call them pragmas
because pragmas are supposed to be ignorable) to select specify
specific features, especially features for which there are only two
versions (the old way and the new way) then it seems okay to use such
a mechanism -- if we can agree on a syntax for directives.

Hm, reading Tim's post again it seems he's mostly objecting against
defaulting to an old version.  I have to agree with him there.
However what I'm proposing here is defaulting to the current version,
and allowing a way to select a "future version" as an alternative.

If we don't adopt directives, all we need to do (in Python 2.1) is add
a new opcode for //, keeping the opcode for / unchanged.  If we do
adopt directives, we'll need to introduce two new opcodes: one for the
new (always float) /, one for the new //, still keeping the old /
opcode with the 2.0 meaning.  The latter is what David Scherer
proposes (and what he needs for his students).  Note that it would be
okay to introduce directives in a later 2.x version -- this won't
break any previous code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:44:57 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:44:57 -0500
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>

[me]
> > - Class/type dichotomy???

[MAL]
> One thing that would probably be implementable is a way to
> maintain "instance" dictionaries for types (which are created
> on-demand whenever an assignment is made).
> 
> This would enable
> extending types with new methods and attributes. "Subclassing"
> could then be emulated by using new contructors which add the
> new or changed methods to each created type instance, e.g.
> 
> class myclose:
> 
>     def __init__(self, object, basemethod):
>         self.object = object
>         self.basemethod = basemethod
> 
>     def __call__(self):
>         print 'Closed file %s' % self.object
>         self.basemethod()
> 
> def myfile(filename):
>     f = open(filename)
>     # add/override attributes
>     f.newattribute = 1
>     # add/override methods
>     f.close = myclose(f, f.close)
>     return f
> 
> Types would have to be made aware of this possibility. Python
> could provide some helping APIs to make life easier for the
> programmer.

But this would require an extra pointer field for *all* built-in
types.  That would seriously impact the space requirements for ints
and floats!

As long as we're proposing hacks like this that don't allow smooth
subclassing yet but let you get at least some of the desired effects,
I'd rather propose to introduce some kind of metaclass that will allow
you to use a class statement to define this.  Thinking aloud:

import types
filemetaclass = metaclass(types.FileType)

class myfile(filemetaclass):

      def __init__(self, filename):
	  filemetaclass.__init__(filename)
	  self.newattribute = 1

      def close(self):
          myclose(self)
	  filemetaclass.close(self)

I'm not quite sure what kind of object "filemetaclass" here should be
or what exactly "metaclass()" should do, but it could create a new
type that has the lay-out of an existing file object, with an instance
dictionary (for newattribute) tacked on the end.  Later maybe (I'm
going to brainstorm with Jim Fulton about types and classes).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:48:47 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:48:47 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>

[me]
> > - Weak references.  This *is* a PEP, but there's no contents yet.  We
> >   could also try to implement (just) weak dictionaries.

[MAL]
> These already exist... http://www.handshake.de/~dieter/weakdict.html
> 
> mx.Proxy also has an implementation which support weak references.

Thanks.  For Fred to read...

> BTW, are these still needed now that we have GC ?

Yes, definitely.  Weak dicts are sometimes needed for situations where
a regular dict would keep objects alive forever.  E.g. we were made
aware of a "leak" in JPython that could only be fixed with weak dicts:
the Swing wrapper code has a global dict mapping widgets to callback
functions, and this keeps all widgets alive forever.  The Java GC
doesn't destroy the widgets, because they are still referenced from
the dict.  A weak dict solves this problem nicely (if it weren't that
JDK 1.1 doesn't support weak dicts).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Mon Nov  6 03:49:25 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 21:49:25 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.52668.117844.28459@bitdiddle.concentric.net>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
	<14853.52668.117844.28459@bitdiddle.concentric.net>
Message-ID: <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I thought we discussed this earlier and agreed that a little bit of
 > control over the process was healthy.  I would prefer to see all PEP
 > creation go through Barry.  We can circulate drafts in email before

  I think I hadn't actually noticed some of that email, or perhaps
there was a conversation I've forgotten.  Fine.  I still don't see a
problem for people creating PEPs; there's always email and the files
can be pubished at alternate locations before a number has been
assigned.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Mon Nov  6 04:06:29 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 22:06:29 -0500 (EST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
	<3A03E340.38F78FA2@lemburg.com>
	<200011060248.VAA04306@cj20424-a.reston1.va.home.com>
Message-ID: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > Yes, definitely.  Weak dicts are sometimes needed for situations where
 > a regular dict would keep objects alive forever.  E.g. we were made
 > aware of a "leak" in JPython that could only be fixed with weak dicts:
 > the Swing wrapper code has a global dict mapping widgets to callback

  That's a perfect example.  I've started working on some text
describing the motivation; hopefully I'll have that fleshed out and
checked in later this week.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:19:28 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:19:28 +1300 (NZDT)
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <200011060319.QAA00004@s454.cosc.canterbury.ac.nz>

Guido:

> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types

> But this would require an extra pointer field for *all* built-in
> types.

Ruby has an interesting solution to this. It keeps such
"extra" instance variables in a global data structure.

The Python version of this would be to have a special global
dict which maps instances of built-in types to dicts holding
their extra instance variables.

The keys in this dict would have to be weak references,
so that they wouldn't keep the objects alive. A flag would
be set in the object header so that, when the object was
deleted, the corresponding entry in the global dict
could be cleaned up.

The overhead would then be one bit in each object, and
one extra test when deleting an object.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:20:37 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:20:37 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <200011060320.QAA00007@s454.cosc.canterbury.ac.nz>

Guido:

> Here's a concrete proposal (could be PEPped pretty easily):

Looks good to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 04:35:26 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:35:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>

Before I fall asleep let me write up my ideas about the warning
framework.


Requirements:

- A C-level API that lets C code issue a warning with a single call
  taking one or two arguments, e.g. Py_Warning(level, message).  (The
  'level' argument is an example only; I'm not sure what if any we
  need.)

- After the first time a specific warning is issued for a given source
  code location, the overhead of calling Py_Warning() should be
  minimal.

- An equivalent Python level API, e.g. sys.warning(level, message).

- Flexible control over which warnings are printed or not; there
  should be a way to set this up from within the Python program but
  also from the command line or possible using an environment
  variable.

- Control over the disposition of warnings; by default they should be
  printed to sys.stderr but an alternative disposition should be
  supported (the mechanism could be either a different file or a
  different callback function).

- By default, a warning is printed once (the first time it is issued)
  for each source line where it is issued.

- For a specific warning at a specific source code location, it should
  be possible to specify the following alternatives:

  - Turn it into an exception

  - Don't print it at all

  - Print it each time it is issued

- It should also be possible to specify these alternatives:

  - For all warnings

  - For all warnings in a specific module or file

  - For all warnings at a specific source code location

  - For a specific warning everywhere in a specific module or file

  - For a specific warning everywhere in the program

  - For all warnings at/above/below (?) a specific level, if we use
    warning levels


Possible implementation:

- Each module can has a dictionary __warnings__ in its global
  __dict__, which records the state of warnings.  It is created as an
  emprt dict if it doesn't exist when it is needed.  The keys are
  (message, linenumber) tuples (the module or file is implicit through
  the use of the module's __dict__).  The value is None if no more
  action is needed for this particular warning and location.  Some
  other values may indicate the options "always print warning" (1?)
  and "raise an exception" (-1?).

- There's a list of "filters" in the sys module
  (e.g. sys.warningfilters) that is checked whenever a warning doesn't
  have a hit in the __warnings__ dict.  Entries in the filter list are
  (file, line, message, action) tuples.  (If we decide to implement
  warning levels, these must also be represented here somewhere.)

  - The file must be None or a shell matching pattern, e.g. "*foo";
    the ".py" suffix is optional; a partial pathname may be given too.
    So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
    "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
    the filter applies regardless of the file.

  - The line must be None or an integer.  If the file is None or "*"
    (indicating all files) the line must be None and is ignored.

  - The message must be a None or a string.  If it is None, the filter
    applies to all messages.  The message string may end in "*" to
    match all messages with the given text (up to the "*").

  - The action must be one of the following strings:

    - "ignore" -- the warning is never printed

    - "always" -- the warning is always printed

    - "once" -- the warning is printed for the first occurrence
      matching the filter

    - "module" -- the warning is printed for the first occurrence in
      each module matching the filter

    - "location" -- the warning is printed for the first occurrence at
      each source code location (module + line) matching the filter

    - "exception" -- the warning is turned into an exception whenever
      it matches the filter

    Note: implementation of "once" and "module" require additional
    state per filter entry; I'm not sure if that's worth the effort.

  - When the warning system decides to print a warning, it is given to
    sys.displaywarning(file, line, message), which by default does
    something like print >>sys.stderr, file, ":", line, ":", message

  - There should be a function sys.addwarningfilter(file, line,
    message, action) that appends items to sys.warningfilters after
    some sanity checking.

  - There should be command line options to specify the most common
    filtering actions, which I expect to include at least:

    - suppress all warnings

    - suppress a particular warning message everywhere

    - suppress all warnings in a particular module

    - turn all warnings into exceptions

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:34:26 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:34:26 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>
Message-ID: <200011060334.QAA00011@s454.cosc.canterbury.ac.nz>

Guido:

> the lookup in
> globals() can be skipped for builtins, but a local is accessed with
> *zero* dict lookups -- how would you do this while still supporting
> __builtin__.__dict__.open = foo?  have "hookable" dictionaries?

With fully static scoping, I envisage that all three kinds
of scope (local, module and builtin) would be implemented
in essentially the same way, i.e. as arrays indexed by
integers.

That being the case, all you need to do is arrange for the
__builtin__ module and the global scope to be one and the
same thing, and __builtin__.open = foo will work just fine
(assuming open() isn't one of the special inlinable functions).

Getting __builtin__.__dict__['open'] = foo to work as well
may require some kind of special dictionary-like object.
But then you're going to need that anyway if you want to
continue to support accessing module namespaces as if they
are dictionaries.

Whether it's worth continuing to support that in Py3k is
something that can be debated separately.

> integrable procedures (whatever they may be :-)

In the Revised^n Report, some builtin procedures are
declared to be "integrable", meaning that the compiler is
allowed to assume that they have their usual definitions
and optimise accordingly.

(This is quite important in Scheme, even more so than
in Python, when you consider that almost every operation in
Scheme, including '+', is a procedure call!)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 04:40:33 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:40:33 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060340.WAA04479@cj20424-a.reston1.va.home.com>

[me]
> > - Internationalization.  Barry knows what he wants here; I bet Martin
> >   von Loewis and Marc-Andre Lemburg have ideas too.

[MAL]
> We'd need a few more codecs, support for the Unicode compression,
> normalization and collation algorithms.

Hm...  There's also the problem that there's no easy way to do Unicode
I/O.  I'd like to have a way to turn a particular file into a Unicode
output device (where the actual encoding might be UTF-8 or UTF-16 or a
local encoding), which should mean that writing Unicode objects to the
file should "do the right thing" (in particular should not try to
coerce it to an 8-bit string using the default encoding first, like
print and str() currently do) and that writing 8-bit string objects to
it should first convert them to Unicode using the default encoding
(meaning that at least ASCII strings can be written to a Unicode file
without having to specify a conversion).  I support that reading from
a "Unicode file" should always return a Unicode string object (even if
the actual characters read all happen to fall in the ASCII range).

This requires some serious changes to the current I/O mechanisms; in
particular str() needs to be fixed, or perhaps a ustr() needs to be
added that it used in certain cases.  Tricky, tricky!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 04:48:22 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:48:22 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: Your message of "Sun, 05 Nov 2000 21:21:47 EST."
             <3A05CF6B.32107.2CAF777@localhost> 
References: <3A05CF6B.32107.2CAF777@localhost> 
Message-ID: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>

> I have put up 6 pages of information about stackless at
> 
> http://www.mcmillan-inc.com/stackless.html

Gordon, thanks for doing this.  I still have a review of Stackless on
my TODO list.  It takes a serious chunk of my time to do it justice,
and this continues to be a problem, but the existience of your
overview certainly helps.  I still think that the current Stackless
implementation is too complex, and that continuations aren't worth the
insanity they seem to require (or cause :-), but that microthreads and
coroutines *are* worth having and that something not completely unlike
Stackless will be one day the way to get there...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Mon Nov  6 04:55:17 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 22:55:17 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
References: <3A05CF6B.32107.2CAF777@localhost>
	<200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <14854.11173.601039.883893@bitdiddle.concentric.net>

[Changed discussion list from general python-list to specific
stackless.] 

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

  >> I have put up 6 pages of information about stackless at
  >> http://www.mcmillan-inc.com/stackless.html

  GvR> Gordon, thanks for doing this.  I still have a review of
  GvR> Stackless on my TODO list.  It takes a serious chunk of my time
  GvR> to do it justice, and this continues to be a problem, but the
  GvR> existience of your overview certainly helps.  I still think
  GvR> that the current Stackless implementation is too complex, and
  GvR> that continuations aren't worth the insanity they seem to
  GvR> require (or cause :-), but that microthreads and coroutines
  GvR> *are* worth having and that something not completely unlike
  GvR> Stackless will be one day the way to get there...

I tend to agree with you, Guido.  I think we would do well to
purposefully omit continuations from the Python language.  There seems
to be little need for a facility to implement arbitrary control
structures in Python.  If Python support coroutines and microthreads,
I am not sure what else would be needed.

It would be very helpful if the PEPs on Stackless could address this
issue.  One way to address it is to ask these questions: What new
control structures do users want in Python?  How best can they be
implemented?  Are continuations necessary to implement them or are
there other options?

The sort of implementation complexity that I worry about with
Stackless is, e.g. clean interaction with the C stack.  If a Python C
API call is made that pushes a C stack frame, e.g. PyObject_Compare,
then a continuation stored before that call can no longer be invokved.
The problem is that continuations break the notion a C API call will
always return an answer; they create a situation in which the C call
that is made should never return, because control is transferred to
the continuation.  I assume Stackless raises an error in this case,
but this seems pretty messy: How do we right a language spec that
explains when an error will occur without appealing to the
language implementation?

Jeremy




From petrilli at amber.org  Mon Nov  6 05:06:35 2000
From: petrilli at amber.org (Christopher Petrilli)
Date: Sun, 5 Nov 2000 23:06:35 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Sun, Nov 05, 2000 at 10:06:29PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060248.VAA04306@cj20424-a.reston1.va.home.com> <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>
Message-ID: <20001105230635.A18694@trump.amber.org>

Fred L. Drake, Jr. [fdrake at acm.org] wrote:
> 
> Guido van Rossum writes:
>  > Yes, definitely.  Weak dicts are sometimes needed for situations where
>  > a regular dict would keep objects alive forever.  E.g. we were made
>  > aware of a "leak" in JPython that could only be fixed with weak dicts:
>  > the Swing wrapper code has a global dict mapping widgets to callback
> 
>   That's a perfect example.  I've started working on some text
> describing the motivation; hopefully I'll have that fleshed out and
> checked in later this week.

Another example is some of the things in Zope use back-references for
ease of traversability (or worse keep weird counts hanging around).
Alot of these are negated by ZODB's ability to break cycles, but... a
lot of data structures would be hugely better from an architecture
perspective if we had a native weak reference.  

Chris
-- 
| Christopher Petrilli
| petrilli at amber.org



From est at hyperreal.org  Mon Nov  6 05:27:52 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Sun, 5 Nov 2000 20:27:52 -0800 (PST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 5, 2000 09:48:47 pm"
Message-ID: <20001106042752.25360.qmail@hyperreal.org>

Guido van Rossum discourseth:
> [MAL]
> > These already exist... http://www.handshake.de/~dieter/weakdict.html
> > 
> > mx.Proxy also has an implementation which support weak references.
> 
> Thanks.  For Fred to read...

He may want to also see my http://www.hyperreal.org/~est/python/weak
(also registered in the vaults), an implementation of weak dicts and
proxy maps that doesn't require the contained objects to be
subclassed.  It even has documentation in standard format (!)
including some words about motivation.

One simple implementation approach to make it work for all objects
might be to have:

int PyObject_PushWeakHandler(PyObject *o,
                             void (*handler)(PyObject *o, PyObject *data),
                             PyObject *data);

When an object is deallocated all its handlers would be called on the
object and the data that was registered with the handler.  I believe
this would make weak dicts a very simple extension module.  With this
approach I suspect DATA should be incref'd by the call to
PyObject_PushWeakHandler() and decref'd after the associated handler
is called.

Best,

Eric



From est at hyperreal.org  Mon Nov  6 05:39:29 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Sun, 5 Nov 2000 20:39:29 -0800 (PST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net> from Jeremy
 Hylton at "Nov 5, 2000 10:55:17 pm"
Message-ID: <20001106043929.2515.qmail@hyperreal.org>

Jeremy Hylton discourseth:
> 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

This point is particularly worrisome to me because of a common pattern
I see in my own Python development work.  I'll define a class which is
parameterized with some callbacks.  Sometimes, when profiling reveals
the need, I'll move these classes to C.  If the client of the class is
using continuations via its callbacks, it may suddenly break.  This
seems a step back in the modularity I can count on in my program
components.

..and I say this as a long-time, die-hard Schemer. :)

I definitely pine for micro-threads in some of my application domains
though.

Eric



From greg at cosc.canterbury.ac.nz  Mon Nov  6 05:50:09 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 17:50:09 +1300 (NZDT)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy at alum.mit.edu>:

> Are continuations necessary to implement them or are
> there other options?

I think you'll find that any implementation of microthreads
or coroutines or whatever you want to call them, that
doesn't rely on playing nonportable tricks with the C
stack, will be just as mindbending as Stackless.

> The problem is that continuations break the notion a C API call will
> always return an answer;

So do threads or coroutines. As soon as you have multiple
threads of control, you have the chance that one of them
will switch to another and never get back.

> I assume Stackless raises an error in this case,
> but this seems pretty messy

This messiness isn't the fault of Stackless itself, but of the
large amount of code which *hasn't* been converted to the
Stackless Way. If the whole of Python and everything that
it calls were made truly stackless, the problem would not
arise.

Doing so, however, would not be fun. It wouldn't be fun
for any future extension writers, either.

I can't see any way out of this. Continuations/coroutines/
microthreads are all basically the same thing underneath, and
they imply an execution model that just doesn't fit well
with C.

Maybe we need to reimplement Python in Scheme, and then
feed it through a good Scheme compiler. SPython, anyone? 

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From moshez at zadka.site.co.il  Mon Nov  6 14:52:55 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 06 Nov 2000 15:52:55 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Sun, 05 Nov 2000 20:19:02 EST." <200011060119.UAA03952@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <200011060119.UAA03952@cj20424-a.reston1.va.home.com> 
Message-ID: <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>

[GvR]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

I think the solution is obvious -- stop going offline to entertain guests.

[GvR]
> I was hoping that Tim would chime in

Me too. I even tried to drag him in by mentioning 754.

[GvR]
> I like the idea of a PEP to rework the numeric model.  I think that
> Moshe, being a mathematician by training, will make a good editor.  I
> think that this has a snowball's chance in hell to make it into Python
> 2.1 however -- there are many deep issues, and it's a bad idea to
> experiment too much in the released version of the language.

The reason the PEP was written now was because you started making sounds
of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
to show a less dangerous (and less innovative) way of getting similar
usability.


[GvR]
> (1) As long as the internal representation is not the same as what is
>     commonly printed, there will be surprises -- with rationals just
>     as much as with floating point.  There are issues with decimal
>     floating point too, but they are only the issues having to do with
>     loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
>     1.0) and not with loss of precision in the printing, where most of
>     the "bug reports" we get seem to concentrate.

My PEP does not yet deal with either written or inputted representation.

[GvR]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

Those calculations, if performed with floating points, would often build
up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
to be wildly inaccurate.

[GvR]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

You'll note that my PEP does not mention floating point explicitly --
and once again I mention that my PEP does not yet deal with number
literals. All it allows (not requires) is for things like math.sqrt()
to return inexact results. Naive implementations (which we might use)
would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
course, good implementations of math.sqrt() would realize that 1 has
an exact root, but that might be too hard to do for not enough gain.

[GvR]
> If we take its meaning literally, the isreal() function should only
> return true for numbers for which isrational() is also true:

Correct -- in my current model. If you later add things like constructive
reals, that is no longer true: if I have a constructive Pi, it's not 
rational. 

[GvR]
> mathematically speaking, real numbers that aren't also rational don't
> have an easy finite representation, since they are numbers like
> sqrt(2) or pi.

But numbers don't have to have a finite (periodic really) representation
to be representable in the computer: what about infinite continued fractions,
for example?

[GvR]
> Finally, the PEP doesn't talk about how the numeric model can be
> extended

That's because its rich enough not to need it. 
Let me explain exactly what I mean: as long as all field operations between
Python numbers give honest to god Python numbers, then everything else
can be solved with Python's current model of coercions, and can be solved
well when the coercion PEP will be written.

[GvR]
> I've
> heard of wild floating point representations that make multiplication
> and division really cheap but addition a pain, rather than the other
> way around;

Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
that instead of inexact() and math.{...}. How these numbers interact
with builtin Python numbers is your responsibility -- and that's what
the coercions are for.

Same goes for gmp: as long as you're not expecting to be able to change
10000000000+10000000000 to be a gmp long rather then a Python long, then
there shouldn't be a problem.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From paulp at ActiveState.com  Mon Nov  6 07:24:27 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Sun, 5 Nov 2000 22:24:27 -0800 (PST)
Subject: [Python-Dev] Warnings PEP
Message-ID: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com>

Abstract

    This PEP describes a generalized warning mechanism for Python 2.1. The
    primary purpose of this mechanism is to alert the programmer or user
    of a program to potential or likely errors which, for whatever reason,
    are not considered exception-worthy. For example, this might be done
    to keep old code working during a transitional period or to alert the
    programmer or user of a recoverable error.

Syntax

    assert >> cls, test[[[, arg], arg]...]

    "cls" may be any callable object that takes a list as a single
    argument argument list and returns an object with the required
    attributes "get_action" and "format_message"
    
       * get_action() -> "warn"|"error"|"suppress"
       * format_message() -> string

    A provided base class implements these methods in a reusable
    fashion. Warning creators are encouraged to merely subclass.

    This extended form of the assertion statement calls the assertion
    handler code in the new "assertions" module. 

    The semantics of the built-in assertion handler are defined by the
    following code. It should be exposed in a new "assertions" module.

    def handle_assertion(cls, message = ""):
        "This code is called when an assertion fails and cls is not None"

	obj = cls(message)
	action = obj.get_action()
            
        if action=="error":
            *** existing assertion code ***
        elif action=="warn":
	    sys.stderr.write(obj.format_message())
        elif action=="suppress":
            pass
        else:
            assert action in ["warn","error","suppress"]

    Even if handle_assertion is implemented in C, it should be exposed as
    assertions.handle_assertion so that it may be overriden. 

    The generic warning base class is defined below:

    class Assertion:
        def __init__(self, *args):
	    if len(args) == 1:
	       self.args = args[0]
	    else:
	        self.args = args

	def format_message(self):
            sys.stderr.write("%s: %s" %(obj.__name__, self.args))

	def get_action(self):
	    return (self.get_user_request(self.__class__) 
                                or sys.default_warning_action)

        def get_user_request(self, cls):
            if cls.__name__ in sys.errors:
                return "error"
	    elif cls.__name__ in sys.warnings:
                return "warn"
            elif cls.__name__ in sys.disabled_warnings:
                return "suppress"
	    for base in cls.__bases__:
	        rc = self.get_user_request(base)
		if rc:
		    return rc
            else:
                return None

    The Assertion class should be implemented in Python so that it can be 
    used as a base class.

    Because this code inherently implements "warning state inheritance",
    it would be rare to override any of the methods, but this is possible
    in exceptional circumstances.

Command line

    By default the special variables have the following contents:

    sys.warnings = []
    sys.errors = []
    sys.suppress = []
    sys.default_warning_action = "warn"
    
    These variables may be changed from the command line. The command line
    arguments are interpreted as described below:

    -w XXX => sys.warnings.append("XXX")
    -e XXX => sys.errors.append("XXX")
    -no-w XXX => sys.suppress.append("XXX")
    -wall => sys.default_warning_action => "warn"
    -eall => sys.default_warning_action => "error"
    -no-wall => sys.default_warning_action => "suppress"

    As per the code above, errors take precedence over warnings and
    warnings over suppressions unless a particular assertion class
    specifies otherwise.

Built-in warning objects:

    class exception_in_del(Assertion):
        "An exception was ignored in an __del__ method"

    class deprecation(Assertion):
        "This feature may be removed in a future version of Python."

    class dubious(Assertion):
        "There is a common error associated with this feature."

    These class definitions are part of the "Assertion" module. They
    should only ever be used when there exists a way for the programmer to 
    accomplish the same thing without triggering the warning. For instance
    the way to suppress the deletion exception is to trap exceptions in
    __del__ methods with a try/except block.





From tim_one at email.msn.com  Mon Nov  6 07:34:38 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 01:34:38 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>

[Guido]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

The scary thing is which one of those you clearly enjoyed more <wink>.

> I was hoping that Tim would chime in, but he's apparently taken the
> weekend off -- very much out of character. :-)

Exactly in character, alas:  I was obsessed with my new cable modem
connection.  I had years of stuff to learn about firewalls in two days --
not to mention years of pornography to download in one <wink>.

Some quickies for now:

+ Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
numeric model forever.

+ Everyone has IEEE-754 fp hardware today; some people actually want to use
it; Moshe doesn't, but whatever revamping we get needs to allow others their
delusions too.

> ...
> For example, Tim has conjectured that using binary floating point will
> always be a problem for the "unwashed masses" -- the only thing they
> might understand is decimal floating point,

At first glance, yes.  Complaints traced to the "binary" part of "binary fp"
vastly outnumber complaints due to the "fp" part *and* integer division
combined, on both Python-Help and the Tutor list.  So if we want to know
what actually trips up newbies, they've been telling us for years.  Decimal
fp would silence most of those complaints; but rationals would silence them
too (provided they're *displayed* in rounded decimal fp notation (restart
"str" vs "repr" rant, and that the interactive prompt uses the wrong one,
and ditto str(container))), plus a few more (non-obvious example:

    (1/49)*49

does not equal 1 in either decimal or IEEE-754 binary double fp, but does
equal 1 using rationals).

Note that Mike Cowlishaw (REXX's dad) has been working on a scheme to merge
REXX's decimal fp with IEEE-854 (the decimal variant of IEEE-754):

    http://www2.hursley.ibm.com/decimal/

I'll talk to Jim Fulton about that, since Cowlishaw is pushing a BCD variant
and Jim was wondering about that (around the change of the year, for use-- I
presume --in Zope).

Note also IBM's enhanced BigDecimal class for Java:

    http://www2.hursley.ibm.com/decimalj/

> ...
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)
> -- there are only inexact operations.  I'm not sure what to do with
> this though.

IEEE-754 defines exactly what to do with this, for binary floats (and your
hardware has an "inexact result" flag set or not after every fp operation).
Conversion of the string "1.0" to float must not set it; conversion of "0.1"
must set it; and similarly for + - * / sqrt:  "inexact result" gets set
whenever the infinitely precise result differs from the computed result.  So
inexactness there is neither a property of types nor of numbers, but of
specific computations.  Extreme example:

    x = 1./3.   # inexact
    y = x-x     # exact result (from inexact inputs!)

I know that this version (operation-based) of inexactness can be useful.  I
see almost no earthly use for calling every number of a given type inexact.
Tagging individual numbers with an exact/inexact bit is an extremely crude
form of interval arithmetic (where the intervals are single points or
infinite).

> ...
> I'll leave it to Tim to explain why inexact results may not be close
> to the truth.

> Tim may also break a lance for IEEE 754.

Somebody else on c.l.py offered to write a 754 PEP; delighted to let them
have it.

if-you-ever-approximate-people-will-get-confused-
    but-if-you-don't-they'll-run-out-of-time-or-memory-ly y'rs  - tim





From paul at prescod.net  Mon Nov  6 08:17:52 2000
From: paul at prescod.net (Paul Prescod)
Date: Sun, 05 Nov 2000 23:17:52 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A065B20.BBD1C1E3@prescod.net>

It's just coincidence that I was working on warnings at the same time
you were.

Our proposals seem to have almost no overlap. I think mine does less,
but is also much simpler. I'm always nervous about over-engineering
rather than incremental development.

-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From pf at artcom-gmbh.de  Mon Nov  6 10:05:12 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Mon, 6 Nov 2000 10:05:12 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 5, 2000  9:34:40 pm"
Message-ID: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

[Guido]:
> David Scherer proposed to spell this pragma as a "magical import"
> (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
at module level as a backward compatible method to introduce module level
pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
I still like David Scherers proposal very much.

[David]:
> I actually implemented 1/2==0.5 in Python 1.5.2, complete with a
> module-level backward compatibility flag.  The flag made an unusual use of
> the "global" statement, which appears to be accepted at toplevel by 1.5.2
> without any effect.  Therefore a statement like "global olddivision" will be
> silently ignored by 1.5.2 and earlier, and will result in the old behavior
> under my patch.  "global" even has the right sort of sound for module-level
> options :)
> 
> An outline of what I did:
> 
> 1.  Add new opcode BINARY_FRACTION to opcode.h and dis.py
> 2.  Add new flag "int c_olddivision" to struct compiling in compile.c
> 3.  Set c_olddivision to base->c_olddivision or 0 in jcompile
> 4.  Check for "global olddivision" outside a function definition in
> com_global_stmt, and set c_olddivision=1
> 5.  Check c_olddivision in com_term, and generate either BINARY_DIVISION or
> BINARY_FRACTION
> 6.  Add PyNumber_Fraction to abstract.h, and define it in abstract.c to
> explicitly check for a pair of integers and do float division
> 7.  Add a BINARY_FRACTION case to ceval.c, which calls PyNumber_Fraction
> instead of PyNumber_Divide.

BTW:  I think the "symbol" '//' is incredible ugly and starting with
IBMs JCL years ago all languages I encountered, that used this symbol
for something, did suck in some way or another.  I would appreaciate
very much, if it could be avoided alltogether to add a symbol '//'
to Python.  '//' will look like a comment delimiter to most people today.

Using a new keyword like 'div' in the tradition of languages like 
Modula-2 looks far more attractive to me.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Mon Nov  6 10:14:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 10:14:12 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>
Message-ID: <3A067664.22D09D03@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Internationalization.  Barry knows what he wants here; I bet Martin
> > >   von Loewis and Marc-Andre Lemburg have ideas too.
> 
> [MAL]
> > We'd need a few more codecs, support for the Unicode compression,
> > normalization and collation algorithms.
> 
> Hm...  There's also the problem that there's no easy way to do Unicode
> I/O.  I'd like to have a way to turn a particular file into a Unicode
> output device (where the actual encoding might be UTF-8 or UTF-16 or a
> local encoding), which should mean that writing Unicode objects to the
> file should "do the right thing" (in particular should not try to
> coerce it to an 8-bit string using the default encoding first, like
> print and str() currently do) and that writing 8-bit string objects to
> it should first convert them to Unicode using the default encoding
> (meaning that at least ASCII strings can be written to a Unicode file
> without having to specify a conversion).  I support that reading from
> a "Unicode file" should always return a Unicode string object (even if
> the actual characters read all happen to fall in the ASCII range).
> 
> This requires some serious changes to the current I/O mechanisms; in
> particular str() needs to be fixed, or perhaps a ustr() needs to be
> added that it used in certain cases.  Tricky, tricky!

It's not all that tricky since you can write a StreamRecoder
subclass which implements this. AFAIR, I posted such an implementation
on i18n-sig.

BTW, one of my patches on SF adds unistr(). Could be that it's
time to apply it :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Mon Nov  6 10:45:21 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 6 Nov 2000 11:45:21 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011061142050.4175-100000@sundial>

On Sun, 5 Nov 2000, Guido van Rossum wrote:

>   - The file must be None or a shell matching pattern, e.g. "*foo";
>     the ".py" suffix is optional; a partial pathname may be given too.
>     So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
>     "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
>     the filter applies regardless of the file.

How about "file" must be None or a callable, and if it's a callable,
it will be called to check whether to print? If I'll want fnmatch,
I know where to find it.

>   - The message must be a None or a string.  If it is None, the filter
>     applies to all messages.  The message string may end in "*" to
>     match all messages with the given text (up to the "*").

Same remark. If I want re, I know where to find it.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Mon Nov  6 11:42:34 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 6 Nov 2000 12:42:34 +0200 (IST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>

[Tim Peters]
> Some quickies for now:
> 
> + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
> numeric model forever.

OK, I hope this e-mail address reaches him: I got it off his webpage.

[Tim Peters]
> + Everyone has IEEE-754 fp hardware today; some people actually want to use
> it; Moshe doesn't, but whatever revamping we get needs to allow others their
> delusions too.

My proposal has nothing *against* 754 either. For example, "all inexact
operations are done compliant to 754" is a perfectly acceptable addition.

[Tim Peters, about rationals]
> (provided they're *displayed* in rounded decimal fp notation (restart
> "str" vs "repr" rant, and that the interactive prompt uses the wrong one,
> and ditto str(container))),

Tim, that's the other PEP <wink>

[Tim Peters]
> IEEE-754 defines exactly what to do with this, for binary floats (and your
> hardware has an "inexact result" flag set or not after every fp operation).

Cool! I didn't know about that.

[Tim Peters]
> Conversion of the string "1.0" to float must not set it; conversion of "0.1"
> must set it; and similarly for + - * / sqrt:  "inexact result" gets set
> whenever the infinitely precise result differs from the computed result.

Is there some API for it in C? If not, we might as well assume that any
floating point number is inexact.

[Tim Peters]
> So
> inexactness there is neither a property of types nor of numbers, but of
> specific computations.  Extreme example:
> 
>     x = 1./3.   # inexact
>     y = x-x     # exact result (from inexact inputs!)
> 
> I know that this version (operation-based) of inexactness can be useful.  I
> see almost no earthly use for calling every number of a given type inexact.
> Tagging individual numbers with an exact/inexact bit is an extremely crude
> form of interval arithmetic (where the intervals are single points or
> infinite).

The tagging is whether you *want* exact operations or inexact operations.

I.e.:

1.0/3 --> inexact
1/3 --> exact.

Well, this is the classical definition of "type": what do the operations
mean? Which is why I need the inexact() function in my PEP.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov  6 13:13:06 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:13:06 +0100
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>  
	            <3A03DBD2.7E024978@lemburg.com> <200011060125.UAA03986@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A052.1990A512@lemburg.com>

Guido van Rossum wrote:
> 
> Marc-Andre:
> > I can take over the coercion PEP: I've been working
> > on this before (see the proposal on my Python Pages).
> 
> Thanks, excellent (although I haven't seen your proposal yet).
> 
> > I would also like to know whether the PEP-0224 will be considered
> > for 2.1 if I update the patch to make it a little more robust
> > w/r to the problems mentioned in that PEP -- I'd really like
> > to see this in Python soon, since it makes documenting Python
> > programs so much easier.
> 
> I "kinda" like the idea of having attribute docstrings (meaning it's
> not of great importance to me) but there are two things I don't like
> in your current proposal:
> 
> 1. The syntax you propose is too ambiguous: as you say, stand-alone
>    string literal are used for other purposes and could suddenly
>    become attribute docstrings.

This can be fixed by introducing some extra checks in the
compiler to reset the "doc attribute" flag in the compiler
struct.
 
> 2. I don't like the access method either (__doc_<attrname>__).

Any other name will do. It will only have to match these
criteria:

* must start with two underscores (to match __doc__)
* must be extractable using some form of inspection (e.g. by using
  a naming convention which includes some fixed name part)
* must be compatible with class inheritence (i.e. should be
  stored as attribute)

> > Note that I won't get around to do much work on these before
> > January... way too busy at the moment :-/
> 
> That's a problem -- we really want to have the PEPs ready for review
> by mid December.  This will also be a problem for the coercion PEP --
> if you think you won't be able to work on it before then, I'd prefer
> to find another (co-)author.

I think I'll need a co-author for the PEPs -- I have high-priority
project running which has a deadline in mid-December too.

Much food-for-thought is already available (see the PEP 214 and
the coercion proposal on my Python Pages) and I should find
time for some review.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Mon Nov  6 13:25:26 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:25:26 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A336.5FD1EBC7@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Class/type dichotomy???
> 
> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types (which are created
> > on-demand whenever an assignment is made).
> >
> > This would enable
> > extending types with new methods and attributes. "Subclassing"
> > could then be emulated by using new contructors which add the
> > new or changed methods to each created type instance, e.g.
> >
> > class myclose:
> >
> >     def __init__(self, object, basemethod):
> >         self.object = object
> >         self.basemethod = basemethod
> >
> >     def __call__(self):
> >         print 'Closed file %s' % self.object
> >         self.basemethod()
> >
> > def myfile(filename):
> >     f = open(filename)
> >     # add/override attributes
> >     f.newattribute = 1
> >     # add/override methods
> >     f.close = myclose(f, f.close)
> >     return f
> >
> > Types would have to be made aware of this possibility. Python
> > could provide some helping APIs to make life easier for the
> > programmer.
> 
> But this would require an extra pointer field for *all* built-in
> types.  That would seriously impact the space requirements for ints
> and floats!

True.

> As long as we're proposing hacks like this that don't allow smooth
> subclassing yet but let you get at least some of the desired effects,
> I'd rather propose to introduce some kind of metaclass that will allow
> you to use a class statement to define this.  Thinking aloud:
> 
> import types
> filemetaclass = metaclass(types.FileType)
> 
> class myfile(filemetaclass):
> 
>       def __init__(self, filename):
>           filemetaclass.__init__(filename)
>           self.newattribute = 1
> 
>       def close(self):
>           myclose(self)
>           filemetaclass.close(self)
> 
> I'm not quite sure what kind of object "filemetaclass" here should be
> or what exactly "metaclass()" should do, but it could create a new
> type that has the lay-out of an existing file object, with an instance
> dictionary (for newattribute) tacked on the end.  Later maybe (I'm
> going to brainstorm with Jim Fulton about types and classes).

I think the problem we currently have with subclassing types
is strongly related to the fact that all Py<type>_Check()
macros only work on a address compare basis.

If we could find a way to change this to some kind of (very)
fast different lookup scheme we'd open a door which could
lead to making subclassing of types a whole lot easier.

<Brainstorming>

Perhaps a simple indirection could help...
instead of obj->ob_type == PyInteger_Type we'd write
obj->ob_type->base_type == PyInteger_Type_ID.

A subclass could then identify itself as integer subclass
by setting the base_type id to PyInteger_Type_ID. It would
of course have to publish the same internal structure in
order to remain compatible to the PyInteger_*() API, but there
would be a possibility to extend the object struct and
slots could also be replaced with new ones.

</>

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jim at interet.com  Mon Nov  6 14:11:25 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Mon, 06 Nov 2000 08:11:25 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com>
Message-ID: <3A06ADFD.5C735BBA@interet.com>

"M.-A. Lemburg" wrote:

I am happy to modify zipfile.py as desired by this group.
 
> I'm having trouble opening ZIP files created using InfoZIP's
> zip utility (which uses zlib) with zipfile.py:
> 
> >>> x = z.read('README')
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
>   File "/home/lemburg/lib/zipfile.py", line 242, in read
>     bytes = dc.decompress(bytes)
> zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> 
> Is this due to the installed zlib on my system being incompatible,
> or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> zip version 2.2.

I "borrowed" the zlib code from someone else, and it uses
undocumented features.  Since WinZip seems happy with the compression
I left it alone.  I wouldn't be surprised if other utilities have
problems, especially if they use zlib.  I am not sure what to
do about this.  My starting view is that there is only one
compression method, and if WinZip accepts it, then InfoZip
is wrong.  Is there perhaps a version difference in zlib
in Python vs InfoZip?

Could you send me an InfoZip zip file?

> Also, I wonder why zipfile forces the mode flag to be 'r',
> 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> to the mode flag instead ?!

The 'b' is not necessary, as it is always added by zipfile.py
when the file is opened.  The [rwa] each do something
different, and aren't really the same as a file open.  I
am not sure what you mean here.
 
> The ZipFile is also missing some kind of method which
> extracts files in the ZIP archive to a file-like object. This
> would be very useful for extracting large files from a ZIP
> archive without having to first read in the whole file into
> memory.

The "read(self, name)" method returns the bytes, which you can
write to a file if you want.  What new method would you like?

JimA



From gvwilson at nevex.com  Mon Nov  6 14:39:37 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 6 Nov 2000 08:39:37 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.10.10011060823080.26810-100000@akbar.nevex.com>

> Jeremy wrote:
> I tend to agree with you, Guido.  I think we would do well to purposefully
> omit continuations from the Python language.  There seems to be little need
> for a facility to implement arbitrary control structures in Python.  If
> Python support coroutines and microthreads, I am not sure what else would be
> needed.

I just finished reading Thomas and Hunt's "Programming Ruby" (the first book in
English on the language).  It's pretty clear that their favorite language
feature in Ruby is the block, which is any group of statements inside either
braces or do...end.  Blocks are invoked using the 'yield' construct, and can
take any number of arguments (enclosed in bars):

    def fibUpTo(max)
      i1, i2 = 1, 1
      while i1 <= max
        yield i1                      # 'call' the block
        i1, i2 = i2, i1 + i2
      end
    end

    fibUpTo(1000) { |f| print f, " " }

Most built-in types have iterators that understand blocks:

    [1, 3, 5].each { |i| puts i }     # prints 1, 3, and 5 on separate lines

Programmers can use blocks and 'yield' to create new control structures, subject
to the limitation that a statement can only be given one block (which means that
a multi-way interleaving loop can't be built out of blocks).  It would be
interesting to see how many of their examples can be done (easily) with
stackless...

Thanks,
Greg





From mal at lemburg.com  Mon Nov  6 15:00:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 15:00:33 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com>
Message-ID: <3A06B981.B3D73091@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> I am happy to modify zipfile.py as desired by this group.
> 
> > I'm having trouble opening ZIP files created using InfoZIP's
> > zip utility (which uses zlib) with zipfile.py:
> >
> > >>> x = z.read('README')
> > Traceback (innermost last):
> >   File "<stdin>", line 1, in ?
> >   File "/home/lemburg/lib/zipfile.py", line 242, in read
> >     bytes = dc.decompress(bytes)
> > zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> >
> > Is this due to the installed zlib on my system being incompatible,
> > or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> > zip version 2.2.
> 
> I "borrowed" the zlib code from someone else, and it uses
> undocumented features.  Since WinZip seems happy with the compression
> I left it alone.  I wouldn't be surprised if other utilities have
> problems, especially if they use zlib.  I am not sure what to
> do about this.  My starting view is that there is only one
> compression method, and if WinZip accepts it, then InfoZip
> is wrong.  Is there perhaps a version difference in zlib
> in Python vs InfoZip?
> 
> Could you send me an InfoZip zip file?

As it turned out it was a false alarm: Python must have picked
up an old zlibmodule.so from somewhere which caused the
problem. A clean rebuild made zipfile work with my InfoZIP
files (all my Python Pages ZIP-archives are built using InfoZIP 2.2).

Now I get this error after working in interactive Python
mode with zipfile:

Exception exceptions.AttributeError: 
"ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

I guess the __del__ method has to be a bit more careful about
what it expects to find... sometimes the builtins may have
already been garbage collected.

> > Also, I wonder why zipfile forces the mode flag to be 'r',
> > 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> > to the mode flag instead ?!
> 
> The 'b' is not necessary, as it is always added by zipfile.py
> when the file is opened.  The [rwa] each do something
> different, and aren't really the same as a file open.  I
> am not sure what you mean here.

Sorry, I was only reading the code and got the impression that
mode testing was done on the complete string you pass to the
constructor ... I missed the line "key = mode[0]".
 
> > The ZipFile is also missing some kind of method which
> > extracts files in the ZIP archive to a file-like object. This
> > would be very useful for extracting large files from a ZIP
> > archive without having to first read in the whole file into
> > memory.
> 
> The "read(self, name)" method returns the bytes, which you can
> write to a file if you want.  What new method would you like?

I would like a method .copy(self, name, output) which
reads the file name from the ZIP archive and writes it directly to
the file-like object output. This should copy the file in chunks
of say 64kB in order to reduce memory load.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From nas at arctrix.com  Mon Nov  6 08:44:53 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 5 Nov 2000 23:44:53 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06A336.5FD1EBC7@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 01:25:26PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com>
Message-ID: <20001105234453.A8255@glacier.fnational.com>

On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> I think the problem we currently have with subclassing types
> is strongly related to the fact that all Py<type>_Check()
> macros only work on a address compare basis.

I don't think this is the problem, although it is closely
related.  The problem is that the interpreter uses these type
checks to special case the handling of certain types.
PyInstance_Check() is a big offender.

Behavior should be based solely on the type structure.  Extension
types would then be able to behave exactly like any other builtin
type.  Your coercion proposal and David's rich comparisions both
remove some of this special casing based on type.

  Neil



From gmcm at hypernet.com  Mon Nov  6 15:54:57 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 09:54:57 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
References: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <3A067FF1.10498.57C83EB@localhost>

Jeremy wrote:

> ...  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.

I'm not sure what you mean here. So far, no one has asked for 
anything to get added to the *language* (although generators 
and coroutines could be made nicer with a bit of keyword 
support, it's not necessary).
 
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?  How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?

If by "control structure" you mean for / while / do type things, 
well, Python's not a functional language, and stackless / 
continuations won't grow you any new ones.

OTOH, if threads added new "control structures", then yes, 
continuations adds new ones (although from a language 
viewpoint, they don't look like anything new).
 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.

You mean when C code (called by Python) ends up calling 
eval_code2, and the Python code so invoked wants to use a 
continuation which "resides" in some other invocation of 
eval_code2? Yes, Christian gives you an error at that point (I 
believe; I've never stumbled on this personally).

> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation. 

Well, sys.exit breaks that notion too :-). If an API says that a 
return is required, it's a programmer error not to return 
something.

I think it's universally agreed that there should be a "coroutine" 
API. Perhaps it is best done from scratch, or perhaps it is to 
the continuation module as threading is to thread. But I think 
Greg Ewing is right - it doesn't make much difference as far as 
stackless is concerned.

> I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

I'm not understanding. 

In practical terms, I toasted my machine more times and more 
thoroughly playing with the thread module than I have with the 
continuation module. In that respect, continuations are a good 
deal safer - they don't have any effect on the state of your OS.

If you're playing with the raw "go-to" features of the 
continuation module, it's very easy to screw up. That's simply 
because code has to "balance", somehow. Call and return are 
the universally accepted way of doing that in the procedural 
community. But they're just a protocol based on go-to. The go-
to is still there, even if it's in the chip's microcode.

Christian's continuation module exposes a bunch of primitives. 
Most of the material on the pages I put up is concerned with 
how to mix and match those to get useful results (one 
example is coded 6 different ways). No one thinks this is a 
good "end state", just like the thread module was a lousy "end 
state".  But we can't steal Java's API here, and we can't steal 
from functional languages either, 'cause Python just ain't 
functional.

Everybody wants to see safer ways of using this stuff, but 
there's still a lot of experimenting to do. And I, for one, don't 
want to slam the door after one or two safe ways have been 
found. (And personally, I've found most of the attempts at, e.g. 
writing a "coroutine" class, less comprehensible than using 
the primitives directly.)



- Gordon



From tismer at tismer.com  Mon Nov  6 15:23:48 2000
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 16:23:48 +0200
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>
Message-ID: <3A06BEF4.95B773BD@tismer.com>


Greg Ewing wrote:
> 
> Jeremy Hylton <jeremy at alum.mit.edu>:
> 
> > Are continuations necessary to implement them or are
> > there other options?
> 
> I think you'll find that any implementation of microthreads
> or coroutines or whatever you want to call them, that
> doesn't rely on playing nonportable tricks with the C
> stack, will be just as mindbending as Stackless.
> 
> > The problem is that continuations break the notion a C API call will
> > always return an answer;
> 
> So do threads or coroutines. As soon as you have multiple
> threads of control, you have the chance that one of them
> will switch to another and never get back.

This is correct!
The only "special" thing with my continuation implementation
is that frames are armed to be able to accept a return
from a callee multiple times.
This little piece on top of frames turns them into full
continuations.

Without doing this, the API doesn't change, just the implementation
gets a little simpler.
What remains is still what Greg says:
The guarantee of stack-like frame execution no longer holds,
and every stackless C extension must either provide a way
to handle calls in a tail-recursive manner, *or* it must
enforce stack-like execution, like it is done today.

The *only* thing out of (coroutines, generators, uthreads)
which can be built without breaking this assumption are
the simple ICON-style generators.
But those can be implemented without becoming stackless
at all.

If you want full coroutines, or uthreads, the non-trivial
change of execution-order which Stackless permits *is*
necessary.

The step from there to supporting full continuations
is tiny, can be done easily or left completely.
The order of complexity of the Python implementation
is not increased by continuations.
In fact, supporting uthreads and coroutines and not only
stack-based generators impose the real problem.

That's one of my reasons to support continuations:
Making Python completely coroutine aware, without
tricking the C stack, is 90 percent of the problem.
But after walking that far, there is no reason
to leave the other 10 percent alone.

...
Greg:
> I can't see any way out of this. Continuations/coroutines/
> microthreads are all basically the same thing underneath, and
> they imply an execution model that just doesn't fit well
> with C.

Yes, you understood it completely.

> Maybe we need to reimplement Python in Scheme, and then
> feed it through a good Scheme compiler. SPython, anyone?

I believe the current implementation is quite pleasant
for a lot of people, while it isn't perfectly stackless.

A lot of internal operations could be made stackless by
introduction of a couple more opcodes, turning these
operations into bytecode. This would probably not cost
much speed, since only those code branches are a problem
which cause an interpreter recursion today, anyway.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From guido at python.org  Mon Nov  6 16:41:02 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 10:41:02 -0500
Subject: [Python-Dev] Three argument slices.
In-Reply-To: Your message of "Sat, 04 Nov 2000 16:21:48 GMT."
             <3a04376a.28016074@smtp.worldonline.dk> 
References: <3a04376a.28016074@smtp.worldonline.dk> 
Message-ID: <200011061541.KAA07351@cj20424-a.reston1.va.home.com>

> While updating the difference page in the Jython documentation, I came
> across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this respect?

Good question.  I haven't pronounced on Michael's patch

http://sourceforge.net/patch/?func=detailpatch&patch_id=100998&group_id=5470

because I don't know whether this is really a good idea.  I don't know
that I ever *will* pronounce uness someone points out how useful it is
for some practical application.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Mon Nov  6 16:50:35 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:50:35 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <14854.54091.14275.140381@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <py-dev at zadka.site.co.il> writes:

    MZ> (which was meant for Barry to assign me a PEP number
    MZ> primarily...)

Looks like Guido beat me to it this morning and assigned 228 to your
PEP.  It really pisses me off though, him stepping on my toes there,
because clearly the PEP should have been assigned PEP .3/1.e-3

:)

-Barry



From barry at wooz.org  Mon Nov  6 16:59:40 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:59:40 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
	<200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
	<E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
	<200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<20001105034044.M10135@lyra.org>
Message-ID: <14854.54636.49888.781754@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

    GS> Allocate yourself a PEP number and publish the darn thing. If
    GS> you don't feel comfortable grabbing a PEP number, then just
    GS> post it to the list or something.

A general rule is that meta-peps usually get assigned numbers < 100
and standards track peps just get the next free number above 200.
Pretty simple really.

However, I'm trying to encourage two things: only serious proposals
get pep'd, and peps ought to have a consistent style.  I want to avoid
approving peps for incomplete proposals (such as 207, not to purposely
pick on David).  There are enough good peps that you should largely be
able to figure out the style to use.  I can clean up any minor issues
after the fact.

I think the way Moshe did this one was fine.  He wrote it up and
posted it, and got assigned a pep number fairly quickly.

-Barry



From rrdn60 at email.sps.mot.com  Mon Nov  6 17:04:41 2000
From: rrdn60 at email.sps.mot.com (Norman Shelley (rrdn60))
Date: Mon, 06 Nov 2000 09:04:41 -0700
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <3A06D699.D3AFAC0@email.sps.mot.com>

Jeremy Hylton wrote:

> [Changed discussion list from general python-list to specific
> stackless.]
>
> >>>>> "GvR" == Guido van Rossum <guido at python.org> writes:
>
>   >> I have put up 6 pages of information about stackless at
>   >> http://www.mcmillan-inc.com/stackless.html
>
>   GvR> Gordon, thanks for doing this.  I still have a review of
>   GvR> Stackless on my TODO list.  It takes a serious chunk of my time
>   GvR> to do it justice, and this continues to be a problem, but the
>   GvR> existience of your overview certainly helps.  I still think
>   GvR> that the current Stackless implementation is too complex, and
>   GvR> that continuations aren't worth the insanity they seem to
>   GvR> require (or cause :-), but that microthreads and coroutines
>   GvR> *are* worth having and that something not completely unlike
>   GvR> Stackless will be one day the way to get there...
>
> I tend to agree with you, Guido.  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.
>
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?

This kind of question/thought disturbs me.  It presumes we can determine
apriori all the ways one might wish to use the features that Stackless
provides.  Closing off or bounding innovation just because we can't
answer the question as to how it will be used will just cause future
forks in Python or promote or  non-Python choices.

> How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?
>
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?
>
> Jeremy
>
> _______________________________________________
> Stackless mailing list
> Stackless at starship.python.net
> http://starship.python.net/mailman/listinfo/stackless




From barry at wooz.org  Mon Nov  6 17:05:51 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:05:51 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
	<14853.52668.117844.28459@bitdiddle.concentric.net>
	<14854.7221.546916.848838@cj42289-a.reston1.va.home.com>
Message-ID: <14854.55007.188953.545215@anthem.concentric.net>

BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
unreasonable that he'd have to wait until Monday to get a PEP number.
I reserve the right to apply Warsaw's 2nd Law to these cases. :)

-Barry

Warsaw's Second Law: Unbending Law of Commit Scheduling 
    Never change anything after 3pm on a Friday. 



From hinsen at cnrs-orleans.fr  Mon Nov  6 17:12:39 2000
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 6 Nov 2000 17:12:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011061235400.4175-100000@sundial> (message from
	Moshe Zadka on Mon, 6 Nov 2000 12:42:34 +0200 (IST))
References: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>
Message-ID: <200011061612.RAA05582@chinon.cnrs-orleans.fr>

> > + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"

I'd like to know what I am about to be sucked into here ;-)

> OK, I hope this e-mail address reaches him: I got it off his webpage.

That's fine, my web page always know how to reach me.


From barry at wooz.org  Mon Nov  6 17:23:43 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:23:43 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
	<20001105201851.C27208@xs4all.nl>
	<20001105135439.V10135@lyra.org>
	<20001105231235.X12776@xs4all.nl>
Message-ID: <14854.56079.737563.729305@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:

    TW> Well, syncmail was written to manage the Python CVS tree on a
    TW> slow Sun (I believe)

Correct.
    
    TW> and did an rsync-over-ssh to another machine as well. That can
    TW> definately take long ;) If we just remove the fork, the rest
    TW> of syncmail might just work, even with new files. In the mean
    TW> time, I'll check in my change. It might be the best thing to
    TW> do anyway, since it shouldn't interfere unless the file isn't
    TW> there.

Your patch looks good Thomas, thanks for tracking this down.  IIRC,
without the fork, the checkin would actually deadlock trying to get
the diff.

-Barry



From guido at python.org  Mon Nov  6 17:28:50 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 11:28:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 15:52:55 +0200."
             <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com>  
            <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
Algol-60 did this.  Fortran does this.  And rational numbers are less
innovative?

And what's dangerous about 1/2 yielding 0.5?

> My PEP does not yet deal with either written or inputted representation.

These affect the numeric model though (see Tim's posts) so should be
considered.

> [GvR]
> > (2) Rational numbers have the unpleasant property of growing
> >     unboundedly during many innocent calculations, thereby using up
> >     exorbitant amounts of memory and slowing down the calculation --
> >     often mysteriously, because what is displayed is truncated.
> 
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
is clearly an issue open for debate, but note that I speak from
experience: ABC used rationals unless you forced it to use reals, and
the rationals *did* cause real users to complain about how slow ABC
was.

> [GvR]
> > Another issue that I might bring up is that there are no inexact
> > numbers (each floating point number is perfectly exact and rational)
> 
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

Without a clear implementation plan your PEP is incomplete, PEP
guidelines notwithstanding.  Quality of implementation is important
for such a basic feature!

> [GvR]
> > If we take its meaning literally, the isreal() function should only
> > return true for numbers for which isrational() is also true:
> 
> Correct -- in my current model. If you later add things like constructive
> reals, that is no longer true: if I have a constructive Pi, it's not 
> rational. 
> 
> [GvR]
> > mathematically speaking, real numbers that aren't also rational don't
> > have an easy finite representation, since they are numbers like
> > sqrt(2) or pi.
> 
> But numbers don't have to have a finite (periodic really) representation
> to be representable in the computer: what about infinite continued fractions,
> for example?

So the question is, what implementation do you have in mind?  You
can't just go prescribe idealistic semantics and hope it gets
implemented by magic (even Tim can't do that :-).

> [GvR]
> > Finally, the PEP doesn't talk about how the numeric model can be
> > extended
> 
> That's because its rich enough not to need it. 
> Let me explain exactly what I mean: as long as all field operations between
> Python numbers give honest to god Python numbers, then everything else
> can be solved with Python's current model of coercions, and can be solved
> well when the coercion PEP will be written.

I think this deserves very explicit mention in your PEP.  An example
of how I would go about implementing my own Rational class or
extension type would help.

Also, the coercions PEP is still in need of an author.  Maybe you want
to take this on too?  It will help your numeric proposal if you can
write down how you think coercions should work.

> [GvR]
> > I've
> > heard of wild floating point representations that make multiplication
> > and division really cheap but addition a pain, rather than the other
> > way around;
> 
> Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
> that instead of inexact() and math.{...}. How these numbers interact
> with builtin Python numbers is your responsibility -- and that's what
> the coercions are for.

So explain how to do the coercions.  This will force you to be
explicit about implementation details.  (Both from Python and from C.)

> Same goes for gmp: as long as you're not expecting to be able to change
> 10000000000+10000000000 to be a gmp long rather then a Python long, then
> there shouldn't be a problem.

Fair enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 18:02:51 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:02:51 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com>
Message-ID: <3A06E43B.D95EC267@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> > I think the problem we currently have with subclassing types
> > is strongly related to the fact that all Py<type>_Check()
> > macros only work on a address compare basis.
> 
> I don't think this is the problem, although it is closely
> related.  The problem is that the interpreter uses these type
> checks to special case the handling of certain types.
> PyInstance_Check() is a big offender.
> 
> Behavior should be based solely on the type structure.  Extension
> types would then be able to behave exactly like any other builtin
> type.  Your coercion proposal and David's rich comparisions both
> remove some of this special casing based on type.

Even though this could remove some of the problems, it doesn't
help much with a common use of Py<Type>_Check(): that of
using fast access macros and native Py<Type>_*() APIs once an
object has been classified as being of a certain type.
This usually improves performance.

By changing the simple address compare to a type handle
system, we might be able to add some more flexibility to
the system while keeping b/w compatibility.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 18:17:22 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:17:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."
             <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>

> [Guido]:
> > David Scherer proposed to spell this pragma as a "magical import"
> > (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

[Peter Funk]
> Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
> at module level as a backward compatible method to introduce module level
> pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
> I still like David Scherers proposal very much.

Oops, you're right.  That URL mentions "global olddivision".  The
"import floatdivision" proposal came from a more recent private mail
from Bruce Sherwood.  I maintain that neither seems the right way to
spell "directive".  I think "import" is slightly better if the import
is supposed to enable a feature that is not supported by previous
versions, because the import will cause a clear failure on systems
that don't have the new feature (rather than silently giving wrong
results sometimes).

> BTW:  I think the "symbol" '//' is incredible ugly and starting with
> IBMs JCL years ago all languages I encountered, that used this symbol
> for something, did suck in some way or another.  I would appreaciate
> very much, if it could be avoided alltogether to add a symbol '//'
> to Python.  '//' will look like a comment delimiter to most people today.

Good point.  I've just used // as a placeholder for a new way to spell
integer division.

> Using a new keyword like 'div' in the tradition of languages like 
> Modula-2 looks far more attractive to me.

That's a possibility too.  It's a new keyword though, which has a much
higher threshold for acceptance than a new two-character operator
symbol.  We could spell it as a built-in function: div(a, b),
(analogous to divmod(a, b)) but that's not very user-friendly either.

Keep looking...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 18:20:03 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:20:03 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:14:12 +0100."
             <3A067664.22D09D03@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>  
            <3A067664.22D09D03@lemburg.com> 
Message-ID: <200011061720.MAA07862@cj20424-a.reston1.va.home.com>

[GvR]
> > Hm...  There's also the problem that there's no easy way to do Unicode
> > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > local encoding), which should mean that writing Unicode objects to the
> > file should "do the right thing" (in particular should not try to
> > coerce it to an 8-bit string using the default encoding first, like
> > print and str() currently do) and that writing 8-bit string objects to
> > it should first convert them to Unicode using the default encoding
> > (meaning that at least ASCII strings can be written to a Unicode file
> > without having to specify a conversion).  I support that reading from
> > a "Unicode file" should always return a Unicode string object (even if
> > the actual characters read all happen to fall in the ASCII range).
> > 
> > This requires some serious changes to the current I/O mechanisms; in
> > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > added that it used in certain cases.  Tricky, tricky!

[MAL]
> It's not all that tricky since you can write a StreamRecoder
> subclass which implements this. AFAIR, I posted such an implementation
> on i18n-sig.
> 
> BTW, one of my patches on SF adds unistr(). Could be that it's
> time to apply it :-)

Adding unistr() and StreamRecoder isn't enough.  The problem is that
when you set sys.stdout to a StreamRecoder, the print statement
doesn't do the right thing!  Try it.  print u"foo" will work, but
print u"\u1234" will fail because print always applies the default
encoding.

The required changes to print are what's tricky.  Whether we even need
unistr() depends on the solution we find there.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tismer at tismer.com  Mon Nov  6 17:19:21 2000
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 18:19:21 +0200
Subject: [Python-Dev] cgi.py and huge uploads problem
Message-ID: <3A06DA09.143D2D9@tismer.com>

Howdy,

there is a problem with the cgi.py implementation of Python 1.5.2
and uploading of huge files.
(found out by Joachim Rudolph, Virtual Photonics)

Class FieldStorage of cgi.py has a couple of methods which
add accumulated lines to a self.lines array. This array fills and
fills until the whole upload is done, with the side effect of
loading the whole file into memory. The memory is freed
after the whole upload is done.

This is no problem, until a company like VPI uses cgi.py to
upload whole distributions of 100 MB and more, via Zope. :-)

Looking into cgi.py, I can't find a reason why this happens.
Is this possibly just a debugging feature which is no longer
needed?
While cgi.py was modified for cosmetic resons, I didn't find
changes for Python 2.0 on this topic.

Does it make sense to use a debug flag for this, or should
the feature vanish completely?
Do you want a patch?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at zadka.site.co.il  Tue Nov  7 02:30:05 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:30:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Mon, 06 Nov 2000 11:28:50 EST." <200011061628.LAA07574@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>  <200011061628.LAA07574@cj20424-a.reston1.va.home.com> 
Message-ID: <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

OK, I take back the innovation claim.

[Guido]
> And what's dangerous about 1/2 yielding 0.5?

Nothing, but then people would expect 1/3 to yield 0.333333333....,
while simultaneusly expecting (1/3)*3 == 1.
IOW, shoving floating points in people's faces is not user-friendly.

[Moshe]
> My PEP does not yet deal with either written or inputted representation.

[Guido]
> These affect the numeric model though (see Tim's posts) so should be
> considered.

I agree. That's why they are in the "unresolved issues" of the PEP.

[Guido]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

[Moshe]
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

[Guido]
> That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> is clearly an issue open for debate, but note that I speak from
> experience: ABC used rationals unless you forced it to use reals, and
> the rationals *did* cause real users to complain about how slow ABC
> was.

Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
list, and nobody ever complained. Besides, if "forcing" is simply saying
either inexact() (and note that inexact-exact operations are performed
inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
be), it might be easier. I don't know ABC well enough to say how the
forcing mechanism worked.

[Guido]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

[Moshe]
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

[Guido]
> Without a clear implementation plan your PEP is incomplete, PEP
> guidelines notwithstanding.  Quality of implementation is important
> for such a basic feature!

Well, I agree. When I set down to complete the PEP, I'll go over all
the math/cmath functions and remark how the implementation should change.
I also plan to draft an implementation design. I just wanted to throw
the idea out into the open, to get some feedback -- in no way can the
PEP be considered complete.

[Guido]
> So the question is, what implementation do you have in mind?  You
> can't just go prescribe idealistic semantics and hope it gets
> implemented by magic (even Tim can't do that :-).

Well, for constructive reals, none. It's too much of pain to implement,
and too little to gain. That wouldn't preclude a later day implementation,
in case it turns out not to be the case. Besides, the tower wouldn't
look clean without it.

[Guido]
> I think this deserves very explicit mention in your PEP.  An example
> of how I would go about implementing my own Rational class or
> extension type would help.

Well, I don't see why there should be a difference from what happens
currently. The thing is, the model will not support you telling it
at runtime what the results for operations on types it already knows
should be: that would make us Scheme, not Python. 

[Guido]
> Also, the coercions PEP is still in need of an author.  Maybe you want
> to take this on too?  It will help your numeric proposal if you can
> write down how you think coercions should work.

Smooth, real smooth. OK, sold to the highest bidder -- I take it.
I'll update it and 0000, and start reading MAL's pages.

[Guido]
> So explain how to do the coercions.  This will force you to be
> explicit about implementation details.  (Both from Python and from C.)

Again this is no different from current day Python, modulu other PEPs. <wink> 
-- 
Moshe Zadka <sig at zadka.site.co.il>



From gmcm at hypernet.com  Mon Nov  6 18:27:22 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 12:27:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."             <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <3A06A3AA.1401.6080F51@localhost>

[Peter Funk]
> > Using a new keyword like 'div' in the tradition of languages like 
> > Modula-2 looks far more attractive to me.
[Guido]:
> That's a possibility too.  It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.  We could spell it as a built-in function: div(a, b),
> (analogous to divmod(a, b)) but that's not very user-friendly either.
> 
> Keep looking...

FWIW, I remember as a newbie being sure that integer 
division was spelled "a div b". In fact when it didn't work, I went 
digging through PP, IPWP and the docs in dumbfounded 
disbelief.

this-is-your-brain-on-Modula-2-ly y'rs

- Gordon



From nas at arctrix.com  Mon Nov  6 11:35:29 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 6 Nov 2000 02:35:29 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06E43B.D95EC267@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 06:02:51PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com>
Message-ID: <20001106023529.B8639@glacier.fnational.com>

On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> Neil Schemenauer wrote:
> > Behavior should be based solely on the type structure.  Extension
> > types would then be able to behave exactly like any other builtin
> > type.  Your coercion proposal and David's rich comparisions both
> > remove some of this special casing based on type.
> 
> Even though this could remove some of the problems, it doesn't
> help much with a common use of Py<Type>_Check(): that of
> using fast access macros and native Py<Type>_*() APIs once an
> object has been classified as being of a certain type.
> This usually improves performance.

Can you clarify what you mean by "it doesn't help much"?  Do you
mean that extension types will not be able to perform as well as
types that get special treatment by the interpreter?  I think the
major problem that extension types _cannot_ behave the same as
the builtin types.

> By changing the simple address compare to a type handle
> system, we might be able to add some more flexibility to
> the system while keeping b/w compatibility.

I don't see what this buys us.  The Python interpreter shouldn't
care about which type object it is dealing with.  Can you give an
example of where you think this would be useful?

  Neil



From moshez at zadka.site.co.il  Tue Nov  7 02:40:30 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:40:30 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model 
In-Reply-To: Message from barry@wooz.org (Barry A. Warsaw) 
   of "Mon, 06 Nov 2000 11:05:51 EST." <14854.55007.188953.545215@anthem.concentric.net> 
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de> <Pine.GSO.4.10.10011051411170.28025-100000@sundial> <14853.36547.748584.450976@cj42289-a.reston1.va.home.com> <14853.52668.117844.28459@bitdiddle.concentric.net> <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>  <14854.55007.188953.545215@anthem.concentric.net> 
Message-ID: <E13sxkU-0005PL-00@darjeeling.zadka.site.co.il>

[Barry]
> BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
> unreasonable that he'd have to wait until Monday to get a PEP number.

It is to me. If I work on Sunday, I might as well do some Python work too...
Problems of truly international open source projects <wink>
-- 
Moshe Zadka <sig at zadka.site.co.il>



From guido at python.org  Mon Nov  6 18:43:07 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:43:07 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:19:21 +0200."
             <3A06DA09.143D2D9@tismer.com> 
References: <3A06DA09.143D2D9@tismer.com> 
Message-ID: <200011061743.MAA08031@cj20424-a.reston1.va.home.com>

> there is a problem with the cgi.py implementation of Python 1.5.2
> and uploading of huge files.
> (found out by Joachim Rudolph, Virtual Photonics)
> 
> Class FieldStorage of cgi.py has a couple of methods which
> add accumulated lines to a self.lines array. This array fills and
> fills until the whole upload is done, with the side effect of
> loading the whole file into memory. The memory is freed
> after the whole upload is done.
> 
> This is no problem, until a company like VPI uses cgi.py to
> upload whole distributions of 100 MB and more, via Zope. :-)
> 
> Looking into cgi.py, I can't find a reason why this happens.
> Is this possibly just a debugging feature which is no longer
> needed?
> While cgi.py was modified for cosmetic resons, I didn't find
> changes for Python 2.0 on this topic.
> 
> Does it make sense to use a debug flag for this, or should
> the feature vanish completely?
> Do you want a patch?

You know, I have *no* idea why this is.  I have looked through various
revisions (this feature is as old as cgi.py:1.8) and cannot find any
use of or need for self.lines!  It just gets appended to.

So I think it's safe to toss all the references to self.lines and see
who complains.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 18:36:54 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:36:54 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>  
	            <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>
Message-ID: <3A06EC36.61B32B4D@lemburg.com>

Guido van Rossum wrote:
> 
> [GvR]
> > > Hm...  There's also the problem that there's no easy way to do Unicode
> > > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > > local encoding), which should mean that writing Unicode objects to the
> > > file should "do the right thing" (in particular should not try to
> > > coerce it to an 8-bit string using the default encoding first, like
> > > print and str() currently do) and that writing 8-bit string objects to
> > > it should first convert them to Unicode using the default encoding
> > > (meaning that at least ASCII strings can be written to a Unicode file
> > > without having to specify a conversion).  I support that reading from
> > > a "Unicode file" should always return a Unicode string object (even if
> > > the actual characters read all happen to fall in the ASCII range).
> > >
> > > This requires some serious changes to the current I/O mechanisms; in
> > > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > > added that it used in certain cases.  Tricky, tricky!
> 
> [MAL]
> > It's not all that tricky since you can write a StreamRecoder
> > subclass which implements this. AFAIR, I posted such an implementation
> > on i18n-sig.
> >
> > BTW, one of my patches on SF adds unistr(). Could be that it's
> > time to apply it :-)
> 
> Adding unistr() and StreamRecoder isn't enough.  The problem is that
> when you set sys.stdout to a StreamRecoder, the print statement
> doesn't do the right thing!  Try it.  print u"foo" will work, but
> print u"\u1234" will fail because print always applies the default
> encoding.

Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
Perhaps we ought to let it call PyObject_Unicode() (which you
find in the patch on SF) instead for Unicode objects. That way
the file-like .write() method will be given a Unicode object
and StreamRecoder could then do the trick.

Haven't tried this, but it could work (the paths objects take
through Python to get printed are somewhat strange at times
-- there are just so many different possiblities and special
cases that it becomes hard telling from just looking at the
code).

> The required changes to print are what's tricky.  Whether we even need
> unistr() depends on the solution we find there.

I think we'll need PyObject_Unicode() and unistr() one way
or another. Those two APIs simply complement PyObject_Str()
and str() in that they always return Unicode objects and
do the necessary conversion based on the input object type.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 18:49:35 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:49:35 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Tue, 07 Nov 2000 03:30:05 +0200."
             <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com>  
            <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>

> [Guido]
> > That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> > is clearly an issue open for debate, but note that I speak from
> > experience: ABC used rationals unless you forced it to use reals, and
> > the rationals *did* cause real users to complain about how slow ABC
> > was.

[Moshe]
> Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
> list, and nobody ever complained. Besides, if "forcing" is simply saying
> either inexact() (and note that inexact-exact operations are performed
> inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
> be), it might be easier. I don't know ABC well enough to say how the
> forcing mechanism worked.

Scheme has no real users -- only CS types. :-)

ABC's forcing was as simple as writing ~x or mixing inexact numbers.
We did have the notion that 1.0 was an exact literal (to me it looks
exact!) so maybe that was an additional problem.

> [Guido]
> > I think this deserves very explicit mention in your PEP.  An example
> > of how I would go about implementing my own Rational class or
> > extension type would help.
> 
> Well, I don't see why there should be a difference from what happens
> currently. The thing is, the model will not support you telling it
> at runtime what the results for operations on types it already knows
> should be: that would make us Scheme, not Python. 

Agreed.  I think what makes me feel uneasy is that your proposal
assumes that there is One True Numeric Type, and all the rest are
second-class numbers.  Python currently makes no such claim: there are
many numeric types built in and you can add your own types that play
along.  The only thing that makes a difference is that there are
literals for the built-in types and not for the extension types; but
apart from that they have all the same privileges, and the coercion
rules work in everybody's favor (except when they don't :-). Placing
more emphasis on One True Numeric Type runs a risk of discriminating
against the others.

> [Guido]
> > Also, the coercions PEP is still in need of an author.  Maybe you want
> > to take this on too?  It will help your numeric proposal if you can
> > write down how you think coercions should work.
> 
> Smooth, real smooth. OK, sold to the highest bidder -- I take it.
> I'll update it and 0000, and start reading MAL's pages.

OK, it's a deal.  But make yourself a co-author with MAL.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 18:53:36 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:53:36 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:36:54 +0100."
             <3A06EC36.61B32B4D@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>  
            <3A06EC36.61B32B4D@lemburg.com> 
Message-ID: <200011061753.MAA08112@cj20424-a.reston1.va.home.com>

[Guido]
> > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > when you set sys.stdout to a StreamRecoder, the print statement
> > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > print u"\u1234" will fail because print always applies the default
> > encoding.

[MAL]
> Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> Perhaps we ought to let it call PyObject_Unicode() (which you
> find in the patch on SF) instead for Unicode objects. That way
> the file-like .write() method will be given a Unicode object
> and StreamRecoder could then do the trick.

That's still not enough. Classes and types should be able to have a
__str__ (or tp_str) that yields Unicode too.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at zadka.site.co.il  Tue Nov  7 03:08:05 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 04:08:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Mon, 06 Nov 2000 12:49:35 EST." <200011061749.MAA08063@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com> <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>  <200011061749.MAA08063@cj20424-a.reston1.va.home.com> 
Message-ID: <E13syBB-0005Zr-00@darjeeling.zadka.site.co.il>

[Guido]
> Scheme has no real users -- only CS types. :-)

I resent that! I'm not a CS type: I never studied CS, and I refuse
to stand for that kind of mud slinging.

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

Let me clarify: exactness *is* in the operations. Since Python is
an OO language, an object decides semantics of operations on it. "1.0"
decides to have inexact operations on it, no matter how exact it
looks to you.

[Guido]
> Agreed.  I think what makes me feel uneasy is that your proposal
> assumes that there is One True Numeric Type, and all the rest are
> second-class numbers.  Python currently makes no such claim: there are
> many numeric types built in and you can add your own types that play
> along.

My proposal doesn't change that, as far as it's true. (And it isn't
really true: I cant have my new types as literals, or as results of
existing operations). It changes the number of built-in numeric types
to 1.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Mon Nov  6 19:15:27 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 19:15:27 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>  
	            <3A06EC36.61B32B4D@lemburg.com> <200011061753.MAA08112@cj20424-a.reston1.va.home.com>
Message-ID: <3A06F53F.2CF6760@lemburg.com>

Guido van Rossum wrote:
> 
> [Guido]
> > > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > > when you set sys.stdout to a StreamRecoder, the print statement
> > > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > > print u"\u1234" will fail because print always applies the default
> > > encoding.
> 
> [MAL]
> > Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> > Perhaps we ought to let it call PyObject_Unicode() (which you
> > find in the patch on SF) instead for Unicode objects. That way
> > the file-like .write() method will be given a Unicode object
> > and StreamRecoder could then do the trick.
> 
> That's still not enough. Classes and types should be able to have a
> __str__ (or tp_str) that yields Unicode too.

Instances are allowed to return Unicode through their __str__
method and PyObject_Unicode() will pass it along. PyObject_Str()
will still convert it to an 8-bit string though because there's
too much code out there which expects a string object (without
checking !) ... even the Python core.

So if you print an instance which returns Unicode through __str__,
the wrapper should see a real Unicode object at its end... at least
I think we're getting closer ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Mon Nov  6 19:32:44 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 13:32:44 -0500 (EST)
Subject: [Python-Dev] cgi.py and huge uploads problem
References: <3A06DA09.143D2D9@tismer.com>
	<200011061743.MAA08031@cj20424-a.reston1.va.home.com>
Message-ID: <14854.63820.201069.434450@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> You know, I have *no* idea why this is.  I have looked
    GvR> through various revisions (this feature is as old as
    GvR> cgi.py:1.8) and cannot find any use of or need for
    GvR> self.lines!  It just gets appended to.

    GvR> So I think it's safe to toss all the references to self.lines
    GvR> and see who complains.

There are two bug reports related to this in the SF database.  The
first one was 110674, which we closed after adding the feature request
to PEP 42.  The second is 119806, which looks fairly new, but wasn't
submitted by Chris or Joachim.

I came to the same conclusion Guido does above when I looked at 110674
in the Python 2.0 time frame, but didn't feel comfortable making that
change for 2.0.  I think it's correct to make the change now.

I will do the following:

- remove self.lines from cgi.py
- close bug #119806
- update pep 42

Attached below is the patch.

-Barry

-------------------- snip snip --------------------
Index: cgi.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/cgi.py,v
retrieving revision 1.55
diff -u -r1.55 cgi.py
--- cgi.py	2000/10/03 13:51:09	1.55
+++ cgi.py	2000/11/06 18:32:18
@@ -497,7 +497,6 @@
 
         self.list = self.file = None
         self.done = 0
-        self.lines = []
         if ctype == 'application/x-www-form-urlencoded':
             self.read_urlencoded()
         elif ctype[:10] == 'multipart/':
@@ -633,7 +632,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             self.file.write(line)
 
     def read_lines_to_outerboundary(self):
@@ -646,7 +644,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:
@@ -676,7 +673,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:



From guido at python.org  Mon Nov  6 20:09:26 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 14:09:26 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 13:32:44 EST."
             <14854.63820.201069.434450@anthem.concentric.net> 
References: <3A06DA09.143D2D9@tismer.com> <200011061743.MAA08031@cj20424-a.reston1.va.home.com>  
            <14854.63820.201069.434450@anthem.concentric.net> 
Message-ID: <200011061909.OAA08466@cj20424-a.reston1.va.home.com>

>     GvR> You know, I have *no* idea why this is.  I have looked
>     GvR> through various revisions (this feature is as old as
>     GvR> cgi.py:1.8) and cannot find any use of or need for
>     GvR> self.lines!  It just gets appended to.
> 
>     GvR> So I think it's safe to toss all the references to self.lines
>     GvR> and see who complains.

[Barry]
> There are two bug reports related to this in the SF database.  The
> first one was 110674, which we closed after adding the feature request
> to PEP 42.  The second is 119806, which looks fairly new, but wasn't
> submitted by Chris or Joachim.
> 
> I came to the same conclusion Guido does above when I looked at 110674
> in the Python 2.0 time frame, but didn't feel comfortable making that
> change for 2.0.  I think it's correct to make the change now.
> 
> I will do the following:
> 
> - remove self.lines from cgi.py
> - close bug #119806
> - update pep 42
> 
> Attached below is the patch.

Patch looks good.  Go for it.

Somehow when I skimmed the original bug reports I never really
understood what was going on -- thanks to Christian for pointing out
*exactly* what was the problem in words I could understand. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 21:01:40 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 21:01:40 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>
Message-ID: <3A070E24.B07FA7D7@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> > Neil Schemenauer wrote:
> > > Behavior should be based solely on the type structure.  Extension
> > > types would then be able to behave exactly like any other builtin
> > > type.  Your coercion proposal and David's rich comparisions both
> > > remove some of this special casing based on type.
> >
> > Even though this could remove some of the problems, it doesn't
> > help much with a common use of Py<Type>_Check(): that of
> > using fast access macros and native Py<Type>_*() APIs once an
> > object has been classified as being of a certain type.
> > This usually improves performance.
> 
> Can you clarify what you mean by "it doesn't help much"?  Do you
> mean that extension types will not be able to perform as well as
> types that get special treatment by the interpreter?  I think the
> major problem that extension types _cannot_ behave the same as
> the builtin types.

If you only define types by interface, the interpreter will have
to apply the interface availability checks every time it calls
a slot. This would cause a major performance hit which would not
be acceptable.

The "doesn't help much" refers to the fact that once an object
has been identified as say Integer you are free to use whatever
access macro or function you choose.

> > By changing the simple address compare to a type handle
> > system, we might be able to add some more flexibility to
> > the system while keeping b/w compatibility.
> 
> I don't see what this buys us.  The Python interpreter shouldn't
> care about which type object it is dealing with.  Can you give an
> example of where you think this would be useful?

Take e.g. dictionaries: you could easily add a new dictionary
type which uses case-insensitive string keys by extending the
existing dictionary type.

The new type would reuse most of the slots of the original
type and only replace the ones needed for lookup with the
new logic for case-insensitivity.

Then it sets the type id to PyDict_TypeID and Python will
use it as if it were an original dictionary object. The
underlying type objects would be different though (and also
the type object address which is currently used to identify
a builtin type).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 22:02:27 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 16:02:27 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Sun, 05 Nov 2000 22:24:27 PST."
             <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> 
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> 
Message-ID: <200011062102.QAA09294@cj20424-a.reston1.va.home.com>

Paul, thanks for submitting a warnings framework.  I'd like to give
some feedback, comparing it to my own proposal.  Please also give
explicit feedback on mine!

[Paul Prescod]
> Abstract
> 
>     This PEP describes a generalized warning mechanism for Python 2.1. The
>     primary purpose of this mechanism is to alert the programmer or user
>     of a program to potential or likely errors which, for whatever reason,
>     are not considered exception-worthy. For example, this might be done
>     to keep old code working during a transitional period or to alert the
>     programmer or user of a recoverable error.

What's missing here is a set of requirements.  Comparing your proposal
to my requirements, I find that you are not addressing two
requirements that are important in my mind: a convenient and efficient
C API, and a mechanism that prints a warning message only the first
time that the warning is issued.  These are important to me, because I
expect that most warnings will probably be generated by C code, and in
order to be useful we must avoid mindless repetition.  If a function
divides two integers using the / operator, this is being detected by C
code (the int or long division implementation) and we only want to
print the warning once per program execution and per source location.

My expectation is that if a single occurrence (in the program) of a
warning condition caused an endless sequence of warnings to be spit
out, people would quickly grow a habit of disabling all warnings, thus
defeating the purposes.

Warnings are more a human engineering issue than a technical issue!
That's also why I am emphasizing a C API -- I want to make it real
easy to ussue quality warnings in the runtime.  It's also why I
specify a rich (probably too rich!) filtering mechanism.

> Syntax
> 
>     assert >> cls, test[[[, arg], arg]...]

I have several problems with this.  First of all, using "assert" means
that in "optimizing" mode (python -O) you won't get *any* warnings.  I
think that the decision to disable all warnings should be independent
from the decision to "optimize".  Second, you're hypergeneralizing the
extended print syntax.  Just because I think it's okay to add >>file
to the print syntax doesn't mean that it's now okay to add >>object
syntax to all statements!

I also don't see what warnings have to do with assertions.  Assertions
are a mechanism to check for error conditions.  What happens if the
error is detected is of less importance -- it could raise an exception
(Python), issue a fatal error (C), or do nothing (in -O mode).

With warnings I believe the issue is not so much the detection of the
condition (for which a regular 'if' statement does just fine) but the
reporting.  Again, this is motivated by the fact that I expect that
flexible filtering is essential for a successful warning mechanism.

>     "cls" may be any callable object that takes a list as a single
>     argument argument list and returns an object with the required
>     attributes "get_action" and "format_message"
>     
>        * get_action() -> "warn"|"error"|"suppress"
>        * format_message() -> string
> 
>     A provided base class implements these methods in a reusable
>     fashion. Warning creators are encouraged to merely subclass.

This is just a matter of exposition, but when I first read your PEP I
had a hard time figuring out the purpose of the cls object.  It wasn't
until I got to the very end where I saw your example classes that I
realized what it is: it represents a specific warning condition or a
group of related warning conditions.

>     This extended form of the assertion statement calls the assertion
>     handler code in the new "assertions" module. 

I won't harp on this each time, but I'd like to point out once more
that "assertion" is the wrong name for a warning feature.  Although it
isn't part of the Zen of Python (by Tim Peters), it should be: a
suggestive name for a feature is worth half a spec!

>     The semantics of the built-in assertion handler are defined by the
>     following code. It should be exposed in a new "assertions" module.
> 
>     def handle_assertion(cls, message = ""):
>         "This code is called when an assertion fails and cls is not None"
> 
>         obj = cls(message)
>         action = obj.get_action()
>             
>         if action=="error":
>             *** existing assertion code ***

That's just

              raise AssertionError, message

Right?

>         elif action=="warn":
>             sys.stderr.write(obj.format_message())
>         elif action=="suppress":
>             pass
>         else:
>             assert action in ["warn","error","suppress"]
> 
>     Even if handle_assertion is implemented in C, it should be exposed as
>     assertions.handle_assertion so that it may be overriden. 

Suggestion: have separate warning and error handlers, so that if I
want to override these branches of the if statement I don't have to
repeat the entire handler.

>     The generic warning base class is defined below:
> 
>     class Assertion:
>       def __init__(self, *args):
>           if len(args) == 1:
>               self.args = args[0]
>           else:
>               self.args = args
> 
>       def format_message(self):
>           sys.stderr.write("%s: %s" %(obj.__name__, self.args))
> 
>       def get_action(self):
>           return (self.get_user_request(self.__class__) 
>                   or sys.default_warning_action)
> 
>       def get_user_request(self, cls):
>           if cls.__name__ in sys.errors:
>               return "error"
>           elif cls.__name__ in sys.warnings:
>               return "warn"
>           elif cls.__name__ in sys.disabled_warnings:

I see no definition of sys.disabled_warnings.  Instead of
sys.disabled_warnings you meant sys.suppress, right?

>               return "suppress"
>           for base in cls.__bases__:
>               rc = self.get_user_request(base)
>               if rc:
>                   return rc
>           else:
>               return None

This approach (searching for the class name or the name of one of its
base classes in a list) doesn't look very object-oriented.  It would
make more sense to store the desired return value as a class or
instance attribute.  The default warning action could be stored on the
base class.

>     The Assertion class should be implemented in Python so that it can be 
>     used as a base class.
> 
>     Because this code inherently implements "warning state inheritance",
>     it would be rare to override any of the methods, but this is possible
>     in exceptional circumstances.
> 
> Command line
> 
>     By default the special variables have the following contents:
> 
>     sys.warnings = []
>     sys.errors = []
>     sys.suppress = []
>     sys.default_warning_action = "warn"
>     
>     These variables may be changed from the command line. The command line
>     arguments are interpreted as described below:
> 
>     -w XXX => sys.warnings.append("XXX")
>     -e XXX => sys.errors.append("XXX")
>     -no-w XXX => sys.suppress.append("XXX")
>     -wall => sys.default_warning_action => "warn"
>     -eall => sys.default_warning_action => "error"
>     -no-wall => sys.default_warning_action => "suppress"

Python doesn't support long options (I don't *like* long options so I
doubt that this is a good occasion to start lobbying for them :-).  We
can come up with different options though.

>     As per the code above, errors take precedence over warnings and
>     warnings over suppressions unless a particular assertion class
>     specifies otherwise.

I would use a different precedence scheme: a more specific filter
takes precedence over a more general filter.  So -eall -wdubious would
mean that "dubious" class warnings are warnings but all others are
errors, and -wall -edubious would mean the opposite.

> Built-in warning objects:
> 
>     class exception_in_del(Assertion):
>         "An exception was ignored in an __del__ method"
> 
>     class deprecation(Assertion):
>         "This feature may be removed in a future version of Python."
> 
>     class dubious(Assertion):
>         "There is a common error associated with this feature."
> 
>     These class definitions are part of the "Assertion" module. They
>     should only ever be used when there exists a way for the programmer to 
>     accomplish the same thing without triggering the warning. For instance
>     the way to suppress the deletion exception is to trap exceptions in
>     __del__ methods with a try/except block.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Mon Nov  6 22:36:49 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:49 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com>

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
new rules, int div always truncates).

So far as innovation goes, no choice on the table so far is innovative
(neither mathematically nor in programming languages), so there's no basis
for choosing there.

Guido, *why* did ABC use rationals by default?  Was that driven by usability
studies?





From tim_one at email.msn.com  Mon Nov  6 22:36:50 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPHHOAA.tim_one@email.msn.com>

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

I seriously wonder whether that was *the* problem with ABC:  not only was
1.0 treated as an exact rational in ABC, so was 6.02e23 and 3.14159e-314
etc.  At least for me, this caused rationals to get used in many places I
*intended* to use floats.  I assume many others got burned by this too, as
I'd say it's impossible for users coming from other languages not to see
6.02e23 etc as float literals.





From paulp at ActiveState.com  Mon Nov  6 23:01:27 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Mon, 06 Nov 2000 14:01:27 -0800
Subject: [Python-Dev] Warnings PEP
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>
Message-ID: <3A072A37.C2CAFED1@activestate.com>

Guido van Rossum wrote:
> 
> Paul, thanks for submitting a warnings framework.  I'd like to give
> some feedback, comparing it to my own proposal.  Please also give
> explicit feedback on mine!

Okay, but I'm a little bit worried about getting into a science project
-- in terms of my time to work on it. I'm perfectly happy with the two
requirements you mentioned but I haven't thought enough about filters to
be able to whip off ideas about them quickly.

> What's missing here is a set of requirements.  Comparing your proposal
> to my requirements, I find that you are not addressing two
> requirements that are important in my mind: a convenient and efficient
> C API, and a mechanism that prints a warning message only the first
> time that the warning is issued.  

I agree that these are both important things and I will add them to my
proposal. Also, I like your requirement that the target for warnings
should be easily customizable.

> Warnings are more a human engineering issue than a technical issue!
> That's also why I am emphasizing a C API -- I want to make it real
> easy to ussue quality warnings in the runtime.  It's also why I
> specify a rich (probably too rich!) filtering mechanism.

Let me put out the strawman proposal that "grep" is a nicely orthogonal
filtering mechanism. <duck!>

I guess my thinking is that you turn off warnings at the source. If you
get a warning about __del__ exceptions then you put in a try/except. If
you get a warning about an unused variable then you assign it to itself.
If you get a warning about integer division then you should pass in a
float and so forth.

> > Syntax
> >
> >     assert >> cls, test[[[, arg], arg]...]
> 
> I have several problems with this.  First of all, using "assert" means
> that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> think that the decision to disable all warnings should be independent
> from the decision to "optimize".  

Arguable. I see it as "release" and "debug" configurations. python and
python_d.

> Second, you're hypergeneralizing the
> extended print syntax.  Just because I think it's okay to add >>file
> to the print syntax doesn't mean that it's now okay to add >>object
> syntax to all statements!

Well getting new syntax into Python is really, really hard so we've got
to squeeze out as much value out of what we have as possible. But
anyhow, assertions are not allowed to 

You and I agree that there is an important sociological dimension to
this. We can't require:

  import warnings

  warnings.warn("foo", "bar")

I prefer:

  warn Foo, "Bar"

just like:

  raise Foo, "Bar"

> I also don't see what warnings have to do with assertions.  Assertions
> are a mechanism to check for error conditions.  What happens if the
> error is detected is of less importance -- it could raise an exception
> (Python), issue a fatal error (C), or do nothing (in -O mode).

Warnings are issued when an error or dubious construct is detected.
Assertions are "fatal warnings". You agree that it is appropriate for
some "warnings" to kill the app in some circumstances. Isn't it just a
hop-skip-and-a-jump to say that warnings and errors are just points on
the spectrum:

Report Once
Report Always
Report and Die

Python trained me to think of function calls and object constructions as
being basically the same thing -- and keyword arguments and variable
names being basically the same thing. etc.

> With warnings I believe the issue is not so much the detection of the
> condition (for which a regular 'if' statement does just fine) but the
> reporting.  Again, this is motivated by the fact that I expect that
> flexible filtering is essential for a successful warning mechanism.

I don't see why assertions need special syntax but warnings do not! I
would have been happy for a "warn" keyword with assert-like syntax but I
don't think I'll see that in my lifetime.

> This is just a matter of exposition, but when I first read your PEP I
> had a hard time figuring out the purpose of the cls object.  It wasn't
> until I got to the very end where I saw your example classes that I
> realized what it is: it represents a specific warning condition or a
> group of related warning conditions.

It sort of evolved into an exception-like mechanism in that the class is
instantiated with arguments just as exception classes are.

> >         if action=="error":
> >             *** existing assertion code ***
> 
> That's just
> 
>               raise AssertionError, message
> 
> Right?

Well I'd like the traceback to emanate from the caller's position not
the warning handler's position. Python doesn't really have a way to say
that simply. This may well be implemented in C so it might not matter.

> Suggestion: have separate warning and error handlers, so that if I
> want to override these branches of the if statement I don't have to
> repeat the entire handler.

Good idea.

> I see no definition of sys.disabled_warnings.  Instead of
> sys.disabled_warnings you meant sys.suppress, right?

Right.

> >               return "suppress"
> >           for base in cls.__bases__:
> >               rc = self.get_user_request(base)
> >               if rc:
> >                   return rc
> >           else:
> >               return None
> 
> This approach (searching for the class name or the name of one of its
> base classes in a list) doesn't look very object-oriented.  It would
> make more sense to store the desired return value as a class or
> instance attribute.  The default warning action could be stored on the
> base class.

The way I came to this odd structure is I wanted most subclasses to be
just no-ops as many exceptions are:

class ActiveStateLintWarning(Warning): pass

class CodeLooksLikePerl(ActiveStateLintWarning): pass

class TooMuchOnOneLine(CodeLooksLikePerl): pass

So what __init__ could I write that would look up -wTooMuchOnOneLine and
then if it failed that, look up -wCodeLooksLikePerl and so forth?

It gets pretty mind-bending because you sort of want one method to call
another and yet you want it to be the *same* inherited method (i.e.
don't have to code it each time). E.g. you want to run the *same method*
at each level of the hierarchy. So I just do that directly.

> >     -w XXX => sys.warnings.append("XXX")
> >     -e XXX => sys.errors.append("XXX")
> >     -no-w XXX => sys.suppress.append("XXX")
> >     -wall => sys.default_warning_action => "warn"
> >     -eall => sys.default_warning_action => "error"
> >     -no-wall => sys.default_warning_action => "suppress"
> 
> Python doesn't support long options (I don't *like* long options so I
> doubt that this is a good occasion to start lobbying for them :-).  We
> can come up with different options though.

Are these long options? Or just overloaded behavior on -w, -e, -n .
Think of "all" as the base class for warnings or something.

> >     As per the code above, errors take precedence over warnings and
> >     warnings over suppressions unless a particular assertion class
> >     specifies otherwise.
> 
> I would use a different precedence scheme: a more specific filter
> takes precedence over a more general filter.  So -eall -wdubious would
> mean that "dubious" class warnings are warnings but all others are
> errors, and -wall -edubious would mean the opposite.

Good idea. I think the code may work that way already. :)

 Paul Prescod



From guido at python.org  Mon Nov  6 23:16:29 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 17:16:29 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:36:49 EST."
             <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com> 
Message-ID: <200011062216.RAA09669@cj20424-a.reston1.va.home.com>

> [Guido]
> > 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> > Algol-60 did this.  Fortran does this.  And rational numbers are less
> > innovative?

[Tim]
> Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
> new rules, int div always truncates).

I stand corrected -- the idea is only 40 years old, not 44. :-)

> So far as innovation goes, no choice on the table so far is innovative
> (neither mathematically nor in programming languages), so there's no basis
> for choosing there.
> 
> Guido, *why* did ABC use rationals by default?  Was that driven by usability
> studies?

I assume that the use of rationals for exact numbers was driven by
usability studies -- like us, the ABC designers were tired of
explaining the vagaries of floating point to novices.

I remember that I pushed for using rationals for 1E1000 and 1E-1000,
probably out of a mistaken sense of consistency.  I don't think I'm
responsible for 1.0 being exact -- in "The B Programmer's Handbook"
(CWI, 1985) 1.0 is exact and 1E10 is approximate.  In "The ABC
Progammer's Handbook (Prentice Hall, 1990) these are all exact.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From martin at loewis.home.cs.tu-berlin.de  Tue Nov  7 01:49:34 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 01:49:34 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070049.BAA01485@loewis.home.cs.tu-berlin.de>

> Is there some API for it in C?

In C99, you have access to the floating-point environment:

               #include <fenv.h>
               /* ... */
               {
                       #pragma STDC FENV_ACCESS ON
                       int set_excepts;
                       feclearexcept(FE_INEXACT | FE_OVERFLOW);
                       // maybe raise exceptions
                       set_excepts = fetestexcept(FE_INEXACT | FE_OVERFLOW);
                       if (set_excepts & FE_INEXACT) f();
                       if (set_excepts & FE_OVERFLOW) g();
                       /* ... */
               }

It defines the following symbolic exception constants:

               FE_DIVBYZERO
               FE_INEXACT
               FE_INVALID
               FE_OVERFLOW
               FE_UNDERFLOW

Few compilers support that, though.

Regards,
Martin



From greg at cosc.canterbury.ac.nz  Tue Nov  7 02:06:04 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 14:06:04 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
Message-ID: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>

> It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.

It could be non-reserved, since a div b is currently
a syntax error.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From martin at loewis.home.cs.tu-berlin.de  Tue Nov  7 02:13:04 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 02:13:04 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070113.CAA01767@loewis.home.cs.tu-berlin.de>

> I seriously wonder whether that was *the* problem with ABC: not only
> was 1.0 treated as an exact rational in ABC, so was 6.02e23 and
> 3.14159e-314 etc.  At least for me, this caused rationals to get
> used in many places I *intended* to use floats.  I assume many
> others got burned by this too, as I'd say it's impossible for users
> coming from other languages not to see 6.02e23 etc as float
> literals.

There seems to be a long tradition in Python of annotating literals to
get them interpreted in a different way; I think it would be
reasonable to tell apart floating point literals and rational literals
(with a power-of-ten denominator).

Specifically, the "scientific notation" could be used: 1.1 would be
exactly the same as 11/10, 1.1e0 would be binary floating point, and
only approximately equal to 11/10.

Regards,
Martin



From guido at python.org  Tue Nov  7 04:07:49 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 22:07:49 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:01:27 PST."
             <3A072A37.C2CAFED1@activestate.com> 
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>  
            <3A072A37.C2CAFED1@activestate.com> 
Message-ID: <200011070307.WAA10555@cj20424-a.reston1.va.home.com>

[Guido]
> > Warnings are more a human engineering issue than a technical issue!
> > That's also why I am emphasizing a C API -- I want to make it real
> > easy to ussue quality warnings in the runtime.  It's also why I
> > specify a rich (probably too rich!) filtering mechanism.

[Paul]
> Let me put out the strawman proposal that "grep" is a nicely orthogonal
> filtering mechanism. <duck!>
> 
> I guess my thinking is that you turn off warnings at the source. If you
> get a warning about __del__ exceptions then you put in a try/except. If
> you get a warning about an unused variable then you assign it to itself.
> If you get a warning about integer division then you should pass in a
> float and so forth.

I'm thinking of a different situation, where the user isn't very Unix
or Python sophisticated or the code generating warnings isn't easily
accessible for editing, or the fix isn't obvious (which I expect to be
often the case for __del__ and int division warnings).

Just as with exception handlers, where unqualified except clauses are
bad because of the risk that they mask real errors, I want to avoid
the likelihood that end users (those who can't or shouldn't fix the
errors) have to turn off all warnings.

> > > Syntax
> > >
> > >     assert >> cls, test[[[, arg], arg]...]
> > 
> > I have several problems with this.  First of all, using "assert" means
> > that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> > think that the decision to disable all warnings should be independent
> > from the decision to "optimize".  
> 
> Arguable. I see it as "release" and "debug" configurations. python and
> python_d.
> 
> > Second, you're hypergeneralizing the
> > extended print syntax.  Just because I think it's okay to add >>file
> > to the print syntax doesn't mean that it's now okay to add >>object
> > syntax to all statements!
> 
> Well getting new syntax into Python is really, really hard so we've got
> to squeeze out as much value out of what we have as possible. But
> anyhow, assertions are not allowed to 

(Did you forget to complete a sentence there?)

I say there's no need for new syntax, this can just be a function.

> You and I agree that there is an important sociological dimension to
> this. We can't require:
> 
>   import warnings
> 
>   warnings.warn("foo", "bar")
> 
> I prefer:
> 
>   warn Foo, "Bar"
> 
> just like:
> 
>   raise Foo, "Bar"

Hm.  Raise was made a statement because (1) Modula-3 did so, and (2)
it is a flow control modifier.  You haven't made a good case for why
warn() can't be a function.

> > I also don't see what warnings have to do with assertions.  Assertions
> > are a mechanism to check for error conditions.  What happens if the
> > error is detected is of less importance -- it could raise an exception
> > (Python), issue a fatal error (C), or do nothing (in -O mode).
> 
> Warnings are issued when an error or dubious construct is detected.
> Assertions are "fatal warnings". You agree that it is appropriate for
> some "warnings" to kill the app in some circumstances. Isn't it just a
> hop-skip-and-a-jump to say that warnings and errors are just points on
> the spectrum:
> 
> Report Once
> Report Always
> Report and Die

Possibly -- and then 'assert' is a poor choice of name for the
feature.  Objection denied.

> Python trained me to think of function calls and object constructions as
> being basically the same thing -- and keyword arguments and variable
> names being basically the same thing. etc.
> 
> > With warnings I believe the issue is not so much the detection of the
> > condition (for which a regular 'if' statement does just fine) but the
> > reporting.  Again, this is motivated by the fact that I expect that
> > flexible filtering is essential for a successful warning mechanism.
> 
> I don't see why assertions need special syntax but warnings do not! I
> would have been happy for a "warn" keyword with assert-like syntax but I
> don't think I'll see that in my lifetime.

Indeed.  But binding arbitrary unrelated semantics to an existing
statement with a carefully chosen name is poor design too.  You might
as well propose print<<file as a new input statement: they're both I/O
related. :-)

> > This is just a matter of exposition, but when I first read your PEP I
> > had a hard time figuring out the purpose of the cls object.  It wasn't
> > until I got to the very end where I saw your example classes that I
> > realized what it is: it represents a specific warning condition or a
> > group of related warning conditions.
> 
> It sort of evolved into an exception-like mechanism in that the class is
> instantiated with arguments just as exception classes are.
> 
> > >         if action=="error":
> > >             *** existing assertion code ***
> > 
> > That's just
> > 
> >               raise AssertionError, message
> > 
> > Right?
> 
> Well I'd like the traceback to emanate from the caller's position not
> the warning handler's position. Python doesn't really have a way to say
> that simply. This may well be implemented in C so it might not matter.

OK.

> > Suggestion: have separate warning and error handlers, so that if I
> > want to override these branches of the if statement I don't have to
> > repeat the entire handler.
> 
> Good idea.
> 
> > I see no definition of sys.disabled_warnings.  Instead of
> > sys.disabled_warnings you meant sys.suppress, right?
> 
> Right.
> 
> > >               return "suppress"
> > >           for base in cls.__bases__:
> > >               rc = self.get_user_request(base)
> > >               if rc:
> > >                   return rc
> > >           else:
> > >               return None
> > 
> > This approach (searching for the class name or the name of one of its
> > base classes in a list) doesn't look very object-oriented.  It would
> > make more sense to store the desired return value as a class or
> > instance attribute.  The default warning action could be stored on the
> > base class.
> 
> The way I came to this odd structure is I wanted most subclasses to be
> just no-ops as many exceptions are:
> 
> class ActiveStateLintWarning(Warning): pass
> 
> class CodeLooksLikePerl(ActiveStateLintWarning): pass
> 
> class TooMuchOnOneLine(CodeLooksLikePerl): pass

The idea of using a class hierarchy to classify warnings is definitely
a good one.

> So what __init__ could I write that would look up -wTooMuchOnOneLine and
> then if it failed that, look up -wCodeLooksLikePerl and so forth?

Yeah, it's not clear what to do.  You would like the -w option to poke
a value in a class variable, but the problem there is that it doesn't
know in which module the class is defined.  (On the other hand, if
that problem could be solved, it would be the preferred solution --
since it would solve the problem of typos in the -w argument neatly.)

> It gets pretty mind-bending because you sort of want one method to call
> another and yet you want it to be the *same* inherited method (i.e.
> don't have to code it each time). E.g. you want to run the *same method*
> at each level of the hierarchy. So I just do that directly.
> 
> > >     -w XXX => sys.warnings.append("XXX")
> > >     -e XXX => sys.errors.append("XXX")
> > >     -no-w XXX => sys.suppress.append("XXX")
> > >     -wall => sys.default_warning_action => "warn"
> > >     -eall => sys.default_warning_action => "error"
> > >     -no-wall => sys.default_warning_action => "suppress"
> > 
> > Python doesn't support long options (I don't *like* long options so I
> > doubt that this is a good occasion to start lobbying for them :-).  We
> > can come up with different options though.
> 
> Are these long options? Or just overloaded behavior on -w, -e, -n .
> Think of "all" as the base class for warnings or something.

Yes: if you change -no-w into -n, they can all be short options.  Note
that no matter what syntax we choose, we'll always be deviants
compared to Perl and GCC: those require a -w or -W option to *enable*
warnings.  (I found some great quotes in the Perl man page:

    Whenever you get mysterious behavior, try the -w switch!!!  Whenever
    you don't get mysterious behavior, try using -w anyway.

    The -w switch produces some lovely diagnostics.

    Did we mention that you should definitely consider using the -w
    switch?

    Bugs
        The -w switch is not mandatory.
)

> > >     As per the code above, errors take precedence over warnings and
> > >     warnings over suppressions unless a particular assertion class
> > >     specifies otherwise.
> > 
> > I would use a different precedence scheme: a more specific filter
> > takes precedence over a more general filter.  So -eall -wdubious would
> > mean that "dubious" class warnings are warnings but all others are
> > errors, and -wall -edubious would mean the opposite.
> 
> Good idea. I think the code may work that way already. :)

Indeed it does. :-)  Then what did you mean by your remark about the
precedence?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Tue Nov  7 05:03:59 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:03:59 +1300 (NZDT)
Subject: [Python-Dev] Warnings PEP
In-Reply-To: <3A072A37.C2CAFED1@activestate.com>
Message-ID: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>

Paul Prescod <paulp at ActiveState.com>:

> Assertions are "fatal warnings".

No, the failure of an assertion *causes* a fatal warning.
Assertions themselves are tests, not warnings.

I agree with Guido -- "assert" is the wrong verb, "warn"
is the right verb.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fdrake at users.sourceforge.net  Tue Nov  7 05:05:34 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Mon, 6 Nov 2000 20:05:34 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011070405.UAA32052@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From py-dev at zadka.site.co.il  Tue Nov  7 13:24:22 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 14:24:22 +0200
Subject: [Python-Dev] Re: Class/type dichotomy thoughts 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Mon, 06 Nov 2000 21:01:40 +0100." <3A070E24.B07FA7D7@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>  <3A070E24.B07FA7D7@lemburg.com> 
Message-ID: <E13t7ne-0005wj-00@darjeeling.zadka.site.co.il>

[MAL]
> Take e.g. dictionaries: you could easily add a new dictionary
> type which uses case-insensitive string keys by extending the
> existing dictionary type.
> 
> The new type would reuse most of the slots of the original
> type and only replace the ones needed for lookup with the
> new logic for case-insensitivity.
> 
> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object. The
> underlying type objects would be different though (and also
> the type object address which is currently used to identify
> a builtin type).

This kind of thing will also help a numeric system like PEP-0228
be implemented more easily.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From greg at cosc.canterbury.ac.nz  Tue Nov  7 05:19:46 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:19:46 +1300 (NZDT)
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A070E24.B07FA7D7@lemburg.com>
Message-ID: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object.

Hang on a minute. What sort of assumptions is the
interpreter going to be making based on the fact that
the type id is PyDict_TypeID? Can it be sure that this
new case-insensitive dictionary doesn't break them
somehow?

In other words, what does this new type_id thing
actually *mean*?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From paul at prescod.net  Tue Nov  7 06:57:40 2000
From: paul at prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 21:57:40 -0800
Subject: [Python-Dev] Warnings PEP
References: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>
Message-ID: <3A0799D4.86FC8148@prescod.net>

Greg Ewing wrote:
> 
> Paul Prescod <paulp at ActiveState.com>:
> 
> > Assertions are "fatal warnings".
> 
> No, the failure of an assertion *causes* a fatal warning.
> Assertions themselves are tests, not warnings.

Fine, some assertions are tests for fatal errors and some assertions are
tests for non-fatal mistakes.
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From moshez at math.huji.ac.il  Tue Nov  7 07:37:01 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:37:01 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <87ofzs39lo.fsf@indy.progeny.com>
Message-ID: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>

On 6 Nov 2000, Eric Gillespie, Jr. wrote:

> There has been a problem in Debian's python-gdk-imlib package for
> quite some time: Transparent PNGs do not display properly (see
> attached example script). According to upstream
> (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> the proper solution is either to have Python use RTLD_GLOBAL in
> dlopen calls when loading extension modules,

Or, possible, using dlmodule to dlopen things ourselves rather then
using Python's import facilities for that.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Tue Nov  7 07:41:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:41:56 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011070841110.13407-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> 
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

Oooops....I didn't mean to send it here (well, I did, then I change
my mind but forgot to tell my e-mail program about that)

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From paul at prescod.net  Tue Nov  7 07:54:14 2000
From: paul at prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 22:54:14 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A07A716.BDF9658B@prescod.net>

Guido van Rossum wrote:
> 
> Before I fall asleep let me write up my ideas about the warning
> framework.
> 
> Requirements:
> 
> - A C-level API that lets C code issue a warning with a single call
>   taking one or two arguments, e.g. Py_Warning(level, message).  (The
>   'level' argument is an example only; I'm not sure what if any we
>   need.)

I like the C-level API.

I think the "level" argument should be some sort of characterization,
not a number or enumeration. I think of it as being like an exception --
just another Python object.

Or it could be just a string. The main point is that filtering on a
number or enum is not flexible enough.

> - An equivalent Python level API, e.g. sys.warning(level, message).

I would prefer something easier to spell and with more of a central "you
should use this alot" feeling.

> Possible implementation:
> 
> - Each module can has a dictionary __warnings__ in its global
>   __dict__, which records the state of warnings.  It is created as an
>   emprt dict if it doesn't exist when it is needed.  The keys are
>   (message, linenumber) tuples (the module or file is implicit through
>   the use of the module's __dict__).  The value is None if no more
>   action is needed for this particular warning and location.  Some
>   other values may indicate the options "always print warning" (1?)
>   and "raise an exception" (-1?).

The problem with filtering based on line number is that it can't really
be done in a static manner because it is too fragile to code changes. In
my proposal every warning was assigned a "type" which would be the key
for filtering. A string would also be fine.

In general, I'm uncomfortable that I don't understand the requirements
enough. Are warnings something that the average Python programmer sees
rarely and then immediately goes in to fix the code so they don't see it
anymore (as compiler warnings are often handled)? Or do we expect most
Python programs to issue hundreds of warnings unless they are filtered.
Is filtering something you do constantly or as a partial workaround for
half-broken code that you can't fix right now?
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From moshez at math.huji.ac.il  Tue Nov  7 07:58:40 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:58:40 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <3A07A716.BDF9658B@prescod.net>
Message-ID: <Pine.GSO.4.10.10011070849580.13407-100000@sundial>

On Mon, 6 Nov 2000, Paul Prescod wrote:

> I like the C-level API.
> 
> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.

These are completely different things -- both should be there. An
exception doesn't need a level -- it's the highest level possible, saying
"get out of here, fast!". Warnings need both a level and characterization.
Having it a Python class is a nice touch, and keeps it consistent with
the way exceptions use classes for characterization.

> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

There are two main sources to copy from here:

gcc: You have -Wall, -Wadd-something, etc. Any warning you do see you
     either fix, or surround with a pragma so you don't see this.
     You also have -Werror to turn all warnings into errors.
Perl: -w gives runtime warnings for things any saner language would
      raise exceptions for. "dereferencing NULL", accessing non-existing
      elements in an array, etc. Warnings are serious bugs, and you must
      always use them.
Perl: "use strict" and friends: die because of some otherwise perfectly
      legal Perl if it's not declared properly.

I'd go for a more gcc-like approach: if you see a warning, you should
either 1. Silence it or 2. Fix it.
Silencing warnings is a serious issue: sometimes the programmer does
no better then the interpreter, and should have the ability to silence
any warning permanently -- otherwise he'll work with -silence_all, and
the benefit is lost.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From tim_one at email.msn.com  Tue Nov  7 09:38:35 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Tue, 7 Nov 2000 03:38:35 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBCHPAA.tim_one@email.msn.com>

[Guido]
> ...
> Note: it is open for debate whether the result of x/y for integer (or
> long integer) arguments should yield an integer (or long integer) in
> those cases where the result *is* representable as such (e.g. 4/2).
> It is possible that the numeric tower will render this problem moot --
> but that depends on what happens to Moshe's PEP 228, and that's a much
> longer story.

Note that for long ints i and j, i/j may (& easily so) be too large to
represent as a native double.  Rationals (of course) have no problem with
that.  It would certainly be curious if i*j didn't overflow but i/j did.

Just an observation -- I'm a fan of unintended consequences <wink>.





From mal at lemburg.com  Tue Nov  7 10:17:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 10:17:56 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>
Message-ID: <3A07C8C4.1F14985F@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com>:
> 
> > Then it sets the type id to PyDict_TypeID and Python will
> > use it as if it were an original dictionary object.
> 
> Hang on a minute. What sort of assumptions is the
> interpreter going to be making based on the fact that
> the type id is PyDict_TypeID? Can it be sure that this
> new case-insensitive dictionary doesn't break them
> somehow?
> 
> In other words, what does this new type_id thing
> actually *mean*?

For the interpreter it means that it can assume the type
interface to be binary compatible to the "original"
type, e.g. by setting the flag to say PyDict_TypeID
the type assures that all PyDict_*() APIs will work
on the type -- basically the same thing as PyDict_Check()
does now except that the type object needn't be the same
anymore.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Tue Nov  7 10:36:42 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 10:36:42 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
References: <200011070911.BAA13249@slayer.i.sourceforge.net>
Message-ID: <01e301c0489e$3e349540$0900a8c0@SPIFF>

Moshe Zadka wrote:
> Modified Files:
> pep-0216.txt 
> Log Message:
> Added structured text consensus.

when/where was this discussed?

</F>




From moshez at math.huji.ac.il  Tue Nov  7 10:49:54 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:49:54 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
In-Reply-To: <01e301c0489e$3e349540$0900a8c0@SPIFF>
Message-ID: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>

On Tue, 7 Nov 2000, Fredrik Lundh wrote:

> Moshe Zadka wrote:
> > Modified Files:
> > pep-0216.txt 
> > Log Message:
> > Added structured text consensus.
> 
> when/where was this discussed?

ummmm....doc-sig, where all things documentation-related are discussed?

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Tue Nov  7 10:58:41 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:58:41 +0200 (IST)
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:
 python/nondist/peps pep-0216.txt,1.3,1.4)
In-Reply-To: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> ummmm....doc-sig, where all things documentation-related are discussed?

Just to clarify my answer: I'm happy to receive comments/complaints
by private e-mail, or you can share thm with all of doc-sig. I prefer
to be CCed on doc-sig e-mail, but if not, I'll read it in the doc-sig.

Python-Dev does not seem the place for documentation-related discussion,
since that's exactly what doc-sig is for. It's quite low-volume, so
please join, or browse the archives.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fredrik at pythonware.com  Tue Nov  7 11:59:38 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 11:59:38 +0100
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:python/nondist/peps pep-0216.txt,1.3,1.4)
References: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>
Message-ID: <027f01c048a9$d324efa0$0900a8c0@SPIFF>

moshe wrote:
> Python-Dev does not seem the place for documentation-related discussion,
> since that's exactly what doc-sig is for. It's quite low-volume, so
> please join, or browse the archives.

oh, I thought I was subscribed.  guess I wasn't.

I'll check the archives.

</F>




From guido at python.org  Tue Nov  7 13:55:26 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 07:55:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Mon, 06 Nov 2000 22:54:14 PST."
             <3A07A716.BDF9658B@prescod.net> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>  
            <3A07A716.BDF9658B@prescod.net> 
Message-ID: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>

> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.
> 
> Or it could be just a string. The main point is that filtering on a
> number or enum is not flexible enough.

OK, let's make this a class then.  Just keep exceptions out of it
-- this is a separate, disjoint set of classes.  Let's call this
"warning category".  There will be standard categories and user code
can add categories.

> > - An equivalent Python level API, e.g. sys.warning(level, message).
> 
> I would prefer something easier to spell and with more of a central "you
> should use this alot" feeling.

OK, let's make this a built-in: warning(category, message).

> > Possible implementation:
> > 
> > - Each module can has a dictionary __warnings__ in its global
> >   __dict__, which records the state of warnings.  It is created as an
> >   emprt dict if it doesn't exist when it is needed.  The keys are
> >   (message, linenumber) tuples (the module or file is implicit through
> >   the use of the module's __dict__).  The value is None if no more
> >   action is needed for this particular warning and location.  Some
> >   other values may indicate the options "always print warning" (1?)
> >   and "raise an exception" (-1?).
> 
> The problem with filtering based on line number is that it can't really
> be done in a static manner because it is too fragile to code changes. In
> my proposal every warning was assigned a "type" which would be the key
> for filtering. A string would also be fine.
> 
> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

All of the above.

If I'm a programmer maintaining a piece of code, of course I turn all
warnings into errors and fix them as they occur.

But if I'm using someone else's code that happens to generate
warnings, it's better to disable warnings in that code until its
author has released a fixed version.  I want to be able to be very
specific, so that turning off warnings in 3rd party code doesn't
disable them in my own code.  If the 3rd party code generates a single
warning during normal use at a single location (e.g. if there's one
unconverted integer divide somehwere) then it's best to turn it off
just at that location, so that when I feed it other data which may
trigger other warnings elsewhere I will still get the benefit of the
warnings -- which *may* mean there's something wrong with my data, not
with the 3rd party code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Tue Nov  7 14:17:02 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 14:17:02 +0100
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Tue, Nov 07, 2000 at 02:06:04PM +1300
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
Message-ID: <20001107141702.H27208@xs4all.nl>

On Tue, Nov 07, 2000 at 02:06:04PM +1300, Greg Ewing wrote:
> > It's a new keyword though, which has a much
> > higher threshold for acceptance than a new two-character operator
> > symbol.

> It could be non-reserved, since a div b is currently
> a syntax error.

Except for the fact our current parser can't handle the 'a div b' syntax
without making 'div' a reserved word, which also makes 'x.div', 'class div:'
and 'def div():' invalid syntax. It might be work around-able, but .... :P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jim at interet.com  Tue Nov  7 14:28:16 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Tue, 07 Nov 2000 08:28:16 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com>
Message-ID: <3A080370.30C36339@interet.com>

"M.-A. Lemburg" wrote:

> Now I get this error after working in interactive Python
> mode with zipfile:
> 
> Exception exceptions.AttributeError:
> "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

Reading the code, I don't see how this could have happened
unless __init__ has already raised an exception.
 
> I would like a method .copy(self, name, output) which
> reads the file name from the ZIP archive and writes it directly to
> the file-like object output. This should copy the file in chunks
> of say 64kB in order to reduce memory load.

This is only a few lines of Python, and I generally omit
any methods which are not absoultely necessary.  Does
anyone else think this should be added?

JimA



From guido at python.org  Tue Nov  7 14:33:46 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 08:33:46 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Tue, 07 Nov 2000 14:17:02 +0100."
             <20001107141702.H27208@xs4all.nl> 
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>  
            <20001107141702.H27208@xs4all.nl> 
Message-ID: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>

> > > It's a new keyword though, which has a much
> > > higher threshold for acceptance than a new two-character operator
> > > symbol.
> 
> > It could be non-reserved, since a div b is currently
> > a syntax error.
> 
> Except for the fact our current parser can't handle the 'a div b' syntax
> without making 'div' a reserved word, which also makes 'x.div', 'class div:'
> and 'def div():' invalid syntax. It might be work around-able, but .... :P

No, we *could* use the 'import as' trick: define the syntax as

term: factor (('*'|'/'|'%'|NAME) factor)*

and add a check that NAME is "div" in the compiler.

But I don't know how comfy I am with a proliferation of hacks like
this -- and it's likely to cause more confusing error messages.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Tue Nov  7 15:46:15 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 15:46:15 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com> <3A080370.30C36339@interet.com>
Message-ID: <3A0815B7.BD0D9C4C@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> > Now I get this error after working in interactive Python
> > mode with zipfile:
> >
> > Exception exceptions.AttributeError:
> > "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored
> 
> Reading the code, I don't see how this could have happened
> unless __init__ has already raised an exception.

Probably has something to do with GC and cleaning up instances --
don't know. The error only shows up sometimes...

Instead of seeing the __del__ exception I now get the previous error
again:
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Nevermind... my setup must be broken in more way than I have
time to figure out now :-(
 
> > I would like a method .copy(self, name, output) which
> > reads the file name from the ZIP archive and writes it directly to
> > the file-like object output. This should copy the file in chunks
> > of say 64kB in order to reduce memory load.
> 
> This is only a few lines of Python, and I generally omit
> any methods which are not absoultely necessary.  Does
> anyone else think this should be added?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From pf at artcom-gmbh.de  Tue Nov  7 15:59:01 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Tue, 7 Nov 2000 15:59:01 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 7, 2000  8:33:46 am"
Message-ID: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum:
> No, we *could* use the 'import as' trick: define the syntax as
> 
> term: factor (('*'|'/'|'%'|NAME) factor)*
> 
> and add a check that NAME is "div" in the compiler.
> 
> But I don't know how comfy I am with a proliferation of hacks like
> this -- and it's likely to cause more confusing error messages.

In Modula-2 it is forbidden to declare a variable or procedure called
'DIV' or 'MOD', since both were reserved words in this language from
the very beginning.  But in Python the situation is different and
people might have used 'div' as an identifier.  So unless Guido is
able to fix this using the famous time machine ...sigh... for the
sake of backward compatibility using this "hack" seems to be the best
available choice.

I believe confusing error messages can be avoided and I see no 
"proliferation of hacks" in this two attempts to avoid defining
new keywords.  Keeping backward compatibility always had its price.
But it is often a price worth to pay.

Just my $0.02, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From gward at mems-exchange.org  Tue Nov  7 16:22:51 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Tue, 7 Nov 2000 10:22:51 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 07, 2000 at 07:55:26AM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com> <3A07A716.BDF9658B@prescod.net> <200011071255.HAA13355@cj20424-a.reston1.va.home.com>
Message-ID: <20001107102251.A15674@ludwig.cnri.reston.va.us>

On 07 November 2000, Guido van Rossum said:
> > Or it could be just a string. The main point is that filtering on a
> > number or enum is not flexible enough.
> 
> OK, let's make this a class then.  Just keep exceptions out of it
> -- this is a separate, disjoint set of classes.  Let's call this
> "warning category".  There will be standard categories and user code
> can add categories.

This sounds right -- I was going to suggest "warning class" instead of
"level", but "category" might be a better word.  My main rationale was
filtering: show me "integer divide" problems, but don't bother me with
"function argument not used".  (Hmm, those two sound more like specific
warnings rather than warning categories.  Probably the categories there
would be "arithmetic" and "dataflow".)

> > I would prefer something easier to spell and with more of a central "you
> > should use this alot" feeling.
> 
> OK, let's make this a built-in: warning(category, message).

Minor spelling nit: I would call it 'warn()' (or 'sys.warn()', or
'Py_Warn()', etc.) since that's a verb.

More importantly: if 'warn(ing)' is meant to be used mainly for
compiler-style warnings -- you're using this language or library feature
inappropriately -- then it should be left in sys.  But if it's meant to
also be used for printing some message to stderr (like Perl's 'warn'),
then there's a good case for making it a builtin.

Almost every Python script I write features

    def warn (msg):
        sys.stderr.write("warning: " + msg + "\n")

That might be a clue that something (albeit a tiny thing) is missing
from the language.  ;-)

        Greg



From fdrake at acm.org  Tue Nov  7 16:30:00 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 10:30:00 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
	<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
	<20001107141702.H27208@xs4all.nl>
	<200011071333.IAA13497@cj20424-a.reston1.va.home.com>
Message-ID: <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > But I don't know how comfy I am with a proliferation of hacks like
 > this -- and it's likely to cause more confusing error messages.

  If "div" is it, I'd rather see it made a keyword and a warning
published to the community soon so that people have a chance to check
their code and either make it compatible with the change or scream
ahead of time.
  A tool to help them out wouldn't hurt, either, and that could be
written before any actual changes are made or even final decisions are
made -- it could search everything on sys.path and report on uses that
would be affected by each candidate change.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Tue Nov  7 17:14:21 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 11:14:21 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."
             <3A06BEF4.95B773BD@tismer.com> 
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>  
            <3A06BEF4.95B773BD@tismer.com> 
Message-ID: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>

I've thought about it a little more, and drawn some pictures in my
head.

I still have to disagree with Christian when he says:

> Making Python completely coroutine aware, without
> tricking the C stack, is 90 percent of the problem.
> But after walking that far, there is no reason
> to leave the other 10 percent alone.

Without continuations, but with microthreads (uthreads) or coroutines,
each (Python) stack frame can simply be "paused" at a specific point
and continued later.  The semantics here are completely clear (except
perhaps for end cases such as unhandled exceptions and intervening C
stack frames).

With continuations, you have to decide how much state to save for a
future continuation.  It would seem easy enough: save all state kept
in the frame except for the local variables.  But now consider this:
the "evaluation stack" contained in each frame could easily be
replaced by a bunch of temporary variables, if we had a slightly
different instruction set (3-address opcodes instead of stack-based
opcodes).  Then would we save those temporary variables or not?  it
can make a difference!  Since the "save continuation" operation is a
function call, you can easily save a continuation while there are some
items on the value stack.  I believe the current implementation saves
these so they are restored when you jump to the continuation.  But if
the compiler were to use temporary variables instead of the evaluation
stack, they might not have been restored!

Here's another example.  Suppose you set up a for loop.  After three
iterations through the loop you save a continuation.  Then you finish
hree more iterations.  Then you return to the saved continuation.
Where does the loop continue: at 3 or at 6 iterations?  Try to answer
this without trying it.  My guess: it gets restarted at 3 iterations,
because the loop index is saved on the value stack.  If you rewrite
this using a while loop, however, it would get restarted at 6
iterations, because then your loop index is an unsaved local variable.
Ditto if you changed the bytecode compiler so for loops use an
anonymous local variable instead of an entry on the evaluation
stack.

This semantic imprecision is one of the reasons why I don't like the
concept of continuations.  (I've been told that the exact semantics of
continuations in Scheme differ greatly between Scheme implementations.)

Now let's look at Jython.  In Jython, we can simulate "paused frames"
well enough by using real Java threads.  However full continuations
would require modifications to the JVM -- which is unacceptable to a
language boasting "100% Pure Java".  Another reason against allowing
continuations.

So, all in all, I don't think of continuations as "the last 10% that
we might as well add to finish the job."  I see it as an ill-specified
hypergeneralization.

What *do* I want to see in a stackless PEP?

Not surprisingly, I'd like to see generators, coroutines, and
uthreads.  These all share a mechanism that pauses one frame and
resumes another.  I propose to make the concept of uthreads
fundamental -- this will simplify the emulation in Jython.


A strawman proposal:

The uthread module provides the new functionality at the lowest level.
Uthread objects represent microthreads.  An uthread has a chain of
stack frames linked by back pointers just like a regular thread.
Pause/resume operations are methods on uthread objects.  Pause/resume
operations do not address specific frames but specific uthreads;
within an uthread the normal call/return mechanisms can be used, and
only the top frame in the uthread's stack of call frames can be
paused/resumed (the ones below it are paused implicitly by the call to
the next frame, and resumed when that call returns).

- u = uthread.new(func) creates a new uthread object, u.  The new
  uthread is poised to call func() but doesn't execute yet.

- u = uthread.current() returns the uthread object for the current
  frame.

- u.yield() pauses the current uthread and resume the uthread u where
  it was paused.  The current uthread is resumed when some other
  uthread calls its yield() method.  Calling uthread.current().yield()
  is a no-op.

- When func() returns, the uthread that was executing it ceases to be
  runnable.  The uthread that most recently yielded to it is resumed,
  unless that is no longer runnable, in which case the uthread that
  most recently yielded to *it* is resumed, and so on until a runnable
  uthread is found or until no runnable uthreads are left, in which
  case the program terminates.  (XXX I need a proof here that this
  works.)

- When func() raises an unhandled exception, the exception gets
  propagated using the same rules as when it returns, and similarly
  its uthread ceases to be runnable.

- u.kill(exc) causes the yield() call that paused u to raise the
  exception exc.  (This can be caught in a try/except of course.)

- Calling u.yield() or u.kill() for a non-runnable uthread is an error
  and raises an exception.

I think this API should enough to implement Gordon's SelectDispatcher
code.  In general, it's easy to create a scheduler uthread that
schedules other uthreads.


Open issues:

- I'm not sure that I got the start conditions right.  Should func() be
  be allowed to run until its first yield() when uthread.new(func) is
  called?

- I'm not sure that the rules for returning and raising exceptions
  from func() are the right ones.

- Should it be possible to pass a value to another uthread by passing
  an argument to u.yield(), which then gets returned by the resumed
  yield() call in that uthread?

- How do uthreads interact with real threads?  Uthreads are explicitly
  scheduled through yield() calls; real threads use preemptive
  scheduling.  I suppose we could create a new "main" uthread for each
  real thread.  But what if we yield() to an uthread that's already
  executing in another thread?  How is that error detected?


Please help!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Tue Nov  7 17:30:16 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 10:30:16 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14856.11800.444274.851825@buffalo.fnal.gov>

 > Guido van Rossum:
 > > No, we *could* use the 'import as' trick: define the syntax as
 > > 
 > > term: factor (('*'|'/'|'%'|NAME) factor)*
 > > 
 > > and add a check that NAME is "div" in the compiler.
 > > 
 > > But I don't know how comfy I am with a proliferation of hacks like
 > > this -- and it's likely to cause more confusing error messages.

And what is the compelling reason for going through all this instead
of just using the '//' symbol?  Because it might be confused for a C++
comment?  This is a weak argument AFAIAC.  Python is not C++ and
everybody knows that.

I think that making "div" an infix operator would be setting a
horrible precedent.  Currently, all infix operators "look like"
operators, i.e. they are non-alphabetic characters, and things that
look like words are either functions or reserved words.

There's already a "divmod" builtin which is a function, not an infix
operator.  I think it would be rather inconsistent to write, on the
one hand:

    divmod(10, 2)

and on the other:

    10 div 2


Long before the creation of python-dev, this issue had been discussed
numerous times on c.l.py, and the '//' operator was suggested several
times, and I don't think anybody ever had a problem with it...



From gvwilson at nevex.com  Tue Nov  7 17:41:25 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Tue, 7 Nov 2000 11:41:25 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>

>  > Guido van Rossum:
>  > > No, we *could* use the 'import as' trick: define the syntax as
>  > > term: factor (('*'|'/'|'%'|NAME) factor)*
>  > > and add a check that NAME is "div" in the compiler.

> Charles G. Waldman:
> And what is the compelling reason for going through all this instead
> of just using the '//' symbol?  Because it might be confused for a C++
> comment?  This is a weak argument AFAIAC.  Python is not C++ and
> everybody knows that.
> Long before the creation of python-dev, this issue had been discussed
> numerous times on c.l.py, and the '//' operator was suggested several
> times, and I don't think anybody ever had a problem with it...

Greg Wilson:
As someone who teaches Python, I'm strongly opposed to using '//' in the
same language as '/', purely on readability grounds:

1. Every C/C++ book includes a warning about "=" vs. "==", because it's
   a common hard-to-spot error.

2. What mark would you give a student who had variables IO and I0 in the
   same module?

Greg

p.s. I was very disappointed to discover that Ruby uses both '..' and '...'
One means "up to but not including", the other means "up to and including".
It would be interesting to estimate the number of programmer-hours this will
cost... :-)




From thomas at xs4all.net  Tue Nov  7 18:26:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 18:26:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <200011071544.HAA31147@slayer.i.sourceforge.net>; from gward@users.sourceforge.net on Tue, Nov 07, 2000 at 07:44:24AM -0800
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
Message-ID: <20001107182621.I27208@xs4all.nl>

On Tue, Nov 07, 2000 at 07:44:24AM -0800, Greg Ward wrote:

>   - when compiling with GCC on any platform, add "-fPIC" to OPT
>     (without this, "$(CC) -shared" dies horribly)

Sorry for the late remark (I did see your earlier message) but after reading
the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
for non-essential stuff: warnings, debug-info and optimizations. Removing
things from OPT shouldn't break anything, and neither should adding options
that fit in the categories above. (Barring broken compilers, of course.)

Instead, the -fPIC option should be added to CFLAGS, I think. The Python
autoconf setup is slightly less versatile than most, though, since it's
doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
without editing configure(.in) :P If noone else does it before me, I'll see
about fixing at least the -fPIC thing later, when I find some time ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From paulp at ActiveState.com  Tue Nov  7 18:29:10 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 09:29:10 -0800
Subject: [Python-Dev] Integer division transition
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
		<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
		<20001107141702.H27208@xs4all.nl>
		<200011071333.IAA13497@cj20424-a.reston1.va.home.com> <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
Message-ID: <3A083BE6.4A502210@activestate.com>

"Fred L. Drake, Jr." wrote:
> 
> ...
>   A tool to help them out wouldn't hurt, either, and that could be
> written before any actual changes are made or even final decisions are
> made -- it could search everything on sys.path and report on uses that
> would be affected by each candidate change.

I think that the standard Python compiler is the appropriate tool for
this sort of thing. Anything that can be caught "statically" might as
well be implemented right in the compiler (at least from the user's
point of view) rather than in a separate "deprecation nanny."

 Paul Prescod



From nas at arctrix.com  Tue Nov  7 12:45:18 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 7 Nov 2000 03:45:18 -0800
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>; from gvwilson@nevex.com on Tue, Nov 07, 2000 at 11:41:25AM -0500
References: <14856.11800.444274.851825@buffalo.fnal.gov> <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <20001107034518.A12431@glacier.fnational.com>

On Tue, Nov 07, 2000 at 11:41:25AM -0500, Greg Wilson wrote:
> As someone who teaches Python, I'm strongly opposed to using '//' in the
> same language as '/', purely on readability grounds:

How do you feel about div(x, y)?

  Neil



From cgw at fnal.gov  Tue Nov  7 19:50:18 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 12:50:18 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
References: <14856.11800.444274.851825@buffalo.fnal.gov>
	<NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <14856.20202.174151.647981@buffalo.fnal.gov>

Greg Wilson writes:
 > 
 > 2. What mark would you give a student who had variables IO and I0 in the
 >    same module?
 > 

I think this is a bit of a stretch - IO and I0 look almost identical
typographically (depending on the font) whereas // and / look pretty
different.  It would be a better analogy to say "What mark would you
give a student who used variables X and XX in the same program".  And,
I wouldn't have a problem with that.

How about URL's?  '/' and '//' have different meanings there and I
don't think people have a big problem with this.

The other point - "=" vs "==" - is a bit harder to answer.  Both of
these symbols are used in Python, but not in the same context.

All-the-good-symbols-are-already-taken-ly y'rs,
					//C





From gstein at lyra.org  Tue Nov  7 19:49:12 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 7 Nov 2000 10:49:12 -0800
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 07, 2000 at 08:37:01AM +0200
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <20001107104912.Q14054@lyra.org>

On Tue, Nov 07, 2000 at 08:37:01AM +0200, Moshe Zadka wrote:
> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

There was quite a long conversation [on python-dev] a while back (geez, a
year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
were pros and cons for both directions, and I believe some compatibility
issues.

You may be able to find the conversation, then figure out why Python chose
its current mechanism. Heck... maybe it should change :-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From akuchlin at mems-exchange.org  Tue Nov  7 20:11:06 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 7 Nov 2000 14:11:06 -0500
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <20001107104912.Q14054@lyra.org>; from gstein@lyra.org on Tue, Nov 07, 2000 at 10:49:12AM -0800
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial> <20001107104912.Q14054@lyra.org>
Message-ID: <20001107141106.A10897@kronos.cnri.reston.va.us>

On Tue, Nov 07, 2000 at 10:49:12AM -0800, Greg Stein wrote:
>There was quite a long conversation [on python-dev] a while back (geez, a
>year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
>were pros and cons for both directions, and I believe some compatibility
>issues.

kronos Python>cvs log importdl.c 
  ...
revision 2.47
date: 1998/05/18 13:42:45;  author: guido;  state: Exp;  lines: +4 -6
Remove use of RTLD_GLOBAL.
----------------------------                                                    ...
revision 2.41
date: 1997/12/02 20:43:18;  author: guido;  state: Exp;  lines: +7 -3
Add the flag RTLD_GLOBAL to the dlopen() options.
This exports symbols defined by the loaded extension to other
extensions (loaded later).
 
(I'm not quite sure about this but suppose it can't hurt...)
----------------------------

Adding RTLD_GLOBAL in one version, removing it in the next: a new
Python tradition!

--amk




From est at hyperreal.org  Tue Nov  7 20:04:33 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Tue, 7 Nov 2000 11:04:33 -0800 (PST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 7, 2000 11:14:21 am"
Message-ID: <20001107190433.26578.qmail@hyperreal.org>

Guido van Rossum discourseth:
> 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest level.

I really like this as a primitive appropriate for Python's evolution.

> - When func() returns, the uthread that was executing it ceases to be
>   runnable.  The uthread that most recently yielded to it is resumed,
>   unless that is no longer runnable, in which case the uthread that
>   most recently yielded to *it* is resumed, and so on until a runnable
>   uthread is found or until no runnable uthreads are left, in which
>   case the program terminates.  (XXX I need a proof here that this
>   works.)

I'd like it added that when a uthread chains to its yielder it drops
(i.e., nulls and decrefs) the reference to that yielder.  I want
uthreads in some of the same applications in which i disable gc for
real-time purposes, and I don't want circular structures of unrunnable
uthreads leaking my memory.

> - Calling u.yield() or u.kill() for a non-runnable uthread is an error
>   and raises an exception.

A runnable() predicate might be nice.

> - I'm not sure that I got the start conditions right.  Should func() be
>   be allowed to run until its first yield() when uthread.new(func) is
>   called?

+1 for no on this.

> - I'm not sure that the rules for returning and raising exceptions
>   from func() are the right ones.

I'm particularly unsure about the exception propagation.  It could
always be disabled by a universal exception handler in the uthread,
but I'm not sure it's even worth the implementation effort.

> - Should it be possible to pass a value to another uthread by passing
>   an argument to u.yield(), which then gets returned by the resumed
>   yield() call in that uthread?

Certainly! :)

As written, the strawman seems to require that the dread intervening C
stack frames are handled transparently (since it doesn't say anything
about them).  This seems pretty important to me.  An instance method
may well know that it should yield, yet not know that it's being
called as a callback from a class/type that's just been moved to C.

OTOH..not handling this transparently would increase my market value
as a Python programmer.  Handling it right might get me some unpaid
work implementing some of the low-level details for Linux.  Hmm! :D

Eric




From paulp at ActiveState.com  Tue Nov  7 21:33:19 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 12:33:19 -0800
Subject: [Python-Dev] Integer division transition
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
		<m13tADF-000Dm8C@artcom0.artcom-gmbh.de> <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <3A08670F.A4703F32@activestate.com>

Charles G Waldman wrote:
> 
> ...
> 
> 
> I think that making "div" an infix operator would be setting a
> horrible precedent.  

I think it would be a good precedent because it is a cleaner upgrade
path to things like

matrixdiv, matrixmul, ...

 Paul Prescod



From tismer at tismer.com  Tue Nov  7 20:54:14 2000
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 07 Nov 2000 21:54:14 +0200
Subject: [Python-Dev] Re: uthread strawman
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>  
	            <3A06BEF4.95B773BD@tismer.com> <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <3A085DE6.28593202@tismer.com>

Just answering/clarifying a few bits,
since I can't change your opinion about
continuations, anyway.

Guido van Rossum wrote:
> 
> I've thought about it a little more, and drawn some pictures in my
> head.

I have to agree with Guido when he says:

> I still have to disagree with Christian when he says:
> 
> > Making Python completely coroutine aware, without
> > tricking the C stack, is 90 percent of the problem.
> > But after walking that far, there is no reason
> > to leave the other 10 percent alone.

... since I meant implementability. Of course there are
other reasons gainst continuations. I just did it since they
were in reach.

> Without continuations, but with microthreads (uthreads) or coroutines,
> each (Python) stack frame can simply be "paused" at a specific point
> and continued later.  The semantics here are completely clear (except
> perhaps for end cases such as unhandled exceptions and intervening C
> stack frames).

I agree. But also with continuations, the situation is identical,
as long as you don't try anything else where continuations
would be needed.
Note that they will not need to be created when the mentioned
structures are implemented well. We don't have to implement them,
but providing support for them in the interpreter framework
is simple. (that's the 10% issue).

> With continuations, you have to decide how much state to save for a
> future continuation.  It would seem easy enough: save all state kept
> in the frame except for the local variables.  But now consider this:
> the "evaluation stack" contained in each frame could easily be
> replaced by a bunch of temporary variables, if we had a slightly
> different instruction set (3-address opcodes instead of stack-based
> opcodes).  Then would we save those temporary variables or not?  it
> can make a difference!  Since the "save continuation" operation is a
> function call, you can easily save a continuation while there are some
> items on the value stack.  I believe the current implementation saves
> these so they are restored when you jump to the continuation.  But if
> the compiler were to use temporary variables instead of the evaluation
> stack, they might not have been restored!

I would consider these temporary variables registers which must
be preserved. They are immutable objects as part of the immutable
continuation, treated as values. Stack or registers, this is part
of an expression evaluation. Temporary results must conceptually
be read-only, whatever way I implement this.

> Here's another example.  Suppose you set up a for loop.  After three
> iterations through the loop you save a continuation.  Then you finish
> hree more iterations.  Then you return to the saved continuation.
> Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> this without trying it.  My guess: it gets restarted at 3 iterations,
> because the loop index is saved on the value stack.  If you rewrite
> this using a while loop, however, it would get restarted at 6
> iterations, because then your loop index is an unsaved local variable.
> Ditto if you changed the bytecode compiler so for loops use an
> anonymous local variable instead of an entry on the evaluation
> stack.

Wrong guess!
Exactly for that reason I changed the loop code to put a mutable
loopcounter object on the stack.
The loop works perfectly.

> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.  (I've been told that the exact semantics of
> continuations in Scheme differ greatly between Scheme implementations.)

In a sense, you have continuations already, also with the restriction
to gen/co/uthread structures. The only difference is to treat a
frame as exactly one continuation and to disallow to have more
than one at any time.
This saves the decision about the ambiguities you mentioned.

I agree that going to this point and not further for the
Python core is a good idea.
A PEP doesn't need to name continuations at all.

On the other hand, I don't see a reason why this hsould mean
that Python *must not* support them. What I'd like to keep
is the possibility to still write such an extension module.

Enabling this for educational purposes is a great value
that comes at a cheap price and no impact for the core.

> Now let's look at Jython.  In Jython, we can simulate "paused frames"
> well enough by using real Java threads.  However full continuations
> would require modifications to the JVM -- which is unacceptable to a
> language boasting "100% Pure Java".  Another reason against allowing
> continuations.

Not quite true, after I heard of a paper that shows how
to implement continuations in Java, using threads.
But I'll come back to that when I have the paper.

> So, all in all, I don't think of continuations as "the last 10% that
> we might as well add to finish the job."  I see it as an ill-specified
> hypergeneralization.

Can we agree to not support them without forbidding them?

...
> A strawman proposal:

Ok, this looks all very well to me. More on that later.
One question: Why do you want an explicit u.yield() ?
Uthreads are scheduled automatically now, like real
threads. Do you see a major drawback in supporting
this, maybe as an option? Or do you see automatic
scheduling as an extra construct on top with a special
"scheduler" uthread?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From jack at oratrix.nl  Tue Nov  7 23:57:53 2000
From: jack at oratrix.nl (Jack Jansen)
Date: Tue, 07 Nov 2000 23:57:53 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts 
In-Reply-To: Message by "M.-A. Lemburg" <mal@lemburg.com> ,
	     Tue, 07 Nov 2000 10:17:56 +0100 , <3A07C8C4.1F14985F@lemburg.com> 
Message-ID: <20001107225758.B77651301D9@oratrix.oratrix.nl>

> > In other words, what does this new type_id thing
> > actually *mean*?
> 
> For the interpreter it means that it can assume the type
> interface to be binary compatible to the "original"
> type, e.g. by setting the flag to say PyDict_TypeID
> the type assures that all PyDict_*() APIs will work
> on the type -- basically the same thing as PyDict_Check()
> does now except that the type object needn't be the same
> anymore.

I would be _very_ happy if this single type_id could somehow be
replaced by an array, or a bitset.

I have a lot of types in MacPython that are acceptable to the APIs of
other types, a sort of poor-mans-inheritance scheme. For instance, all 
operating system calls that accept a MacOS WindowPtr will also happily 
accept a DialogPtr. Major magic is needed to get this to work
reasonably in Python, and the Python user can still accidentally mess
up the refcounting scheme and free things s/he isn't aware of.

As the number of types in a given run of the interpreter appears to be 
limited (am I right here?) and type-identity-tests are valid within a
single interpreter run only (am I right here?) an API like
  typeindex = Py_TypeToTypeIndex(typeobject);
which would use a dictionary as storage for the mapping and generate
the index numbers on the fly would do the trick. Call it once during
module initalization and the
  Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
macro would be a oneliner to test the bit in the set.

A few hundred bits in the set would get us a long way, I guess.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen at oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 



From fdrake at acm.org  Wed Nov  8 00:06:25 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:06:25 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
	<14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > I think that making "div" an infix operator would be setting a
 > horrible precedent.  Currently, all infix operators "look like"
 > operators, i.e. they are non-alphabetic characters, and things that
 > look like words are either functions or reserved words.

  Like "is", "in", "is not", and "not in"?

 > Long before the creation of python-dev, this issue had been discussed
 > numerous times on c.l.py, and the '//' operator was suggested several
 > times, and I don't think anybody ever had a problem with it...

  I don't have a strong preference for either // or div, but
definately want this to be an operator.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov  8 00:17:10 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:17:10 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <3A083BE6.4A502210@activestate.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
	<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
	<20001107141702.H27208@xs4all.nl>
	<200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
	<3A083BE6.4A502210@activestate.com>
Message-ID: <14856.36214.267784.220958@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I think that the standard Python compiler is the appropriate tool for
 > this sort of thing. Anything that can be caught "statically" might as
 > well be implemented right in the compiler (at least from the user's
 > point of view) rather than in a separate "deprecation nanny."

  For linting programs using the final specification, this fine.  I'm
thinking that a tool to read over people's sources and say "'div' is
used in 120 places out of 56K lines of code." would be helpful because
we could determine the extent of the effect of using "div" instead of
"//".  If there are a limited number of projects affected, it may be
entirely reasonable to find out that there aren't enough uses to worry
about, and it becomes acceptable to make it a keyword (like the other
textual infix operators).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From greg at cosc.canterbury.ac.nz  Wed Nov  8 01:42:40 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 13:42:40 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.20202.174151.647981@buffalo.fnal.gov>
Message-ID: <200011080042.NAA00284@s454.cosc.canterbury.ac.nz>

Charles G Waldman <cgw at fnal.gov>:

> The other point - "=" vs "==" - is a bit harder to answer.

I think the reason this causes so much trouble is that many
programming languages, not to mention mainstream mathematics,
use "=" to mean what C uses "==" for.

Other such pairs in C, e.g. "&" vs "&&" and "+" vs "++",
don't seem to cause anywhere near as much difficulty, so
I don't think the problem is one of visual confusion, but
of semantic confusion.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+





From guido at python.org  Wed Nov  8 02:34:49 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 20:34:49 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Tue, 07 Nov 2000 11:04:33 PST."
             <20001107190433.26578.qmail@hyperreal.org> 
References: <20001107190433.26578.qmail@hyperreal.org> 
Message-ID: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>

[Note: the stackless at starship.python.net list seems to have trouble
again.  BeOpen may have crashed the machine again. :-( ]

[Note: response to Christian at end.]

[Guido]
> > 
> > A strawman proposal:
> > 
> > The uthread module provides the new functionality at the lowest level.

Eric (Tiedemann, right?  There are two Erics here -- it would help if
you signed your full name :-) writes:

> I really like this as a primitive appropriate for Python's evolution.

Cool.  I think I spoke too soon when I called it uthreads -- these are
really more like coroutines.

I also forgot to mention that I am assuming with this strawman that we
can use an essentially stackless implementation of the PVM.  I hope
that it will be possible to make it a lot simpler than current
stackless though, by not doing continuations.  Freezing a frame in
place is a lot simpler than freezing for multiple uses, which requires
one to decide what to copy and what to share!

> > - When func() returns, the uthread that was executing it ceases to be
> >   runnable.  The uthread that most recently yielded to it is resumed,
> >   unless that is no longer runnable, in which case the uthread that
> >   most recently yielded to *it* is resumed, and so on until a runnable
> >   uthread is found or until no runnable uthreads are left, in which
> >   case the program terminates.  (XXX I need a proof here that this
> >   works.)
> 
> I'd like it added that when a uthread chains to its yielder it drops
> (i.e., nulls and decrefs) the reference to that yielder.  I want
> uthreads in some of the same applications in which i disable gc for
> real-time purposes, and I don't want circular structures of unrunnable
> uthreads leaking my memory.

Good point.

> > - Calling u.yield() or u.kill() for a non-runnable uthread is an error
> >   and raises an exception.
> 
> A runnable() predicate might be nice.

Yes.

> > - I'm not sure that I got the start conditions right.  Should func() be
> >   be allowed to run until its first yield() when uthread.new(func) is
> >   called?
> 
> +1 for no on this.

You're being unnecessarily cryptic.  "Yes for no"?  So you're for the
original proposal (which doesn't start func() at all until it is
yielded to for the first time).

> > - I'm not sure that the rules for returning and raising exceptions
> >   from func() are the right ones.
> 
> I'm particularly unsure about the exception propagation.  It could
> always be disabled by a universal exception handler in the uthread,
> but I'm not sure it's even worth the implementation effort.

Agreed.  We may have to experiment.

> > - Should it be possible to pass a value to another uthread by passing
> >   an argument to u.yield(), which then gets returned by the resumed
> >   yield() call in that uthread?
> 
> Certainly! :)

This affects the initial condition.  If u hasn't called func() yet,
and I call u.yield(42), where does the 42 go?  Does it call func(42)?
That may make it make it unnecessarily hard to get the end conditions
right for func().  Again, we'll have to write some sample code to see
how this turns out in practice.

> As written, the strawman seems to require that the dread intervening C
> stack frames are handled transparently (since it doesn't say anything
> about them).  This seems pretty important to me.  An instance method
> may well know that it should yield, yet not know that it's being
> called as a callback from a class/type that's just been moved to C.

Not sure what you meant by intervening.  I certainly intended Python
to Python calls to be handled without creating extra C stack frames.
When Python calls C which calls back into Python, this is considered
all part of the same uthread.

Where it gets tricky is when this spawns a new uthread, which also
calls C which calls Python.  Now the second uthread has a C stack
frame above the C stack frame that's part of the first uthread.  This
means that the second uthread must return from its C code before the
first uthread can return to its C code!  A little extra bookkeeping
will be necessary to check for this -- so that when it is attempted an
exception is raised, rather than a return attempted from the wrong
uthread back into C.  This is the same as for current stackless.  The
solution is simply that the application "shouldn't do that."

> OTOH..not handling this transparently would increase my market value
> as a Python programmer.  Handling it right might get me some unpaid
> work implementing some of the low-level details for Linux.  Hmm! :D

If you want do hack continuations in C, be my guest -- as long as you
stay 10,000 kilometers away from core Python. :-)

[Now replying to Christian:]

> Just answering/clarifying a few bits,
> since I can't change your opinion about
> continuations, anyway.

Right!

> Guido van Rossum wrote:
> > 
> > I've thought about it a little more, and drawn some pictures in my
> > head.
> 
> I have to agree with Guido when he says:
> 
> > I still have to disagree with Christian when he says:
> > 
> > > Making Python completely coroutine aware, without
> > > tricking the C stack, is 90 percent of the problem.
> > > But after walking that far, there is no reason
> > > to leave the other 10 percent alone.
> 
> ... since I meant implementability. Of course there are
> other reasons gainst continuations. I just did it since they
> were in reach.

Hm.  Having seen a few fragments of your implementation today (just a
very little bit, since we were having an all-day meeting) I feel that
there are a lot of extra hacks needed to make the reuse of
continuations necessary.  This shouldn't be needed in my version.

> > Without continuations, but with microthreads (uthreads) or coroutines,
> > each (Python) stack frame can simply be "paused" at a specific point
> > and continued later.  The semantics here are completely clear (except
> > perhaps for end cases such as unhandled exceptions and intervening C
> > stack frames).
> 
> I agree. But also with continuations, the situation is identical,
> as long as you don't try anything else where continuations
> would be needed.

But the complexity in the code still exists because a continuation
*could* be reused and you don't know if it will ever happen so you
must be prepared.

> Note that they will not need to be created when the mentioned
> structures are implemented well. We don't have to implement them,
> but providing support for them in the interpreter framework
> is simple. (that's the 10% issue).

Having seen your code (frankly, a mess!) I don't believe that it's
only 10% at all.

> > With continuations, you have to decide how much state to save for a
> > future continuation.  It would seem easy enough: save all state kept
> > in the frame except for the local variables.  But now consider this:
> > the "evaluation stack" contained in each frame could easily be
> > replaced by a bunch of temporary variables, if we had a slightly
> > different instruction set (3-address opcodes instead of stack-based
> > opcodes).  Then would we save those temporary variables or not?  it
> > can make a difference!  Since the "save continuation" operation is a
> > function call, you can easily save a continuation while there are some
> > items on the value stack.  I believe the current implementation saves
> > these so they are restored when you jump to the continuation.  But if
> > the compiler were to use temporary variables instead of the evaluation
> > stack, they might not have been restored!
> 
> I would consider these temporary variables registers which must
> be preserved. They are immutable objects as part of the immutable
> continuation, treated as values. Stack or registers, this is part
> of an expression evaluation. Temporary results must conceptually
> be read-only, whatever way I implement this.

I heard from Tim that he helped you get this right.  The fact that it
is so hard to know the requirements for a practical implementation
makes me very worried that continuations may have hidden bugs.

> > Here's another example.  Suppose you set up a for loop.  After three
> > iterations through the loop you save a continuation.  Then you finish
> > hree more iterations.  Then you return to the saved continuation.
> > Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> > this without trying it.  My guess: it gets restarted at 3 iterations,
> > because the loop index is saved on the value stack.  If you rewrite
> > this using a while loop, however, it would get restarted at 6
> > iterations, because then your loop index is an unsaved local variable.
> > Ditto if you changed the bytecode compiler so for loops use an
> > anonymous local variable instead of an entry on the evaluation
> > stack.
> 
> Wrong guess!
> Exactly for that reason I changed the loop code to put a mutable
> loopcounter object on the stack.
> The loop works perfectly.

Wow.  i'm impressed.  You must have borrowed my time machine. :-)

Still, I believe there was a time when the loop *didn't* work
perfectly yet. It is really hard to know what is needed.  Are you
*sure* that it now *always* does the right thing?  What if I save a
continuation in the middle of a shortcut Boolean expression (and/or
stuff)?  Or in cases like a<b<c?  (Here b gets saved on the stack to
avoid loading it again.)

> > This semantic imprecision is one of the reasons why I don't like the
> > concept of continuations.  (I've been told that the exact semantics of
> > continuations in Scheme differ greatly between Scheme implementations.)
> 
> In a sense, you have continuations already, also with the restriction
> to gen/co/uthread structures. The only difference is to treat a
> frame as exactly one continuation and to disallow to have more
> than one at any time.
> This saves the decision about the ambiguities you mentioned.

Yeah, right.  You can call pausable/resumable frames use-once
continuations if you want to.  And if that gives you the happy feeling
that I "support" continuations, fine.

> I agree that going to this point and not further for the
> Python core is a good idea.
> A PEP doesn't need to name continuations at all.

Good.

> On the other hand, I don't see a reason why this hsould mean
> that Python *must not* support them. What I'd like to keep
> is the possibility to still write such an extension module.
> 
> Enabling this for educational purposes is a great value
> that comes at a cheap price and no impact for the core.

I doubt it.  I'm not going to allow any compromises just to make it
easier to reuse continuations.  (Such as using a mutable counter in
the for-loop code.)

> > Now let's look at Jython.  In Jython, we can simulate "paused frames"
> > well enough by using real Java threads.  However full continuations
> > would require modifications to the JVM -- which is unacceptable to a
> > language boasting "100% Pure Java".  Another reason against allowing
> > continuations.
> 
> Not quite true, after I heard of a paper that shows how
> to implement continuations in Java, using threads.
> But I'll come back to that when I have the paper.

I've heard that his margin was too small to contain the proof.  I
expect that it will be a disappointment from a practical point of
view: perhaps he emulates the JVM in Java.

> > So, all in all, I don't think of continuations as "the last 10% that
> > we might as well add to finish the job."  I see it as an ill-specified
> > hypergeneralization.
> 
> Can we agree to not support them without forbidding them?

I won't forbid them, but I won't make compromises to the core PVM that
would make them easier to implement.  Your patch set would still be a
heck of a lot smaller of course.

> ...
> > A strawman proposal:
> 
> Ok, this looks all very well to me. More on that later.
> One question: Why do you want an explicit u.yield() ?
> Uthreads are scheduled automatically now, like real
> threads. Do you see a major drawback in supporting
> this, maybe as an option? Or do you see automatic
> scheduling as an extra construct on top with a special
> "scheduler" uthread?

See my response near the top to Eric about this.  I was thinking of a
lower-level concept, like coroutines.  I might consider automatic
scheduling of uthreads too.  But I've noticed that there are some ugly
hacks in that code, too. :-)

I've lived for years with a system (early Amoeba) that had threads
with only explicit scheduling: other threads would only run when you
were blocked for I/O or waiting for a semaphore.  It made for very
easy coding in some cases, since you didn't need to protect critical
sections with mutexes.  Unles, that is, you invoke stuff that might do
I/O (maybe for debugging :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Wed Nov  8 03:12:09 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 15:12:09 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>
Message-ID: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz>

Guido:

> I hope that it will be possible to make it a lot simpler than current
> stackless though, by not doing continuations.

My feeling is that this won't be the case. The fundamental
change of structure needed to make it stackless will be
much the same, as will the thought processes necessary
to understand how it works.

> Where it gets tricky is when this spawns a new uthread, which also
> calls C which calls Python... The solution is simply that the 
> application "shouldn't do that."

I worry that this is going to be a rather severe restriction.
For instance, it may make it impossible to spawn a uthread
from within a callback from a GUI framework. Since with many
GUI frameworks the entire application gets executed in
callbacks, you wouldn't be able to use uthreads at all with
such a framework.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Wed Nov  8 03:58:20 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 21:58:20 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 15:12:09 +1300."
             <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> 
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>

> Guido:
> 
> > I hope that it will be possible to make it a lot simpler than current
> > stackless though, by not doing continuations.

[Greg Ewing]
> My feeling is that this won't be the case. The fundamental
> change of structure needed to make it stackless will be
> much the same, as will the thought processes necessary
> to understand how it works.

I hope you are wrong but you may be right.  I'll have to have a good
look -- or someone else (not Christian!  With all due respect his code
is unreadable :-).

> > Where it gets tricky is when this spawns a new uthread, which also
> > calls C which calls Python... The solution is simply that the 
> > application "shouldn't do that."
> 
> I worry that this is going to be a rather severe restriction.
> For instance, it may make it impossible to spawn a uthread
> from within a callback from a GUI framework. Since with many
> GUI frameworks the entire application gets executed in
> callbacks, you wouldn't be able to use uthreads at all with
> such a framework.

But that's the same problem that current stackless has.  I take it
that you don't see the point of stackless then?  That's fine.  Maybe
this is not an application that could use uthreads.  They seem more
something for servers anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Wed Nov  8 04:18:10 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 16:18:10 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <200011080318.QAA00324@s454.cosc.canterbury.ac.nz>

Guido:

> I take it that you don't see the point of stackless then?

I have mixed feelings about it. I really like the idea of
uthreads, but I get the impression that Stackless as it
stands is only a partial implementation of the idea, with
no easy way in sight to evolve it into a full implementation.

> Maybe this is not an application that could use uthreads.

The point is that the user can be mixing Python and C stack
frames without even realising it. I was just giving one example
of how that can come about.

Saying "don't do that" isn't very helpful when "that" is
something which poeple often do quite unconsciously.

So, while I'd love to see uthreads as a core Python feature
one day, I think I'm in agreement with you that Stackless
isn't yet ready to be made into the standard Python
implementation.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From pf at artcom-gmbh.de  Wed Nov  8 08:33:00 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 08:33:00 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com> from "Fred L. Drake, Jr." at "Nov 7, 2000  6: 6:25 pm"
Message-ID: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>

> Charles G Waldman writes:
>  > I think that making "div" an infix operator would be setting a
>  > horrible precedent.  Currently, all infix operators "look like"
>  > operators, i.e. they are non-alphabetic characters, and things that
>  > look like words are either functions or reserved words.
 
Fred L. Drake, Jr.:
>   Like "is", "in", "is not", and "not in"?

And not to forget "and", "or" which were also infix operators from
the very beginning.  So "div" is no precedent at all.

IMHO the term "horrible" applies to operator symbols composed out
of non-alphabetic characters, where the meaning of these operators 
is hard to guess.  

counter-example: Using "><" as a vector cross product operator might 
still make some sense.  

But what would be the meaning of all those other arbitrary combinations
like ".+", ".%", ".*", "//", "@.", "~*" or what else has been proposed
to extend Python in the numeric area?  As long as the meaning of such
an operator isn't obvious from basic math knowledge, I clearly prefer
keyword operators.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Wed Nov  8 11:13:51 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 11:13:51 +0100
Subject: [Python-Dev] Starship down again ?!
References: <200011080620.WAA20659@slayer.i.sourceforge.net>
Message-ID: <3A09275F.CE4B01FA@lemburg.com>

> 
> +     Marc-Andr? Lemburg's mx.Proxy package.  These Web pages appear to
> +     be unavailable at the moment.
> + 
> +         http://starship.python.net/crew/lemburg/
> + 

Looks like Starship is down again. Is this due to the move from
BeOpen to DC or has someone pulled the plug on that ADSL line ? ...

 9  sl-bb1-rly-0-0-0.sprintlink.net (144.232.14.6)  120 ms  117 ms  113 ms
10  beth1sr2-2-0-0.md.us.prserv.net (165.87.97.226)  114 ms  116 ms  114 ms
11  beth1br2-ge-6-0-0-0.md.us.prserv.net (165.87.29.182)  122 ms  121 ms  116 ms
12  sfra1br1-so-2-1-0-0.ca.us.prserv.net (165.87.233.42)  193 ms  192 ms  191 ms
13  sfra1sr3-ge-2-0-0-0.ca.us.prserv.net (165.87.33.121)  191 ms  189 ms  190 ms
14  165.87.161.13 (165.87.161.13)  191 ms  191 ms  189 ms
15  core4-g2-0.snfc21.pbi.net (209.232.130.77)  197 ms  192 ms  190 ms
16  rback26-fe2-0.snfc21.pbi.net (216.102.187.153)  212 ms  197 ms  197 ms
17  adsl-63-202-160-65.dsl.snfc21.pacbell.net (63.202.160.65)  206 ms  212 ms  204 ms
18  * * *
19  * * *

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Wed Nov  8 12:23:46 2000
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 13:23:46 +0200
Subject: [Python-Dev] uthread strawman
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <3A0937C2.903D42@tismer.com>


Guido van Rossum wrote:
> 
> > Guido:
> >
> > > I hope that it will be possible to make it a lot simpler than current
> > > stackless though, by not doing continuations.
> 
> [Greg Ewing]
> > My feeling is that this won't be the case. The fundamental
> > change of structure needed to make it stackless will be
> > much the same, as will the thought processes necessary
> > to understand how it works.
> 
> I hope you are wrong but you may be right.  I'll have to have a good
> look -- or someone else (not Christian!  With all due respect his code
> is unreadable :-).

Are you talking of my changes to ceval.c or the continuationmodule?
I think it can't be the latter, since that does not matter at all
if we talk about Stackless. Stackless was written to make continuations
possible. It does not implement them.

My changes to ceval.c are written in the same style as your
original code, and it uses the same level of commenting as yours:
Nothing at all. :-)
With all due respect, I consider both versions equally unreadable,
unless one understands what the intent of the code is.

Until last October, I tried to keep everything as readable
and understandable as possible. Then it became clear that
this implementation would never make it into the core.
Then I gave up my efforts, and I also added a lot of
optimizations to the interpreter, by some systematic use
of macroes. Shurely his doesn't increase readability.

Forgetting about these optimizations, the code doesn't do
much more than the following:

eval_code2_setup
  is split off of the original eval_code2 function.
  It prepares a new frame for execution and puts it
  on top of the frame stack.

PyEval_Frame_Dispatch
  is a new function. It controls the execution of frames.
  Every initial or recursive interpreter call starts
  such a dispatcher. The nested dispatchers control the
  remaining "stackful" execution of Python.
  In its central loop, it runs the topmost frame of the
  frame stack, receives its return value and runs the
  next frame, until it sees the frame appear that
  spawned this dispatcher. Then it returns.

eval_code2_loop
  is the "real" part of the original eval_code2 function.
  It is not much different from the original. Major changes
  have been done to the entry code, the periodic checks
  in the loop, and the handling of function calls.

  The "big switch" has been simplified in the sense, that
  errors are no longer treated with various variables
  which have to be checked outside the switch.
  Instead, errors are directly mapped on a pseudo-opcode
  that allows to handle exceptions as just another case
  of the big switch.

  Every function call has got additional code that checks
  for the so-called unwind token, which tells us to leave
  this frame and to return to the scheduler.
  
  On entry to the frame, on every trip through the main loop,
  and after every function call, a callback f_callguard is
  checked for existence. If it exists, it is called, and
  if it returns -42, again the frame is left and we return
  to the scheduler.

  Entry into a frame has become a bit difficult, since we
  no longer know in advance whether a frame is expected
  to return a value or not. Due to uthread scheduling,
  switches occour between opcodes, and no values are
  transferred. When switching in the context of a function
  call, there *are* return values expected.
  This is all handled via some flags, in the frame entry
  code, line 948ff.

  Then, there are some very simple changes to the loop
  construct.

  Generally, more state variables are in the frames and
  kept up-to-date, like the instruction pointer.

  I'm omitting the extra code for uthread support here.

  Some functions were pulled out of the main loop, in order
  to make it smaller and easier to read. I would undo this
  today, since it makes comparison to the old version
  quite impossible, and it didn't yield more speed.

This is about all of it.
As you can see, there is no explicit support for co-anything
in the code. There are some generalisations to frame calling
and some callback hooks which actually do all co operations.

An implementation targeted for core integration would look
quite much different. It would provide more functionality
directly, without using callbacks.

A pure coroutine based implementation as you proposed
would not need the generalization of the frame parameter
passing, since switches can only occour in the context
of a function call.

Supporting auto-scheduled uthreads needs to distinguish
explicit and implicit switching, since implicit switching
occours between opcodes, not *in* opcodes.

The techniques for this can be written in quite a different
manner than I did.

Again, this code is not intended for inclusion in the core,
and not for drawing conclusions for the feasibility of
Stackless at all. The latter has been shown by applications
like the uthreads, and by its central use in the EVE game.
We need to judge the priciple, not the implementaiton.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From phil at river-bank.demon.co.uk  Wed Nov  8 13:26:54 2000
From: phil at river-bank.demon.co.uk (Phil Thompson)
Date: Wed, 08 Nov 2000 12:26:54 +0000
Subject: [Python-Dev] What to choose to replace Tkinter?
References: <049d01c0471f$d7899450$8119fea9@neil>
Message-ID: <3A09468E.9E767720@river-bank.demon.co.uk>

I've come to this discussion rather late...

Most modern GUI toolkits have (or will soon have) the widgets to compete
with Tk strengths. The difficult question with a Tkinter replacement is
the complete fragmentation of the GUI toolkit "market". I don't believe
that you can, today, identify a toolkit that you are sure is going to
have widespread support and the longevity needed (in 5 years time you
don't want to be in the position you are in today with Tk).

I see two alternatives...

- make the Tkinter replacement an abstraction layer between Python and
the *user's* choice of toolkit. The developer gets a consistent API, and
toolkits can be adopted and dropped as fashions change. This is the
approach taken by VeePee (http://www.thekompany.com/projects/vp/).

- don't bundle Tkinter with Python. At least you then make people think
a bit more about what they want from a toolkit and make an appropriate
choice - let Tkinter's replacement be found by natural selection. At the
very least let's have a more up-front presentation of the different
options, strengths/weaknesses etc on the web site. Cameron must be
getting bored of pointing people to his toolkit summary.

For the record, Qt has a good Canvas widget, Unicode support, user
selectable Windows/Mac/Unix look & feel etc, etc.

Phil



From guido at python.org  Wed Nov  8 14:20:02 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 08:20:02 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 13:23:46 +0200."
             <3A0937C2.903D42@tismer.com> 
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>  
            <3A0937C2.903D42@tismer.com> 
Message-ID: <200011081320.IAA21990@cj20424-a.reston1.va.home.com>

> Again, this code is not intended for inclusion in the core,
> and not for drawing conclusions for the feasibility of
> Stackless at all. The latter has been shown by applications
> like the uthreads, and by its central use in the EVE game.
> We need to judge the priciple, not the implementaiton.

Of course.  Thanks by the way for the clear explanation of what needs
to be done!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gward at mems-exchange.org  Wed Nov  8 15:02:49 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Wed, 8 Nov 2000 09:02:49 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001107182621.I27208@xs4all.nl>; from thomas@xs4all.net on Tue, Nov 07, 2000 at 06:26:21PM +0100
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>
Message-ID: <20001108090249.A28202@ludwig.cnri.reston.va.us>

On 07 November 2000, Thomas Wouters said:
> Sorry for the late remark (I did see your earlier message) but after reading
> the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
> for non-essential stuff: warnings, debug-info and optimizations. Removing
> things from OPT shouldn't break anything, and neither should adding options
> that fit in the categories above. (Barring broken compilers, of course.)
> 
> Instead, the -fPIC option should be added to CFLAGS, I think. The Python
> autoconf setup is slightly less versatile than most, though, since it's
> doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
> without editing configure(.in) :P If noone else does it before me, I'll see
> about fixing at least the -fPIC thing later, when I find some time ;P

Good point -- fixing CFLAGS instead of OPT sounds right to me.  I'm not
really sure on where to do this, though.

Ooh, I just noticed this in configure.in:

  # DG/UX requires some fancy ld contortions to produce a .so from an .a
  case $MACHDEP in
  dguxR4)
        LDLIBRARY='libpython$(VERSION).so'
        OPT="$OPT -pic"
        ;;

No prize for guessing that "-pic" on the DG/UX compiler has a similar
effect to GCC's -fPIC, and based on the comment this is required.  I'm
guessing this should be in CFLAGS as well.

Oh wait: CFLAGS is not exported from configure.in -- it's *only* defined
in the Makefile.  From Makefile.in:

  CFLAGS=		$(OPT) -I. $(DEFS)

IOW, it looks like OPT is used for all non-preprocessor compiler flags,
whether they're "frills" like optimization/debugging or not.

Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
right thing, but for the wrong reason.  Fixing it would require a little
more involved surgery on configure.in and the Makefiles.  And it would
also require reexamining every use of OPT in configure.in (not too hard,
"grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
so forth at config time or at make time.

I'm willing to spend some time on this; does anyone think it's a
pointless waste of time?

        Greg



From fdrake at acm.org  Wed Nov  8 15:04:32 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:04:32 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001108090249.A28202@ludwig.cnri.reston.va.us>
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
	<20001107182621.I27208@xs4all.nl>
	<20001108090249.A28202@ludwig.cnri.reston.va.us>
Message-ID: <14857.23920.646760.779849@cj42289-a.reston1.va.home.com>

Greg Ward writes:
 > I'm willing to spend some time on this; does anyone think it's a
 > pointless waste of time?

  I'm certainly willing to allocate some of your cycles to this.  ;)
I'll even help test it once you've checked it in.
  Seriously, I think you're right -- there needs to be a separation of
what's needed and optional stuff added from the make command line.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Wed Nov  8 15:21:20 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 09:21:20 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: Your message of "Wed, 08 Nov 2000 09:02:49 EST."
             <20001108090249.A28202@ludwig.cnri.reston.va.us> 
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>  
            <20001108090249.A28202@ludwig.cnri.reston.va.us> 
Message-ID: <200011081421.JAA22160@cj20424-a.reston1.va.home.com>

> Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
> right thing, but for the wrong reason.  Fixing it would require a little
> more involved surgery on configure.in and the Makefiles.  And it would
> also require reexamining every use of OPT in configure.in (not too hard,
> "grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
> if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
> so forth at config time or at make time.
> 
> I'm willing to spend some time on this; does anyone think it's a
> pointless waste of time?

No, please fix it right!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Wed Nov  8 15:23:47 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Wed, 8 Nov 2000 08:23:47 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
	<m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14857.25075.482728.276846@buffalo.fnal.gov>

 > > Charles G Waldman writes:
 > >  > I think that making "div" an infix operator would be setting a
 > >  > horrible precedent.  Currently, all infix operators "look like"
 > >  > operators, i.e. they are non-alphabetic characters, and things that
 > >  > look like words are either functions or reserved words.
 >  
 > Fred L. Drake, Jr.:
 > >   Like "is", "in", "is not", and "not in"?
 > 
 > Peter Funk writes:
 > And not to forget "and", "or" which were also infix operators from
 > the very beginning.  So "div" is no precedent at all.

OK, I stand corrected and feel suitably foolish.  However I still
think it's quite inconsistent to have divmod(a,b) but a div b.




From fdrake at acm.org  Wed Nov  8 15:38:51 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:38:51 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14857.25075.482728.276846@buffalo.fnal.gov>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
	<m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
	<14857.25075.482728.276846@buffalo.fnal.gov>
Message-ID: <14857.25979.244131.879387@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > OK, I stand corrected and feel suitably foolish.  However I still
 > think it's quite inconsistent to have divmod(a,b) but a div b.

  I suspect div would be much more widely used than divmod(), which is
essentially a performance optimization when you need both results.
One reason *not* to make divmod() an operator, aside from issues of
legacy code, is that it really returns two results (never mind that it
returns exactly one tuple); I can't think of another operator that
conceptually returns two values.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From pf at artcom-gmbh.de  Wed Nov  8 16:32:22 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 16:32:22 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: <200011081517.HAA16919@slayer.i.sourceforge.net> from Guido van Rossum at "Nov 8, 2000  7:17:51 am"
Message-ID: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

this rather old module still contains string exceptions.
Since string exceptions are depreceated in favour of class based
exceptions wouldn't it be better to tweak those few lines into
class based exceptions now?

> Add 1994 Coroutine module by Tim Peters
[...]
> Killed = 'Coroutine.Killed'
> EarlyExit = 'Coroutine.EarlyExit'

Regards, Peter



From guido at python.org  Wed Nov  8 16:42:41 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 10:42:41 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: Your message of "Wed, 08 Nov 2000 16:32:22 +0100."
             <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011081542.KAA22738@cj20424-a.reston1.va.home.com>

> this rather old module still contains string exceptions.
> Since string exceptions are depreceated in favour of class based
> exceptions wouldn't it be better to tweak those few lines into
> class based exceptions now?
> 
> > Add 1994 Coroutine module by Tim Peters
> [...]
> > Killed = 'Coroutine.Killed'
> > EarlyExit = 'Coroutine.EarlyExit'

No.  This code is of historic interest only.  Don't touch it please!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From pf at artcom-gmbh.de  Wed Nov  8 17:06:56 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 17:06:56 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: <200011081542.KAA22738@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 8, 2000 10:42:41 am"
Message-ID: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de>

I was nitpicking:
> > this rather old module still contains string exceptions.
> > Since string exceptions are depreceated in favour of class based
> > exceptions wouldn't it be better to tweak those few lines into
> > class based exceptions now?
> > 
> > > Add 1994 Coroutine module by Tim Peters
> > [...]
> > > Killed = 'Coroutine.Killed'
> > > EarlyExit = 'Coroutine.EarlyExit'

Guido van Rossum answered:
> No.  This code is of historic interest only.  Don't touch it please!

Hmmmm.... I always thought of the Demo directory as a repository for
example code, which may be used to teach Python programming to
beginners.  I know, that some pieces are a way out of date.  

But I think it would be a worthwile goal to update at least some of
those pieces step by step to reflect current Python coding habits.
The README in Demo says:

"""This directory contains various demonstrations of what you can do with
Python. [...]"""

If you want to turn the Demo directory into a museum of code snippets of
historic interest, at least the README should say so. ;-)

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From gmcm at hypernet.com  Wed Nov  8 18:09:48 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Wed, 8 Nov 2000 12:09:48 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com> 
Message-ID: <3A09428C.12529.1044B070@localhost>

[Guido]
> Without continuations, but with microthreads (uthreads) or
> coroutines, each (Python) stack frame can simply be "paused" at a
> specific point and continued later.  The semantics here are
> completely clear (except perhaps for end cases such as unhandled
> exceptions and intervening C stack frames).

Exceptions require some thought, particularly because the "for" 
protocol uses an IndexError as a signal. In my own stuff I've found I 
need to catch all exceptions in the coroutine, primarily because I've 
always got resources to clean up, but clearly the implementation has 
to do the right thing when an exception crosses the boundary.
 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest
> level. Uthread objects represent microthreads.  An uthread has a
> chain of stack frames linked by back pointers just like a regular
> thread. Pause/resume operations are methods on uthread objects. 
> Pause/resume operations do not address specific frames but
> specific uthreads; within an uthread the normal call/return
> mechanisms can be used, and only the top frame in the uthread's
> stack of call frames can be paused/resumed (the ones below it are
> paused implicitly by the call to the next frame, and resumed when
> that call returns).

I'm not convinced (though I'm not asking you to convince me - I need to 
ponder some more) that this is the right approach. My worry is that to 
do coroutines, we end up with a bunch of machinery on top of 
uthreads, just like Tim's old coroutine stuff, or implementations of 
coroutines in Java.

My mental test case is using coroutines to solve the impedance 
mismatch problem. SelectDispatcher is a simple example (write 
"client" code that looks like it's using blocking sockets, but multiplex 
them behind the curtain). Using a "pull" parser as a "push" parser is 
another case, (that is, letting it think it's doing its own reads).

But what about using a "pull" lexer and a "pull" parser, but tricking 
them with coroutines so you can "push" text into them? Tim's 
implementation of the  Dahl & Hoare example (which I 
rewrote in mcmillan-inc.com/tutorial4.html) shows 
you *can* do this kind of thing on top of a thread 
primitive, but might it not be much better done on 
a different primitive?

Again, I'm not really asking for an answer, but I think this type of problem is not uncommon, and a wonderful use of coroutines; so I'm wondering if this is a good trade-off.

> - u.yield() pauses the current uthread and resume the uthread
> u where it was paused.  The current uthread is resumed when
> some other uthread calls its yield() method.  Calling
> uthread.current().yield() is a no-op. 

This doesn't seem like enough: sort of as though you designed a 
language in which "call" and "return" were spelled the same way. 
Certainly for coroutines and generators, people gravitate towards 
paired operations (eg. "suspend" and "resume"). Again, Tim's 
demonstrated you can do that on top of threads, but it sure seems to 
me like they should be primitives.

> I think this API should enough to implement Gordon's
> SelectDispatcher code.  In general, it's easy to create a
> scheduler uthread that schedules other uthreads.

Thank you :-).

> Open issues:
> 
> - I'm not sure that I got the start conditions right.  Should
> func() be
>   be allowed to run until its first yield() when
>   uthread.new(func) is called?

For coroutine stuff, that doesn't bother me. For uthreads, I'd think (like 
real threads) that creation and starting are different things.
 
> - I'm not sure that the rules for returning and raising
> exceptions
>   from func() are the right ones.
> 
> - Should it be possible to pass a value to another uthread by
> passing
>   an argument to u.yield(), which then gets returned by the
>   resumed yield() call in that uthread?

Argument (in and / or out) passing is a necessity for generators and 
coroutines. But, as mentioned above, I don't think a symmetrical 
"yield" is the right answer.
 
> - How do uthreads interact with real threads?  Uthreads are
> explicitly
>   scheduled through yield() calls; real threads use preemptive
>   scheduling.  I suppose we could create a new "main" uthread for
>   each real thread.  But what if we yield() to an uthread that's
>   already executing in another thread?  How is that error
>   detected?

I think I would (perhaps naively) expect that I could create a uthread in 
one (real) thread, and then pass it off to another (real) thread to 
execute.

Another post brings up GUIs and uthreads. I already expect that I'm 
going to have to dedicate a (real) thread to the GUI and ponder very 
carefully how that thread interacts with others. Of course, that's from 
lessons learned the hard way; but personally I'm not expecting 
uthreads / coroutines to make that any easier.

[About continuations: while I love the fact that Christian has made 
these available for playing, I have so far not found them productive. I 
wrote a simple minded backtracking parser using them, but found it no 
better than a coroutine based one. But I am interested in how a *real* 
pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good* 
idea, boss", please.]

- Gordon



From akuchlin at mems-exchange.org  Wed Nov  8 19:11:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 08 Nov 2000 13:11:26 -0500
Subject: [Python-Dev] Catalog-SIG created
Message-ID: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>

A mailing list for the Catalog SIG has been created, to discuss the
design and construction of a Vaults/CPAN/LSM-like index for Python.
Web pages for the SIG don't exist yet, but will be created soon.

The SIG's charter:

The Python Catalog SIG aims at producing a master index of Python
software and other resources.  It will begin by figuring out what the
requirements are, converging on a design for the data schema, and
producing an implementation.  ("Implementation" will almost certainly
include mean a set of CGI scripts for browsing the catalog, and may
also contain a standard library module for automatically fetching &
installing modules, if the SIG decides that's a worthwhile feature.)

--amk



From guido at python.org  Wed Nov  8 19:21:39 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 13:21:39 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: Your message of "Wed, 08 Nov 2000 17:06:56 +0100."
             <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011081821.NAA23572@cj20424-a.reston1.va.home.com>

> Hmmmm.... I always thought of the Demo directory as a repository for
> example code, which may be used to teach Python programming to
> beginners.  I know, that some pieces are a way out of date.  
> 
> But I think it would be a worthwile goal to update at least some of
> those pieces step by step to reflect current Python coding habits.
> The README in Demo says:
> 
> """This directory contains various demonstrations of what you can do with
> Python. [...]"""
> 
> If you want to turn the Demo directory into a museum of code snippets of
> historic interest, at least the README should say so. ;-)

If you want to make a proposal for reorganizing the Demo directory,
please do so.  There are more important problems with the Demo
directlry than the fact that some code still uses string exceptions.
Please don't start fixing the small nits without seeing the big
picture.

(That's all I have time for now.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tismer at tismer.com  Wed Nov  8 18:20:43 2000
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 19:20:43 +0200
Subject: [Python-Dev] uthread strawman
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com> <3A09428C.12529.1044B070@localhost>
Message-ID: <3A098B6B.522EA5E8@tismer.com>


Gordon McMillan wrote:

[snipped all the good stuff away for bigger brains than mine ]

> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

Yes, I saw Gordon making heavy use of naked continuations,
but actually they were not really what he needed. I believe
this since he made much use of co.update(), which moves
a continaution to the most current state of the frame.

In fact, what Gordon would need (and most probably most of
us as well) is just the handle to a frame, and the ability
to switch to it. In Gordon's case, these would probably
be "continuation" which are not frozen, but simply track
the frame as it is. I'm not absolutely shure, but quite.

I'm happy to toss continuations for core Python, if we
can find the right building blocks for coro/gen/uthreads.
I think Guido comes quite near this, already.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From fdrake at acm.org  Wed Nov  8 20:50:50 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 14:50:50 -0500 (EST)
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>
	<Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
Message-ID: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>

Mark W. Alexander writes:
 > Is there another way to subscribe, or did I just jump the gun?

  You can get to the Mailman interface at:

	http://www.python.org/mailman/listinfo/catelog-sig/

  The Web pages aren't actually there yet; Andrew will get to this
when he can, I'm sure.  ;)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Wed Nov  8 21:34:49 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 21:34:49 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <20001107225758.B77651301D9@oratrix.oratrix.nl>
Message-ID: <3A09B8E9.E247125F@lemburg.com>

Jack Jansen wrote:
> 
> > > In other words, what does this new type_id thing
> > > actually *mean*?
> >
> > For the interpreter it means that it can assume the type
> > interface to be binary compatible to the "original"
> > type, e.g. by setting the flag to say PyDict_TypeID
> > the type assures that all PyDict_*() APIs will work
> > on the type -- basically the same thing as PyDict_Check()
> > does now except that the type object needn't be the same
> > anymore.
> 
> I would be _very_ happy if this single type_id could somehow be
> replaced by an array, or a bitset.

I guess a bit array would be a possibility...

#define PyList_Check(obj) ((obj)->ob_type->\
                            capabilities[Py_ListType->cap_slot])
 
cap_slot could be set at type object creation time using
some Python slot id generator (a function which outputs integers
up to the maximum length of the capabilities array and raises
a Py_FatalError() if this limited is excceded).

> I have a lot of types in MacPython that are acceptable to the APIs of
> other types, a sort of poor-mans-inheritance scheme. For instance, all
> operating system calls that accept a MacOS WindowPtr will also happily
> accept a DialogPtr. Major magic is needed to get this to work
> reasonably in Python, and the Python user can still accidentally mess
> up the refcounting scheme and free things s/he isn't aware of.
> 
> As the number of types in a given run of the interpreter appears to be
> limited (am I right here?) and type-identity-tests are valid within a
> single interpreter run only (am I right here?) 

Right * 2

> an API like
>   typeindex = Py_TypeToTypeIndex(typeobject);
> which would use a dictionary as storage for the mapping and generate
> the index numbers on the fly would do the trick. Call it once during
> module initalization and the
>   Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
> macro would be a oneliner to test the bit in the set.
> 
> A few hundred bits in the set would get us a long way, I guess.

One thing I'm unsure about is whether changing cap_slot ids
between runs of the interpreter are a good idea. Also, I think that
the basic types should be given constant cat_slot ids to enhance
performance (the id generator could be made to start at say 10 
and the basic types be fixed in the range 0-9).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From trentm at ActiveState.com  Wed Nov  8 22:07:39 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 8 Nov 2000 13:07:39 -0800
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 08, 2000 at 02:50:50PM -0500
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us> <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net> <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>
Message-ID: <20001108130739.H27185@ActiveState.com>

On Wed, Nov 08, 2000 at 02:50:50PM -0500, Fred L. Drake, Jr. wrote:
> 	http://www.python.org/mailman/listinfo/catelog-sig/ 

.replace("catelog", "catalog")

:)


-- 
Trent Mick
TrentM at ActiveState.com



From jeremy at alum.mit.edu  Thu Nov  9 01:14:45 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:14:45 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
Message-ID: <14857.60533.766291.786182@bitdiddle.concentric.net>

I just uploaded a patch to sourceforge that revises the CALL_FUNCTION
implementation to use a bunch of functions and avoid the long block of
inline code in eval_code2.

The overall performance of the patch is about the same.  The current
patch causes a big slowdown for the use of keyword arguments, but is
otherwise as fast or faster then the old version.  The keyword
slowdown should be avoidable without too much effort.

I wrote a short benchmark that demonstrates the effect on many
variations of function calls.  The output lists the test time, and the
median, min, and max time of 10 executions.  (The benchmark script is
attached to this message.)

The vanilla CVS tree produces these results:

time_arg1 0.09 0.09 0.11
time_arg2 0.1 0.09 0.12
time_arg3 0.11 0.1 0.12
time_fact 0.12 0.1 0.14
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.1 0.08 0.11
time_callable 0.37 0.33 0.38
time_keyword 0.14 0.13 0.18
time_star 0.12 0.12 0.14
time_kw 0.25 0.24 0.27
time_kw2 0.69 0.66 0.73
time_starkw 0.24 0.23 0.26
time_init 0.64 0.63 0.68
total 3.18

The CVS tree with the CALL_FUNCTION patch applied produces these
results: 
time_arg1 0.09 0.09 0.1
time_arg2 0.1 0.09 0.1
time_arg3 0.11 0.09 0.13
time_fact 0.11 0.11 0.14
time_meth 0.09 0.09 0.1
time_umeth 0.1 0.1 0.11
time_builtin 0.08 0.07 0.09
time_callable 0.35 0.34 0.38
time_keyword 0.42 0.4 0.44      (*** big slowdown ***)
time_star 0.13 0.13 0.15
time_kw 0.25 0.23 0.29
time_kw2 0.66 0.61 0.79
time_starkw 0.24 0.22 0.27
time_init 0.66 0.6 0.72
total 3.39

Jeremy

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: callbench.py
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001108/b099a7be/attachment.asc>

From jeremy at alum.mit.edu  Thu Nov  9 01:25:11 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:25:11 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <14857.60533.766291.786182@bitdiddle.concentric.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
	<14857.60533.766291.786182@bitdiddle.concentric.net>
Message-ID: <14857.61159.339908.704603@bitdiddle.concentric.net>

Looks like I jumped the gun with my last message.  A trivial change to
the logic prevented the keyword arguments slowdown.  I've revised the
SF patch.  The new numbers show that the patch is just a tad faster on
the benchmark.  (And the difference between gcc -O2 and -O3 makes a
big difference for this ceval-intensive benchmark.)

Jeremy

time_arg1 0.1 0.09 0.12
time_arg2 0.1 0.1 0.12
time_arg3 0.1 0.1 0.12
time_fact 0.12 0.11 0.13
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.09 0.08 0.1
time_callable 0.36 0.34 0.41
time_keyword 0.13 0.13 0.14
time_star 0.12 0.12 0.13
time_kw 0.23 0.22 0.24
time_kw2 0.64 0.61 0.66
time_starkw 0.24 0.23 0.26
time_init 0.64 0.64 0.69
total 3.08



From tim.one at home.com  Thu Nov  9 09:44:07 2000
From: tim.one at home.com (Tim Peters)
Date: Thu, 9 Nov 2000 03:44:07 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>

[Guido]

[examples of presumably fuzzy semantics in the presence of continuations]

> ...
> But now consider this:  the "evaluation stack" contained in each
> frame could easily be replaced by a bunch of temporary variables,
> if we had a slightly different instruction set ...
> Then would we save those temporary variables or not?

Yes, provided they remained anonymous (to the user) they're simply internal
implementation details, and continuations preserve all things "like that".
It's only visible name bindings that continuations let change across
resumptions.

> it can make a difference!

Indeed yes.

> ...
> But if the compiler were to use temporary variables instead of the
> evaluation stack, they might not have been restored!

The technical term for that would be "bug" <0.5 wink>.

Christian already covered the for-loop example.  A more interesting
variation is a for loop without a named "indexing vrbl":

    for x in sequence:
        etc

Scheme has nothing direct to say about this because Scheme has no loops.
Writing it as a recursive function instead leads to the same kind of result,
though.

> ...
> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.

It's clearer in Scheme because Scheme has fewer "primitive concepts" than
Python.

> (I've been told that the exact semantics of continuations in Scheme
> differ greatly between Scheme implementations.)

I believe you've been told that, but not by a clueful Scheme user!
Continuations are rigorously well-defined in Scheme.  What *isn't*
well-defined in Scheme is order of evaluation in many cases, so an
expression like

   (+ (f) (g))

can display wildly different behavior across implementations if one or both
of the functions {f, g}, directly or indirectly, creates or invokes a
continuation (say that g does:  it's not defined whether f has been invoked
by the time g is -- indeed, it's even OK to invoke f and g in parallel).

Note:  the Python eye sees something like that and leaps to the rule "OK,
chowderhead, so don't muck with continuations in the middle of
expressions!".  What that misses is that *everything* in Scheme is "an
expression".  Scheme implementations do "differ greatly" in *this* respect.
BTW, note that Scheme implementations can also display wildly different
behavior if f and g merely have side effects (i.e., this really has nothing
to do with continuations specifically:  they're just another form of
side-effect you can't always predict without knowing the order of evaluation
first).

[skipping the proposal because we talked about it instead]

[Gordon McMillan]
> ...
> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

I don't know of any comprehensible application of continuations that can't
be done without them.  The real appeal of continuations is in their
theoretical elegance (they're a single mechanism that can be used to build
all sorts of stuff, much as all of "if" and "while" and "for" can be built
out of compares and gotos -- continuations can be used to implement all of
calls, non-resumable and resumable exceptions, generators, coroutines,
non-deterministic evaluation, "thread like" stuff, ...).  In any specific
case, though, you want to live at the higher level, not at the raw
continuation level.

WRT a backtracking parser, note that this is what Icon *lives for*, and
generators alone suffice (and are indeed very pleasant) for natural
expression of that task.  Icon became what it is when Griswold decided to
open up the backtracking pattern-matching engine underlying SNOBOL4, and
make it the basis for all expression evaluation.  It took him 3 full
languages (SL5 and Rebus came before Icon) and 20 years to get this right.

Building a backtracking parser directly out of continuations sounds to me
mostly painful.  Building generators out of continuations *is* painful (I've
done it).  Curiously, the symmetry of coroutines appears to make building
them out of continuations easier (than building generators).

I'm not in love w/ continuations:  I *could* be if Guido got Continuation
Religion and wanted to redo exceptions and calls on top of continuations
too, but given that whatever happens here is destined (Christian will say
"doomed" <wink>) to co-exist with everything that's already here, the appeal
of continuations is minimal.  I've got no desire to play with novel new
control structures in Python (note that I don't consider generators-- or
even coroutines --to be "novel", not after they've been in multiple
languages for more than 30 years), and Python doesn't have the syntactic
flexibility that makes such experiments *pleasant* in Scheme anyway.  So
it's enough for me if Python supports the handful of new control-flow
gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
if we don't need continuations for those, fine by me.  BTW, I'd also like to
pickle a pure-Python computation in mid-stream, save it to disk, and resume
it later after a reboot (or on another machine!); we don't need
continuations for that either (although we do need much of what Stackless
does).

switching-from-redefining-truth-to-redefining-falsehood-ly y'rs  - tim




From tismer at tismer.com  Thu Nov  9 12:39:48 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 13:39:48 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>
Message-ID: <3A0A8D04.3881E4FC@tismer.com>


Tim Peters wrote:
...

> Building a backtracking parser directly out of continuations sounds to me
> mostly painful.  Building generators out of continuations *is* painful (I've
> done it).  Curiously, the symmetry of coroutines appears to make building
> them out of continuations easier (than building generators).

Some things work very well, built with plain continuations.
See the attached ICON-style generator/backtrack framework
(going to post the 8 queens puzzle, soon).

> I'm not in love w/ continuations:  I *could* be if Guido got Continuation
> Religion and wanted to redo exceptions and calls on top of continuations
> too, but given that whatever happens here is destined (Christian will say
> "doomed" <wink>) to co-exist with everything that's already here, the appeal
> of continuations is minimal.  I've got no desire to play with novel new
> control structures in Python (note that I don't consider generators-- or
> even coroutines --to be "novel", not after they've been in multiple
> languages for more than 30 years), and Python doesn't have the syntactic
> flexibility that makes such experiments *pleasant* in Scheme anyway.

That's a very good point. Tricking Python to make continuations
useful is a pain in the a** and has led me to a quite weird API.
After the "sane" constructs are defined well, there is no much
reason to support continuations in the first place.

> So it's enough for me if Python supports the handful of new control-flow
> gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
> if we don't need continuations for those, fine by me.  BTW, I'd also like to
> pickle a pure-Python computation in mid-stream, save it to disk, and resume
> it later after a reboot (or on another machine!); we don't need
> continuations for that either (although we do need much of what Stackless
> does).

There is one application of continuations which I still consider
worthy. I'm shure that many people find it incredibly ugly.
Using continuations, I can build method-like functions
without classes and instances, which perform incredibly
fast. This cannot be done with simple one-shot continuations;
of course a class method would do the same, but slower:

<script labguage="python">
function expensive_to_init_and_cheap_to_call(*args):
    pass # initialize many local variables
    param = continuation.return_current() # ***
    return do_cheap_calculation(param)

# generate and initialize such a function    

fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

# now we are at ***

fastfunc(42)  # can call it many times, at high speed.
</script>

This is a function with expensive initialization and many
local variables involved. After initializing, the
continuation of *** is returned as a callable object.
All initialization is done, all locals are set, and
now we can pull out many results by repeatedly calling
this continuation.
This cannot be modelled as efficiently today with classes.

ciao - chris

p.s.: Here the simple ICON-like generator/backtrack framework.



---------------------------------------------------------------------

import continuation

class control:
    """ ICON style generators """
    
    def __init__(self):
        # the chain of alternatives is a stack of tuples
        self.more = None

    def suspend(self, value):
        """ return a value, but keep the caller for re-use """
        # push the caller on the alternatives stack
        self.more = (continuation.caller(), self.more)
        # simulate a return of the caller with the current value
        continuation.caller(2)(value)

    def fail(self):
        """ restart an alternative, if there is some.
        Otherwise do nothing """
        if self.more:
            back, self.more = self.more
            back()

    def clear(self):
        """ clear alternatives stack """
        self.more = None

    def asn(self, old, val):
        """ an undoable assignment. Usage:
        var = c.asn(var, value)
        Like the ICON operator "<-"
        """
        self.suspend(val)
        print "asn returning"
        return(old)

    def choice(self, *args):
        """ iterator over a fixed sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        for val in args[:-1]:
            self.suspend(val)
        return args[-1]

    # the above works only for sequences of known size.
    # The version below is better since it does not need to
    # know the size, but it has to do a look-ahead.
    
    def choice(self, *args):
        """ iterator over a general sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        
        # problem: how do we *return* the last element for any sequence?
        # solution: do a look-ahead by 1
        first = 1
        for nextval in args:
            if first:
                val = nextval
                first = 0
                continue
            self.suspend(val)
            val = nextval
        return val

    def to_by(self, a, b=None, c=None):
        """ a loop construct, modelled after the ICON
        for .. to .. by expression, but using xrange style
        """
        if c is None:
            if b is None:
                iterator = xrange(a)
            else:
                iterator = xrange(a,b)
        else:
            iterator = xrange(a,b,c)
        # problem: how do we *return* the last element?
        # solution: splitting the xrange is handy!
        for i in iterator[:-1]:
            self.suspend(i)
        return iterator[-1]

    # trying to get *slightly* more structure, here an
    # attempt to introduce something like "every".

---------------------------------------------------------------------

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From tismer at tismer.com  Thu Nov  9 14:04:00 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:04:00 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com> <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <3A0AA0C0.EA9C137B@tismer.com>


Christian Tismer wrote:

I wrote:
> There is one application of continuations which I still consider
> worthy. I'm shure that many people find it incredibly ugly.
> Using continuations, I can build method-like functions
> without classes and instances, which perform incredibly
> fast. This cannot be done with simple one-shot continuations;
> of course a class method would do the same, but slower:
> 
> <script language="python">
> function expensive_to_init_and_cheap_to_call(*args):
>     pass # initialize many local variables
>     param = continuation.return_current() # ***
>     return do_cheap_calculation(param)
> 
> # generate and initialize such a function
> 
> fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

But this is again no argument for continuations, since
generators provide an equally performant solution:
(assuming a generator implementation, which passes the
generator object as first argument to itself. The
initialization part ends at self.start(), which
leaves the generator frame as is, and acts as an
entry point.)

function easily_done_with_generators(self, *args):
    pass # initialize many local variables
    param = self.start()
    while 1:
        result = do_cheap_calculation(param)
        param = self.suspend(result)

fastfunc = generator(easily_done_with_generators, manyargs)

fastfunc(42) # ...

----------------------

This implementation works without continuations and gives
equally fast generators.

Note the self.start() method: It is equivalent to the
continuation.return_current() function:
Leave the current frame as its own result, and provide
an entry point which accepts a value.

self.suspend() returns from the frame without ending it.
It returns a value, but preserves the entry point.

Does anybody know of a useful example where continuations
are really needed? The only remaining idea is at the moment
modelling of a case construct, but this reason is easily
removed by adding a switch statement to Python. :-)

cheers - chris   (stackless and continuationless)

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at math.huji.ac.il  Thu Nov  9 15:07:14 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:07:14 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA0C0.EA9C137B@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>

On Thu, 9 Nov 2000, Christian Tismer wrote:

> Does anybody know of a useful example where continuations
> are really needed?

Well, it may be a bit unpythonic (for several reasons), but restartable
exceptions AFAICS cannot be implemented by generators or coroutines.
Restartable exceptions are not always appropriate, but tend to be
a pain to simulate when they are needed.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From tismer at tismer.com  Thu Nov  9 14:26:17 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:26:17 +0200
Subject: [Python-Dev] uthread strawman
References: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>
Message-ID: <3A0AA5F9.67C782E2@tismer.com>


Moshe Zadka wrote:
> 
> On Thu, 9 Nov 2000, Christian Tismer wrote:
> 
> > Does anybody know of a useful example where continuations
> > are really needed?
> 
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

(sorry for posting to python-dev/null but Starship is still down)

How would restartable exceptions work? Like so?

try:     # here, a continuation is saved
    pass # some operations which may fail,
         # raising something that inherits from RestartableException

except RestartableException:
    pass # repair the condition
    sys.try_again()   # go back to the try statement

Probably not, since this would be doable by just an internal
jump operation.
But if the restartable exception were a callable object,
continuations might be necessary, since we now have a fork
of two concurrently existing execution paths in the frame:
We might continue with the exception handling but pass the
restartable to someone else, who tries to call it later.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at math.huji.ac.il  Thu Nov  9 15:38:30 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:38:30 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA5F9.67C782E2@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091627510.5020-100000@sundial>

[Christian Tismer]
> Does anybody know of a useful example where continuations
> are really needed?

[Moshe Zadka]
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

[Christian Tismer]
> How would restartable exceptions work? Like so?
> 
> try:     # here, a continuation is saved
>     pass # some operations which may fail,
>          # raising something that inherits from RestartableException
> 
> except RestartableException:
>     pass # repair the condition
>     sys.try_again()   # go back to the try statement

I imagined something like

try:
	raise RestartableException, continuation.current_continuation()
except RestartableException, ex:
	# correct the problem
	ex.saved_continuation()

IOW, just put the continuation in the exception object.
I don't think you can use generators/coroutines to simulate something
like this
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  9 16:30:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 10:30:03 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 16:38:30 +0200."
             <Pine.GSO.4.10.10011091627510.5020-100000@sundial> 
References: <Pine.GSO.4.10.10011091627510.5020-100000@sundial> 
Message-ID: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>

[Moshe]
> I imagined something like
> 
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
> 
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

You can make the exception handler into a separate coroutine -- then
no reusable continuations are needed.  The coroutine can repair the
exception and resume the other.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at pythonware.com  Thu Nov  9 17:02:31 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 9 Nov 2000 17:02:31 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
Message-ID: <00d801c04a66$7828bff0$0900a8c0@SPIFF>

> Remove AC_C_INLINE test from configure.in, since the only place the symbol
> occurs in the Python sources appears to be as text in comments.  We do not
> want to interfere with C++ keywords!

did you benchmark SRE before and after this change?

</F>




From fdrake at acm.org  Thu Nov  9 17:16:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Nov 2000 11:16:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: <00d801c04a66$7828bff0$0900a8c0@SPIFF>
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
	<00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <14858.52676.775791.32394@cj42289-a.reston1.va.home.com>

Fredrik Lundh writes:
 > did you benchmark SRE before and after this change?

  No, but I don't see how this change would affect SRE.  _sre looks
like it takes care of itself with regard to inlining; the only thing
you might lose is:

	#undef inline

  Am I misreading something?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fredrik at effbot.org  Thu Nov  9 21:00:59 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Thu, 9 Nov 2000 21:00:59 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <00c101c04a87$c8d497f0$3c6340d5@hagrid>

I wrote:


> > Remove AC_C_INLINE test from configure.in, since the only place the symbol
> > occurs in the Python sources appears to be as text in comments.  We do not
> > want to interfere with C++ keywords!
> 
> did you benchmark SRE before and after this change?

following up on myself: SRE uses inline only if USE_INLINE is enabled,
and it's appears to be disabled in the 2.0 code base, for all platforms
except Windows (don't recall doing this; maybe someone else thought
that 10-20% performance hit was good enough?).  and now that Fred
has removed portable inline support, I cannot switch it on again :-(

besides, do people really expect to be able to use one compiler when
running configure, and another to compile Python?  sounds like they're
asking for it...

</F>




From billtut at microsoft.com  Thu Nov  9 21:28:41 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 12:28:41 -0800 
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com>

Christian Tismer wrote:
> Does anybody know of a useful example where continuations
> are really needed? 

Well, I don't know about needed, but let me explain a possible sever
architecture, and then see what new fangled control structure could help it
become more manageable. (irrespective if the perf #s of such an architecture
would actually be worth the architecture overhead in current CPython)

In a high performance server, any non-CPU intensive operation which blocks
the thread you're on reduces your scalability potential.

A fairly common way of writing multi-threaded servers is to have one client
per thread , whether this is either a thread pool, or just a simplistic
"Create thread, execute work, end thread" approach.

Threads are very expensive, and increases the context switch penalty your
server is inflicted with.

An alternative which reduces the context switch penalty dramatically is to
use a thread safe work item queue, and N threads where N is usually some
small multiple of the # of CPUs, and is < than the thread pool # of threads.
The solution to avoid these threads from blocking is an asynchronous state
machine. You asynchronously start the long-time operation, and on completion
of this operation insert the request state back in the thread safe queue.

You can further extend this idea to reduce the # of threads that you have so
that you have only 1 thread/CPU. Each thread being bound directly to the
CPU, and not allowed to run on other CPUs.
This tries to prevent the threads from CPU switching and ruining the CPU
cache, etc.
A work item must be able to be executed on any of these threads.

Another extension of this idea is to bundle these work items into separate
queues based on the state in the state machine.
The reason for doing this is trying to prevent unnecessary CPU cache
flushing.

The downside of this approach is that asynchronous state machines are a pain
to debug, maintain, understand, write, etc...
(BTW for the curious, the above architecture does allow you to achieve much
higher perf for certain tasks than other ways of handling the code, it's
been tested and used extensively in some internal C/C++ code (not mine))

The thought occurs to me that continuations would definitely help in this
situation. 
* You'd have more debugging state
* The code is organized around other needs besides boundaries between
blocking operations.

But it's not clear to me (mostly because I haven't applied a lot of thought
about it) if coroutines would suffice here.

Thoughts?

Bill




From loewis at informatik.hu-berlin.de  Thu Nov  9 22:11:00 2000
From: loewis at informatik.hu-berlin.de (Martin von Loewis)
Date: Thu, 9 Nov 2000 22:11:00 +0100 (MET)
Subject: [Python-Dev] [Python-Help] Driver?????
Message-ID: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>

> My name is Kelli Collins, and I work for the Data Management Team at
> IBM.  I am currently working with a customer that is considering
> using DB2 as their database software, and has asked me to see if I
> can locate a driver for Python and DB2...... can you tell if one
> exists??????

Hi Kelli,

I believe there is actually more than one way to get a database module
for Python and DB2. Please have a look at the DB SIG pages, at

  http://www.python.org/topics/database/

You'll see a DB/2 module being maintained at

  ftp://people.linuxkorea.co.kr/pub/DB2/

In addition, since DB2 offers the CLI, and an ODBC driver, you can
also use the ODBC module. For that, you have two options again: on
Windows, you can use the ODBC module that is in PythonWin. On all
systems, you can use mxODBC (make sure the licensing is acceptable,
though).

In any case, all of these comply with the DB API, so application code
should port easily among these options. I guess the difference then is
ease-of-installation, performance, and access to advanced features;
since I don't use DB2 myself, I can't recommend a particular choice.

Good luck,
Martin



From guido at python.org  Thu Nov  9 22:19:21 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 16:19:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: Your message of "Thu, 09 Nov 2000 21:00:59 +0100."
             <00c101c04a87$c8d497f0$3c6340d5@hagrid> 
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>  
            <00c101c04a87$c8d497f0$3c6340d5@hagrid> 
Message-ID: <200011092119.QAA32327@cj20424-a.reston1.va.home.com>

> following up on myself: SRE uses inline only if USE_INLINE is enabled,
> and it's appears to be disabled in the 2.0 code base, for all platforms
> except Windows (don't recall doing this; maybe someone else thought
> that 10-20% performance hit was good enough?).  and now that Fred
> has removed portable inline support, I cannot switch it on again :-(

Can you explain yourself better, Fredrik?  I don't think that
USE_INLINE was ever defined.  I'd gladly accept a patch to
configure.in that defines USE_INLINE if 'inline' seems accepted by the
compiler.

> besides, do people really expect to be able to use one compiler when
> running configure, and another to compile Python?  sounds like they're
> asking for it...

Yes!  C++ uses include the same headers that were used to compile
Python, and C++ definitely qualifies as a different compiler...

--Guido van Rossum (home page: http://www.python.org/~guido/)




From greg at cosc.canterbury.ac.nz  Thu Nov  9 23:30:08 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Nov 2000 11:30:08 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <200011092230.LAA00785@s454.cosc.canterbury.ac.nz>

Christian Tismer <tismer at tismer.com>:

> Using continuations, I can build method-like functions
> without classes and instances

Couldn't you achieve the same thing with a closure, if 
Python acquires nested scopes?

My first attempt at implementing classes in Scheme
used continuations, until I realised that closures
were all that you needed.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Thu Nov  9 23:54:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 17:54:16 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 12:28:41 PST."
             <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com> 
References: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com> 
Message-ID: <200011092254.RAA04553@cj20424-a.reston1.va.home.com>

> Christian Tismer wrote:
> > Does anybody know of a useful example where continuations
> > are really needed? 

[Bill Tutt]
> Well, I don't know about needed, but let me explain a possible sever
> architecture, and then see what new fangled control structure could help it
> become more manageable. (irrespective if the perf #s of such an architecture
> would actually be worth the architecture overhead in current CPython)
[...]
> But it's not clear to me (mostly because I haven't applied a lot of thought
> about it) if coroutines would suffice here.

Indeed, this kind of server architecture is typically done with
coroutines -- it's not a good argument for the full power of
continuations. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)




From mal at lemburg.com  Fri Nov 10 00:30:20 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 10 Nov 2000 00:30:20 +0100
Subject: [Python-Dev] [Python-Help] Driver?????
References: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>
Message-ID: <3A0B338C.D1DD7CB2@lemburg.com>

Martin von Loewis wrote:
> 
> > My name is Kelli Collins, and I work for the Data Management Team at
> > IBM.  I am currently working with a customer that is considering
> > using DB2 as their database software, and has asked me to see if I
> > can locate a driver for Python and DB2...... can you tell if one
> > exists??????
> 

FYI, the next version of mxODBC will support DB/2 on all
platforms. It won't be freeware anymore, but in return you'll
get better support for Unicode and other ODBC 3.5 features.

The next round of mx.Modules will be published in the first
quarter of 2001. Until then you can use the existing mxODBC 1.1.1
which will work nicely on Windows and with some effort on Unix
for DB/2 too -- it'll give you a consistent interface to rely
on fo Windows and all supported Unix platforms.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From billtut at microsoft.com  Fri Nov 10 01:30:20 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 16:30:20 -0800 
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>

Guido van Rossum [mailto:guido at python.org] wrote:
> > Christian Tismer wrote:
> > > Does anybody know of a useful example where continuations
> > > are really needed? 
> [Bill Tutt]
> > Well, I don't know about needed, but let me explain a possible sever
> > architecture, and then see what new fangled control structure could help
it
> > become more manageable. (irrespective if the perf #s of such an
architecture
> > would actually be worth the architecture overhead in current CPython)
> [...]
> > But it's not clear to me (mostly because I haven't applied a lot of
thought
> > about it) if coroutines would suffice here.

> Indeed, this kind of server architecture is typically done with
> coroutines -- it's not a good argument for the full power of
> continuations. :-)

Hrm, but can I get enough info from the coroutine support to
automatically determine which "work item queue" the paused routine
should go into after the "blocking" operation completes?
Additionally, the coroutine can't be stuck to only one system
thread. That would be bad.

If the above can't be done with coroutines, then I'm either stuck with
an async state machine, or something that might not scale as well as
it could. :(

(Ignoring the interesting discussion about how exactly you make this
architecture portable, since AFAIK only NT lets you bind threads to
CPUs)

I wish I had the time to play with stackless and hack up something
like I mentioned. The performance benefits to async state machine C++ code
that
use this idea are pretty impressive. CPU cache misses hurt. :(

Bill



From tismer at tismer.com  Fri Nov 10 12:39:24 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 10 Nov 2000 13:39:24 +0200
Subject: [Python-Dev] uthread strawman
References: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>
Message-ID: <3A0BDE6C.1712F87E@tismer.com>


Bill Tutt wrote:
> 
> Guido van Rossum [mailto:guido at python.org] wrote:
> > > Christian Tismer wrote:
> > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > [Bill Tutt]

Bill on [server architecture, async state machine]

> > Indeed, this kind of server architecture is typically done with
> > coroutines -- it's not a good argument for the full power of
> > continuations. :-)
> 
> Hrm, but can I get enough info from the coroutine support to
> automatically determine which "work item queue" the paused routine
> should go into after the "blocking" operation completes?
> Additionally, the coroutine can't be stuck to only one system
> thread. That would be bad.

Coroutine vs. continuation isn't the point here, I think.
What you would likely need is gathering information about
the frames and variables in your co-whatever. This is quite
simple to implement, with either architecture.
My next version will not only allow to inspect the eval
stack of a frame, but also inspection and modification
of local variables; still this is no continuation feature.
We can simply inspect anything.

What makes the difference is callability:
Whenever we expose arbitrary frames as callable objects,
or we provide an unrestricted other way to execute them,
then exactly all the problems and consequences of
continuations pop up: we need to clone execution state,
since now it is possible to have more than one state for
a frame.
If we avoid this, we can still have all control and debugging
features with coroutines.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From billtut at microsoft.com  Fri Nov 10 20:49:40 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Fri, 10 Nov 2000 11:49:40 -0800
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B7@red-msg-07.redmond.corp.microsoft.com>


From: 	Christian Tismer [mailto:tismer at tismer.com] 

> Bill Tutt wrote:
> > 
> > Guido van Rossum [mailto:guido at python.org] wrote:
> > > > Christian Tismer wrote:
> > > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > > [Bill Tutt]

> Bill on [server architecture, async state machine]

> > > Indeed, this kind of server architecture is typically done with
> > > coroutines -- it's not a good argument for the full power of
> > > continuations. :-)
> 
> > Hrm, but can I get enough info from the coroutine support to
> > automatically determine which "work item queue" the paused routine
> > should go into after the "blocking" operation completes?
> > Additionally, the coroutine can't be stuck to only one system
> > thread. That would be bad.

> Coroutine vs. continuation isn't the point here, I think.
> What you would likely need is gathering information about
> the frames and variables in your co-whatever. This is quite
> simple to implement, with either architecture.
> My next version will not only allow to inspect the eval
> stack of a frame, but also inspection and modification
> of local variables; still this is no continuation feature.
> We can simply inspect anything.

Coolness. 

> What makes the difference is callability:
> Whenever we expose arbitrary frames as callable objects,
> or we provide an unrestricted other way to execute them,
> then exactly all the problems and consequences of
> continuations pop up: we need to clone execution state,
> since now it is possible to have more than one state for
> a frame.
> If we avoid this, we can still have all control and debugging
> features with coroutines.

Ah, well this definitely would cause all sorts of confusing things.
I did mention I hadn't spent much time thinking about all of the
implementation details, I just know the requirements the architecture idea
has. :) As long as the co-whatevers can migrate to any system thread, I'm
one happy camper.

Thanks for helping clear stuff up folks,
Bill




From tim_one at email.msn.com  Sat Nov 11 10:47:10 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Sat, 11 Nov 2000 04:47:10 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEMOHPAA.tim_one@email.msn.com>

[Moshe]
> I imagined something like
>
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
>
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

[Guido]
> You can make the exception handler into a separate coroutine -- then
> no reusable continuations are needed.  The coroutine can repair the
> exception and resume the other.

Since I'm not a fan of resumable exceptions, I won't beat it to death.

Note that there are at least three meanings for "resumable exception" in
actual use (and I believe, e.g., that modern (ANSI) Smalltalk supports all
three):

1. Handler returns a value, and the failing operation acts as if that were
what it had computed from the start.  Very common in HW (e.g., a page fault
on a memory load acts this way; ditto 1e300**2 returning +Inf under 754
double arithmetic when the overflow signal is masked out).  Note that no
lexically enclosing "try block" is needed in this case (it is in the next
two, although they don't need "except" clauses attached to their lexically
enclosing "try").

2. Handler tells the entire originating "try block" to start over.

3. Handler resumes right after the originating try block.

In all of these, you have to search "up the call stack" for a handler, which
then wants to resume as if the call stack had not been unwound (albeit that
in #2 and #3, wants to resume at different places than where the exception
was raised!).  "Up the call stack" means within your current thread of
control, so it's hard to see how to make an independent coroutine out of
that on demand (the handler needs to work in its natural lexical context,
incl. modifying values of *its* scope's locals that will be visible again
when the deeper routine with the exception eventually does normal returns
back to it).

The details are excruciating -- so it's a good thing nobody in their right
mind wants any of these <wink>.

relievedly y'rs  - tim





From mal at lemburg.com  Sat Nov 11 10:52:27 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 10:52:27 +0100
Subject: [Python-Dev] Help wanted...
Message-ID: <3A0D16DB.A8724B9B@lemburg.com>

Hi everybody,

I just wanted to inform you that I won't be able to work on the
patches needed to fix the bugs assigned to me for the next month.
I currently have to finish a huge project with a deadline in mid-
December which has high priority (anyone ever write a B2B
eCommerce order solution in one month ?) ... I hope Python
get's me there in time.

Anyway, if any of the bugs currently assigned to me need a
fast fix, I'd suggest someone else taking a look at them, since
I won't be able to do much about these until around Christmas.
If you need help about some Unicode detail, just drop me line.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov 11 15:05:38 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 15:05:38 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <3A0D5232.CF6A884@lemburg.com>

I am looking for a way to do DNS resolution with a timeout
on the lookup. The easy way to do these lookups using the
socket module often result in processes hanging for several
seconds (up to minutes on Windows), so I figured emulating the
standard DNS lookup (including the /etc/hosts and /etc/resolv.conf
files) would be a good idea.

I know of the Demo/dns/, but I'm unsure whether it can be
tweaked to allow timeouts.

Are there any other such implementations out there ?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From thomas at xs4all.net  Sat Nov 11 15:23:58 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 11 Nov 2000 15:23:58 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: <3A0D5232.CF6A884@lemburg.com>; from mal@lemburg.com on Sat, Nov 11, 2000 at 03:05:38PM +0100
References: <3A0D5232.CF6A884@lemburg.com>
Message-ID: <20001111152358.O27208@xs4all.nl>

On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:

> I am looking for a way to do DNS resolution with a timeout on the lookup.
> [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> allow timeouts.

> Are there any other such implementations out there ?

Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
timeouts, but it does do async lookups ;) He posted this a couple of days
ago, to the python list:

---------------------------------------------------------------------------
From: Anthony Baxter <anthony at interlink.com.au>
To: support at internetdiscovery.com (Mike Clarkson)
Cc: python-list at python.org
Subject: Re: Async DNS? 

The python DNS library I released a while ago (based on Guido's code)
did this. I just had a poke around and realised that it's not visible
anywhere on the net - I just dropped a copy into

http://www.zope.org/Members/anthony/software/dns.tar.gz
(sorry DC guys for abusing the zope.org site - to make it fair, I'll
whip up a DNS zope product at some point :)

I haven't looked at it in an age or two, but it used to work fine. :)

Anthony

>>> Mike Clarkson wrote
> Does anyone have asynchronous DNS lookup for Python?
> 
> Presumably it could be done in Python using sockets, but someone
> must have done a C implementation.
> 
> Many thanks in advance,
> 
> Mike.
> -- 
> http://www.python.org/mailman/listinfo/python-list
> 

-- 
Anthony Baxter     <anthony at interlink.com.au>   
It's never too late to have a happy childhood.
---------------------------------------------------------------------------

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Sat Nov 11 19:15:52 2000
From: guido at python.org (Guido van Rossum)
Date: Sat, 11 Nov 2000 13:15:52 -0500
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: Your message of "Sat, 11 Nov 2000 15:05:38 +0100."
             <3A0D5232.CF6A884@lemburg.com> 
References: <3A0D5232.CF6A884@lemburg.com> 
Message-ID: <200011111815.NAA12217@cj20424-a.reston1.va.home.com>

> I know of the Demo/dns/, but I'm unsure whether it can be
> tweaked to allow timeouts.

Hm, typically the protocol used is UDP, and it's easy to set timeouts
there: just use select on the socket.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From martin at loewis.home.cs.tu-berlin.de  Sun Nov 12 00:58:12 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 12 Nov 2000 00:58:12 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>

> Are there any other such implementations out there ?

How about http://dustman.net/andy/python/adns-python?

Regards,
Martin

P.S. Is this really on-topic for python-dev?



From moshez at zadka.site.co.il  Sun Nov 12 21:03:00 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sun, 12 Nov 2000 22:03:00 +0200
Subject: [Python-Dev] Weak References
Message-ID: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>

For those of you who do not read c.l.py

From: shindich at my-deja.com
Newsgroups: comp.lang.python
Subject: Weak References
Date: Sun, 12 Nov 2000 11:07:32 GMT

Ever wanted to have weak references in Python?
Pre-alpha release of PyWeakReference project is now available on Source
Forge (http://sourceforge.net/projects/pyweakreference/)
Documentation is coming soon... (some docs are available in the Doc
subdirectory of the zip file)


Sent via Deja.com http://www.deja.com/
Before you buy.
- -- 
http://www.python.org/mailman/listinfo/python-list



------- End of Forwarded Message

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Sun Nov 12 13:55:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:55:42 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>
Message-ID: <3A0E934E.8BC64ED3@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Are there any other such implementations out there ?
> 
> How about http://dustman.net/andy/python/adns-python?

Thanks for the hint.
 
> Regards,
> Martin
> 
> P.S. Is this really on-topic for python-dev?

Not really... but I thought since this question often arises
when using sockets, it might be a good idea to have something
like dnslib in the standard lib together with some support
for doing DNS lookups with predefined timeouts.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 13:58:44 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:58:44 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <200011111815.NAA12217@cj20424-a.reston1.va.home.com>
Message-ID: <3A0E9404.68BD77D0@lemburg.com>

Guido van Rossum wrote:
> 
> > I know of the Demo/dns/, but I'm unsure whether it can be
> > tweaked to allow timeouts.
> 
> Hm, typically the protocol used is UDP, and it's easy to set timeouts
> there: just use select on the socket.

Hmm, yes that would be a possibility. I'll see if I can come
up with some code which implements this.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 14:00:05 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:00:05 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <20001111152358.O27208@xs4all.nl>
Message-ID: <3A0E9455.83AE479E@lemburg.com>

Thomas Wouters wrote:
> 
> On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:
> 
> > I am looking for a way to do DNS resolution with a timeout on the lookup.
> > [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> > allow timeouts.
> 
> > Are there any other such implementations out there ?
> 
> Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
> timeouts, but it does do async lookups ;) He posted this a couple of days
> ago, to the python list:

I did know of that lib -- but it seems a bit over the top for the
simple lookups I hav in mind.
 
> ---------------------------------------------------------------------------
> From: Anthony Baxter <anthony at interlink.com.au>
> To: support at internetdiscovery.com (Mike Clarkson)
> Cc: python-list at python.org
> Subject: Re: Async DNS?
> 
> The python DNS library I released a while ago (based on Guido's code)
> did this. I just had a poke around and realised that it's not visible
> anywhere on the net - I just dropped a copy into
> 
> http://www.zope.org/Members/anthony/software/dns.tar.gz
> (sorry DC guys for abusing the zope.org site - to make it fair, I'll
> whip up a DNS zope product at some point :)
> 
> I haven't looked at it in an age or two, but it used to work fine. :)
> 
> Anthony
> 
> >>> Mike Clarkson wrote
> > Does anyone have asynchronous DNS lookup for Python?
> >
> > Presumably it could be done in Python using sockets, but someone
> > must have done a C implementation.
> >
> > Many thanks in advance,
> >
> > Mike.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 14:17:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:17:33 +0100
Subject: [Python-Dev] Weak References
References: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <3A0E986D.43B170BA@lemburg.com>

Moshe Zadka wrote:
> 
> For those of you who do not read c.l.py
> 
> From: shindich at my-deja.com
> Newsgroups: comp.lang.python
> Subject: Weak References
> Date: Sun, 12 Nov 2000 11:07:32 GMT
> 
> Ever wanted to have weak references in Python?
> Pre-alpha release of PyWeakReference project is now available on Source
> Forge (http://sourceforge.net/projects/pyweakreference/)
> Documentation is coming soon... (some docs are available in the Doc
> subdirectory of the zip file)

Since more and more of these implementations are popping up
like mushrooms ;-) ... could you add some discussion about the
different methods of adding weak refs to Python to the PEP 205 ?

--
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Sun Nov 12 14:55:51 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 12 Nov 2000 15:55:51 +0200 (IST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011121554400.1528-100000@sundial>

On Sun, 12 Nov 2000, M.-A. Lemburg wrote:

> Since more and more of these implementations are popping up
> like mushrooms ;-) ... could you add some discussion about the
> different methods of adding weak refs to Python to the PEP 205 ?

Well, it's one of the few PEPs which are not yet mine <wink>:
I suppose Fred will do that pretty soon, though.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Mon Nov 13 13:54:43 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 07:54:43 -0500
Subject: [Python-Dev] Nested functions and class scope
In-Reply-To: Your message of "Mon, 13 Nov 2000 10:24:51 +1100."
             <3A0F26C3.7CD823BF@ozemail.com.au> 
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
            <3A0F26C3.7CD823BF@ozemail.com.au> 
Message-ID: <200011131254.HAA22379@cj20424-a.reston1.va.home.com>

John Max Skaller wrote:
> Guido van Rossum wrote:
> ===============>>>>
> > Other issues that need sorting out:
> > 
> > x = 2
> > class C:
> >   x = 1
> >   C = 'some string'
> >   def a(self):
> >      print x
> >   def b(self):
> >      global x
> >      x = 3
> > 
> > class D(C):
> >   C = 'some string'
> >   def a(self):
> >      C.a(self)
> >      print C
> > 
> > o = C()
> > o.a()
> > o.b()
> > o.a()
> > 
> > o = D()
> > o.a()
> > 
> > What would the output look like under your proposal ?
> 
> This is a good point!  If we considered the class as a nested scope
> here, I think it might break too much code, plus it would allow a new
> coding style where you could reference class variables without a self
> or <classname> prefix.  I don't like that prospect, so I'm in favor
> for ruling this out.
> <<<===================
> 
> You're missing the point. you can refer to class variables without
> a prefix right now:
> 
> class X:
> 	x = 1
> 	y = x
> 
> 
> It would be madness NOT to support lexically scoped nested classes
> as well as functions. But note that these scopes only exist
> when a class is executed, that is, when the class is defined.
> This has NO impact on finding class variables from an instance.
> 
> output of above from Vyper:
> --------------------------------------------------------------
> Viperi 2.0.1
> Copyright Maxtal P/L, John Skaller, Australia, 1999
> >>>x = 2
> ...class C:
> ...  x = 1
> ...  C = 'some string'
> ...  def a(self):
> ...     print x
> ...  def b(self):
> ...     global x
> ...     x = 3
> ...
> >>>class D(C):
> ...  C = 'some string'
> ...  def a(self):
> ...     C.a(self)
> ...     print C
> ...
> >>>o = C()
> ...o.a()
> ...o.b()
> ...o.a()
> ...
> 1 
> 3 
> >>>o = D()
> ...o.a()
> ...
> Error Near line 4
> 
> Uncaught Python Exception at top level!!
>   .. Kind: Instance of AttributeError
>   .. Attributes:
>       args --> Cannot find attribute of "some string" "a"
> 
> Traceback (innermost first)
> File: <string> Line: 4
> <cannot get line from file>
> File: <string> Line: 2
> <cannot get line from file>
> 
> -- 
> John (Max) Skaller, mailto:skaller at maxtal.com.au
> 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
> checkout Vyper http://Vyper.sourceforge.net
> download Interscript http://Interscript.sourceforge.net

Looks like you are missing the point.  Of course I want to keep the
class as a local scope *inside the class def*.  But I need it out of
the way when executing methods.  The example code should print

2			# a() references global x, not C.x
3			# global x got set to three by b()
2			# C refers to global class C
<class C ...>		# ditto

I agree that this is surprising.  But I believe it would break too
much code if we implemented your (simpler) scheme.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Mon Nov 13 15:09:46 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 13 Nov 2000 09:09:46 -0500 (EST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
References: <3A0E986D.43B170BA@lemburg.com>
	<Pine.GSO.4.10.10011121554400.1528-100000@sundial>
	<E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <14863.63018.678116.930743@cj42289-a.reston1.va.home.com>

M.-A. Lemburg writes:
 > Since more and more of these implementations are popping up
 > like mushrooms ;-) ... could you add some discussion about the
 > different methods of adding weak refs to Python to the PEP 205 ?

Moshe Zadka writes:
 > Well, it's one of the few PEPs which are not yet mine <wink>:
 > I suppose Fred will do that pretty soon, though.

  I plan to do this sometime this week.  I have a bunch of text that
isn't already checked in, so this isn't the time to open up editorial
control just yet.
  The notice Moshe forwarded didn't slip past me, but I've been a
little under the weather the past few days.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From nas at arctrix.com  Mon Nov 13 09:36:53 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 13 Nov 2000 00:36:53 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001113003653.A13421@glacier.fnational.com>

I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil



From mal at lemburg.com  Mon Nov 13 16:32:04 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 13 Nov 2000 16:32:04 +0100
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
References: <20001113003653.A13421@glacier.fnational.com>
Message-ID: <3A100974.969045C0@lemburg.com>

Neil Schemenauer wrote:
> 
> I making this available now in the hope that it will generate
> some dicussion:
> 
>     http://arctrix.com/nas/python/coerce-2.diff
> 
> The patch is based on Marc-Andre Lemburg's coercion patch for
> 1.5.1 downloaded from the starship:
> 
>     http://starship.python.net/~lemburg/CoercionProposal.html
> 
> All credit goes to him.  The bugs are probably introduced by me.
> What the patch does:
> 
>   - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
>     set then the tp_as_number methods are treated differently.
>     Coercion is not done before passing arguments to the method.
>     New style methods do their own coercion or return the
>     NotImplemented singleton.  See Marc's page for more details.

Very nice !

I wanted to suggest the usage of a type flag
instead of the slot magic in the original patch too -- your time
machine was faster ;-)
 
>   - Decide on handling of nb_coerce slot (eg. should it be called
>     on new style numbers).

Probably not, since all coercion is now handled by the slots
themselves.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From trentm at ActiveState.com  Mon Nov 13 22:16:26 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:16:26 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011131948.LAA04697@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Mon, Nov 13, 2000 at 11:48:25AM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net>
Message-ID: <20001113131626.B4553@ActiveState.com>

On Mon, Nov 13, 2000 at 11:48:25AM -0800, Guido van Rossum wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv4689
> 
> Modified Files:
> 	fileobject.c 
> Log Message:
> Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> hack for TELL64()...  Sounds like there's something else going on
> really.  Does anybody have a clue I can buy?

Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
when it should not really be. Essentially, these platforms lie when they say
they support largefiles, at least according to the configure test for
HAVE_LARGEFILE_SUPPORT>

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From guido at python.org  Mon Nov 13 22:38:01 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 16:38:01 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:16:26 PST."
             <20001113131626.B4553@ActiveState.com> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net>  
            <20001113131626.B4553@ActiveState.com> 
Message-ID: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>

> > Log Message:
> > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > hack for TELL64()...  Sounds like there's something else going on
> > really.  Does anybody have a clue I can buy?
> 
> Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> when it should not really be. Essentially, these platforms lie when they say
> they support largefiles, at least according to the configure test for
> HAVE_LARGEFILE_SUPPORT>

Mh...  Thanks.  And how are we testing for large file support?
Perhaps the test should be improved rather than hacks piled on top of
each other?  What's wrong with the current test?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From trentm at ActiveState.com  Mon Nov 13 22:59:05 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:59:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 04:38:01PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>
Message-ID: <20001113135905.B7508@ActiveState.com>

On Mon, Nov 13, 2000 at 04:38:01PM -0500, Guido van Rossum wrote:
> > > Log Message:
> > > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > > hack for TELL64()...  Sounds like there's something else going on
> > > really.  Does anybody have a clue I can buy?
> > 
> > Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> > when it should not really be. Essentially, these platforms lie when they say
> > they support largefiles, at least according to the configure test for
> > HAVE_LARGEFILE_SUPPORT>
> 
> Mh...  Thanks.  And how are we testing for large file support?
> Perhaps the test should be improved rather than hacks piled on top of
> each other?  What's wrong with the current test?
> 

I agree: the test should improve. Here it is:

AC_MSG_CHECKING(whether to enable large file support)
if test "$have_long_long" = yes -a \
    "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
    "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
  AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
  AC_MSG_RESULT(yes)
else
  AC_MSG_RESULT(no)
fi


BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
(i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
think). In windows is is _tell64, I think. ...anyway there are a lot of ways
to spell it.

I don't know right away how to translate that into an appropriate configure
test. I.e. how do you test that the platform has an ftell/fseek-like function
that uses an index variable whose sizeof() is at least 8. 

Note that, IIRC, windows had some funny case where the test above would have
failed but that was okay because there were lower level I/O functions that
used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
details. 



Trent


-- 
Trent Mick
TrentM at ActiveState.com



From akuchlin at mems-exchange.org  Mon Nov 13 23:13:58 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Mon, 13 Nov 2000 17:13:58 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113135905.B7508@ActiveState.com>; from trentm@ActiveState.com on Mon, Nov 13, 2000 at 01:59:05PM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com>
Message-ID: <20001113171358.A7407@kronos.cnri.reston.va.us>

On Mon, Nov 13, 2000 at 01:59:05PM -0800, Trent Mick wrote:
>I don't know right away how to translate that into an appropriate configure
>test. I.e. how do you test that the platform has an ftell/fseek-like function
>that uses an index variable whose sizeof() is at least 8. 

If you look at the very latest autoconf alpha release, autoconf 2.49a,
at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
macro that may be what's needed.  Perhaps we can back-port this macro
to the current autoconf release.

--amk



From thomas at xs4all.net  Mon Nov 13 23:20:20 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:20:20 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113171358.A7407@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 13, 2000 at 05:13:58PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <20001113171358.A7407@kronos.cnri.reston.va.us>
Message-ID: <20001113232020.R27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:13:58PM -0500, Andrew Kuchling wrote:

> If you look at the very latest autoconf alpha release, autoconf 2.49a,
> at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
> macro that may be what's needed.  Perhaps we can back-port this macro
> to the current autoconf release.

Aahh, hmm. I'll look at what this macro does tomorrow, unless someone else
beats me to it. I've been busy, between gobs of work, trying to find out how
*other* applications determine large file support... that's suprisingly
tricky in some cases -- I still haven't figured out how zsh does it.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Mon Nov 13 23:36:06 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:36:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:59:05 PST."
             <20001113135905.B7508@ActiveState.com> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>  
            <20001113135905.B7508@ActiveState.com> 
Message-ID: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>

> > Mh...  Thanks.  And how are we testing for large file support?
> > Perhaps the test should be improved rather than hacks piled on top of
> > each other?  What's wrong with the current test?
> > 
> 
> I agree: the test should improve. Here it is:
> 
> AC_MSG_CHECKING(whether to enable large file support)
> if test "$have_long_long" = yes -a \
>     "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
>     "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
>   AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
>   AC_MSG_RESULT(yes)
> else
>   AC_MSG_RESULT(no)
> fi
> 
> 
> BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
> (i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
> posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
> think). In windows is is _tell64, I think. ...anyway there are a lot of ways
> to spell it.
> 
> I don't know right away how to translate that into an appropriate configure
> test. I.e. how do you test that the platform has an ftell/fseek-like function
> that uses an index variable whose sizeof() is at least 8. 
> 
> Note that, IIRC, windows had some funny case where the test above would have
> failed but that was okay because there were lower level I/O functions that
> used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
> details. 

After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
means that the low-level system calls (lseek(), stat() etc.)  support
large files, through an off_t type that is at least 8 bytes (assumed
to be equivalent with a long long in some places, given the use of
PyLong_FromLongLong() and PyLong_AsLongLong()).

But the problems occur in fileobject.c, where we're dealing with the
stdio library.  Not all stdio libraries seem to support long files in
the same way, and they use a different typedef, fpos_t, which may be
larger or smaller in size than off_t.

Aha!  It looks like the problem is that under a variety of
circumstances TELL64(fd) is required; but these have in common that
HAVE_LARGEFILE_SUPPORT is defined, and in all cases except MS_WIN64 it
is defined as equal to lseek((fd),0,SEEK_CUR).

So wouldn't the problem be solved if we changed the #ifdefs so that
that is the default definition, instead of only defining that for
specific platforms?  As more and more platforms develop kernel support
for 64-bit offsets but their stdio libraries lag behind (or use APIs
we don't know about, or assume that fsetpos() is sufficient), we'll
run into this more and more often.

Here's the current logic for defining TELL64() in fileobject.c:

#if defined(MS_WIN64)
#define TELL64 _telli64
#elif defined(__NetBSD__) || ... /* etc. */
/* NOTE: this is only used on older
   NetBSD prior to f*o() funcions */
#define TELL64(fd) lseek((fd),0,SEEK_CUR)
#endif

There's exactly one place where TELL64() is subsequently used, which
is inside

#elif defined(HAVE_LARGEFILE_SUPPORT) && SIZEOF_FPOS_T >= 8
#else

Any objections???

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Mon Nov 13 23:41:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:41:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:36:06PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>
Message-ID: <20001113234121.S27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:

> After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> means that the low-level system calls (lseek(), stat() etc.)  support
> large files, through an off_t type that is at least 8 bytes (assumed
> to be equivalent with a long long in some places, given the use of
> PyLong_FromLongLong() and PyLong_AsLongLong()).

> But the problems occur in fileobject.c, where we're dealing with the
> stdio library.  Not all stdio libraries seem to support long files in
> the same way, and they use a different typedef, fpos_t, which may be
> larger or smaller in size than off_t.

This isn't the problem. The problem is that we assume that because off_t is
8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
bytes, but none of the available fseek/ftell variations take an off_t as
argument ;P The TELL64() workaround works around that problem, but still
doesn't enable large file support, because there isn't any such support in
BSDI.

(Trust me... we've had logfiles and IP-traffic databases truncated because
of that... 2Gb is the limit, currently, on BSDI.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Mon Nov 13 23:52:46 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:52:46 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 23:41:21 +0100."
             <20001113234121.S27208@xs4all.nl> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>  
            <20001113234121.S27208@xs4all.nl> 
Message-ID: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>

> On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> 
> > After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> > means that the low-level system calls (lseek(), stat() etc.)  support
> > large files, through an off_t type that is at least 8 bytes (assumed
> > to be equivalent with a long long in some places, given the use of
> > PyLong_FromLongLong() and PyLong_AsLongLong()).
> 
> > But the problems occur in fileobject.c, where we're dealing with the
> > stdio library.  Not all stdio libraries seem to support long files in
> > the same way, and they use a different typedef, fpos_t, which may be
> > larger or smaller in size than off_t.
> 
> This isn't the problem. The problem is that we assume that because off_t is
> 8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
> bytes, but none of the available fseek/ftell variations take an off_t as
> argument ;P The TELL64() workaround works around that problem, but still
> doesn't enable large file support, because there isn't any such support in
> BSDI.
> 
> (Trust me... we've had logfiles and IP-traffic databases truncated because
> of that... 2Gb is the limit, currently, on BSDI.)

Sure, but the #ifdef isn't really about how well the kernel supports
large files -- it is about what code you must use.  There's one set of
places that uses off_t, and those are guided just fine by
HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
larger than 2Gb!

But there's a *different* set of test that must be used to determine
what to do for the stdio calls.  Note that on all platforms so far
where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
and Mac OSX), there were only compilation problems in fileobject.c,
and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).

What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
doesn't actually support files larger than 2Gb?  I suppose you get
some sort of I/O error when you try to write such files.  Since they
can't be created, you can't run into trouble when trying to read
them! :-)

Or am I still missing something?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From trentm at ActiveState.com  Tue Nov 14 00:07:05 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 15:07:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:52:46PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com> <20001113234121.S27208@xs4all.nl> <200011132252.RAA25873@cj20424-a.reston1.va.home.com>
Message-ID: <20001113150705.A12976@ActiveState.com>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> Sure, but the #ifdef isn't really about how well the kernel supports
> large files -- it is about what code you must use.  There's one set of
> places that uses off_t, and those are guided just fine by
> HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
> larger than 2Gb!
> 
> But there's a *different* set of test that must be used to determine
> what to do for the stdio calls.  Note that on all platforms so far
> where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
> and Mac OSX), there were only compilation problems in fileobject.c,
> and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).
> 
> What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
> doesn't actually support files larger than 2Gb?  I suppose you get
> some sort of I/O error when you try to write such files.  Since they
> can't be created, you can't run into trouble when trying to read
> them! :-)
> 
> Or am I still missing something?

I don't *think* so. Though having HAVE_LARGEFILE_SUUPORT still be defined for
a platform where this is not true still nags.

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From jeremy at alum.mit.edu  Tue Nov 14 05:16:01 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 13 Nov 2000 23:16:01 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
Message-ID: <14864.48257.324726.976892@bitdiddle.concentric.net>

I just saw your note about whether IDLE was packaged with the Python
RPMs.  It isn't, nor is anything in the Tools directory.  I did the
simplest thing possible with the RPM and ran "make install".  The RPM
installs whatever is installed as a result of that.

We've discussed installation of other tools once or twice, but I've
never decided what exactly to do.  The compiler package, for example,
comes with a distutils setup.py script that will do the job.  I think
the best way to move forward is to write setup.py scripts for all the
other tools that ought to be installed.

I don't know how to package up idle cleanly, though.  I need to decide
what to install and where to install it.  The Tools/idle directory is
nominally a package, but it doesn't seem like that will help much for
installing it.  If we want to call the executable script idle.py, it
will clash with the idle package name.  Any advice here?  It doesn't
seem like there is any obvious way to "install" IDLE; it looks like it
is intended to be run out of the source directory.

If anyone else owns (or otherwise feels responsible for) a package in
the Tools directory that ought to be included with the RPMs, please
write a setup script for it.

Jeremy



From moshez at zadka.site.co.il  Tue Nov 14 14:19:11 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 14 Nov 2000 15:19:11 +0200
Subject: [Python-Dev] installing IDLE & other Tools 
In-Reply-To: Message from Jeremy Hylton <jeremy@alum.mit.edu> 
   of "Mon, 13 Nov 2000 23:16:01 EST." <14864.48257.324726.976892@bitdiddle.concentric.net> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> 
Message-ID: <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>

[Jeremy Hylton]
> We've discussed installation of other tools once or twice, but I've
> never decided what exactly to do.  The compiler package, for example,
> comes with a distutils setup.py script that will do the job.  I think
> the best way to move forward is to write setup.py scripts for all the
> other tools that ought to be installed.

That does seem reasonable. It seems the most non-intrusive, since we
get out of the way of proffessional packages, while still giving them
something to start from.

> I don't know how to package up idle cleanly, though.  I need to decide
> what to install and where to install it.  The Tools/idle directory is
> nominally a package, but it doesn't seem like that will help much for
> installing it.  If we want to call the executable script idle.py, it
> will clash with the idle package name.  Any advice here?  It doesn't
> seem like there is any obvious way to "install" IDLE; it looks like it
> is intended to be run out of the source directory.

I don't think we want to call the executable script idle.py. Each platform
will have it's own problems with it: for UNIXish stuff, call it "idle".
For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
should *not* be in the same directory as the idle package. In the UNIX
case it should be in $prefix/bin, and in the windows case, it should be
where python.exe is.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From tim.one at home.com  Tue Nov 14 07:06:01 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 14 Nov 2000 01:06:01 -0500
Subject: [Python-Dev] RE: This Project Has Not Released Any Files
In-Reply-To: <200010191731.TAA01114@loewis.home.cs.tu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECAIAAA.tim.one@home.com>

[Tim, on 19 October]
> From BeOpen.com's POV, so long as they were paying major bills, they
> would rather have download traffic tickle their ad banners than SF's
> ad banners.

[Martin v. Loewis]
> Even though this should have been clear to me all the time, stating it
> explicitly triggers alarms for me.
> ...
> Of course, having ftp mirrors would mean that many downloads do not
> tickle anybody's ad banners - which would probably be in the interest
> of other users as well, just not in the interest of BeOpen. So I'm
> curious how this conflict of interest is resolved...

I hope it's clear now how it got resolved:  at the time I wrote that, the
PythonLabs guys were officially but secretly unemployed.  The issue of
distributing Python was one bone of contention at The End, but not the
mongoose that pecked the python's eyes out.

or-if-you're-not-american-it-wasn't-the-straw-that-tickled-the-
    camel-to-death-ly y'rs  - tim




From moshez at math.huji.ac.il  Tue Nov 14 10:20:01 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 14 Nov 2000 11:20:01 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: <200011132021.MAA09003@slayer.i.sourceforge.net>
Message-ID: <Pine.GSO.4.10.10011141118470.16939-100000@sundial>

On Mon, 13 Nov 2000, Guido van Rossum wrote:

> +     - Killing a thread from another thread.  Or maybe sending a
> +       signal.  Or maybe raising an asynchronous exception.
> + 
> +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470

In general, killing threads in considered a bad idea. It is usually much
better to design the application properly, and being aware that a thread
doing nothing is not that big a deal.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Tue Nov 14 13:06:48 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 07:06:48 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:20:01 +0200."
             <Pine.GSO.4.10.10011141118470.16939-100000@sundial> 
References: <Pine.GSO.4.10.10011141118470.16939-100000@sundial> 
Message-ID: <200011141206.HAA28215@cj20424-a.reston1.va.home.com>

> On Mon, 13 Nov 2000, Guido van Rossum wrote:
> 
> > +     - Killing a thread from another thread.  Or maybe sending a
> > +       signal.  Or maybe raising an asynchronous exception.
> > + 
> > +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470
> 
> In general, killing threads in considered a bad idea. It is usually much
> better to design the application properly, and being aware that a thread
> doing nothing is not that big a deal.

Sorry.  I agree 99% (see the referenced thread).  But this keeps
coming up as a feature request, so I figured that there should be
*something* we could offer -- if only some support library calls to
make implementing this common pattern smooth.

For that 1%, I believe that raising an asynchronous exception takes away
*most* of the objections against killing threads -- it gives the
thread control over recovery, since exceptions are to be expected
anyway.

Also note that in a long-running server, losing a thread every once in
a while means leaking a lot of memory!  So (in that case) the
application design cannot afford to just "park" unproductive threads
forever -- it must contain a robust way of recovering their resources.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From skaller at ozemail.com.au  Tue Nov 14 14:25:27 2000
From: skaller at ozemail.com.au (John Max Skaller)
Date: Wed, 15 Nov 2000 00:25:27 +1100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
	            <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>
Message-ID: <3A113D47.ABF1D625@ozemail.com.au>

Guido van Rossum wrote:

> Looks like you are missing the point.  Of course I want to keep the
> class as a local scope *inside the class def*.  

	Ah, ok: seem's I was missing your intent.

> But I need it out of the way when executing methods.  

	I don't think this is the right idea. Consider:

	class X:
		x = 'x'
		def f(self): print x
	a = X()
	a.f() # should print 'x'

and then

	def g(self): print x
	class X:
		f = g
	a = X()
	a.f() # should fail

and then:

	def maker():
		x = 'x'
		def f(self): print x
		return f
	class X:
		f = maker()
	a = X()
	a.f() # should work

The point is that I think there is not really much choice
in the matter: when you lookup an unqualified name in a function,
you must look in the function locals, then in the scope of the
object in which the function is defined, and then in the scope
in which that is defined .. and so ontil you reach the bottom
of the stack (which presumably is 'builtins').

	There's no real issue of keeping the class of the object
of a bound method 'out of the way' when executing the function
of the bound method. The scope of that class should be searched
if, and only if, the function happens to have been defined
inside that class. Note there is no such thing as a method
in python: executing a bound (or unbound) method results
in executing a plain old function.

	You _could_ exclude class scopes from the search stack
associated with a function, and just search locals, then
enclosing functions, then the enclosing module and finally
builtins. But it would be highly inconsistent, and not what
people used to lexical scoping would expect. It will be hard
enough to explain the _consistent_ rule that only local
bindings can be changed by assignments -- unless a global
directive changes that. But that rule IS consistent (and the
global directive much less so!)

	I think there is general agreement that lexically
scoping functions inside functions will not break much code.
Do you have an example of real application code where searching a class
in
which a function is defined will break code?

-- 
John (Max) Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
checkout Vyper http://Vyper.sourceforge.net
download Interscript http://Interscript.sourceforge.net



From guido at python.org  Tue Nov 14 14:29:17 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 08:29:17 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 15:19:11 +0200."
             <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net>  
            <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>

> [Jeremy Hylton]
> > We've discussed installation of other tools once or twice, but I've
> > never decided what exactly to do.  The compiler package, for example,
> > comes with a distutils setup.py script that will do the job.  I think
> > the best way to move forward is to write setup.py scripts for all the
> > other tools that ought to be installed.
> 
> That does seem reasonable. It seems the most non-intrusive, since we
> get out of the way of proffessional packages, while still giving them
> something to start from.

Agreed.

> > I don't know how to package up idle cleanly, though.  I need to decide
> > what to install and where to install it.  The Tools/idle directory is
> > nominally a package, but it doesn't seem like that will help much for
> > installing it.  If we want to call the executable script idle.py, it
> > will clash with the idle package name.  Any advice here?  It doesn't
> > seem like there is any obvious way to "install" IDLE; it looks like it
> > is intended to be run out of the source directory.
> 
> I don't think we want to call the executable script idle.py. Each platform
> will have it's own problems with it: for UNIXish stuff, call it "idle".
> For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
> should *not* be in the same directory as the idle package. In the UNIX
> case it should be in $prefix/bin, and in the windows case, it should be
> where python.exe is.

For Windows, the IDLE problem is solved: it's installed in directory
<topdir>/Tools/idle/ and there's a link to it in the start menu.  It's
not directly accessible from the command line -- but on Windows,
nobody cares about the command line.

Maybe we can extend this idea to Unix.  Concrete proposal: install
IDLE as <prefix>/lib/python<version>/site-packages/Tools/idle/, and
place a symlink or 2-line bootstrap script in <prefix>/bin/idle.

The introduction of the extra directory level "Tools" implies that
idle is not importable as a package from regular Python.  This is how
I want it -- otherwise I could/would have made IDLE a package in the
Lib/lib-tk directory instead of a directory in Tools.

I'd gladly accept a contribution of a setup.py file that does this for
me.  (I looked at the distutils manual for a bit but decided that I'd
have to read it cover-to-cover to figure out whether what I want is
even possible...)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Tue Nov 14 15:00:39 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 09:00:39 -0500
Subject: [Python-Dev] Re: Nested functions and class scope
In-Reply-To: Your message of "Wed, 15 Nov 2000 00:25:27 +1100."
             <3A113D47.ABF1D625@ozemail.com.au> 
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com> <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>  
            <3A113D47.ABF1D625@ozemail.com.au> 
Message-ID: <200011141400.JAA28465@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> 
> > Looks like you are missing the point.  Of course I want to keep the
> > class as a local scope *inside the class def*.  

John Max Skaller replied:

> 	Ah, ok: seem's I was missing your intent.
> 
> > But I need it out of the way when executing methods.  
> 
> 	I don't think this is the right idea. Consider:
> 
> 	class X:
> 		x = 'x'
> 		def f(self): print x
> 	a = X()
> 	a.f() # should print 'x'

We understand each other perfectly.  I just disagree with you on what
this example should do!  I believe that in the sake of backwards
compatibility *as well as* not "giving in" to Java and C++ conventions
that I believe are a bad idea for Python, the reference to 'x' from
inside 'f' should not find the class variable 'x' but only a
module-global 'x'.

Backwards compatibility: currently this program prints '1':

    x = 1
    class C:
	x = 2
	def f(self): print x
    C().f()

I believe it should not be broken; I expect that this is, for better
or for worse, a popular idiom, especially when you look at methods
rather than varibles.  I believe it might be common to find this:

    from lower_level_module import some_call
    class C:
	def some_call(self):
	    "OO wrapper around some_call()"
	    return some_call(self.data)

Your rule would break this, because the reference to some_call()
inside C.some_call() would be changed to make a (senseless, recursive)
reference to itself instead of to the globally imported some_call()
function.

Java/C++ expectations: in Java/C++, the rules are different.  Inside a
method, not just names defined in the current class are in scope, but
also all names defined in its base classes.  Look at this:

    # Module a.py
    class B:
        def f(self): pass

    # Module b.py
    import a
    class C(a.B):
	def g(self): pass
        def h(self):
	    g(self)
            f(self)

Under your rules, the reference to g() in h() would work, but the
reference to f() would fail.  In similarly structured Java or C++
code, h() could reference both g() and f() without a naming
disambiguation.

Python requires that you specify the class when you reference a class
attribute (method or class variable).  I don't want to lose this rule,
it is important (to me) for readability.

[...more examples elided...]
> The point is that I think there is not really much choice
> in the matter: when you lookup an unqualified name in a function,
> you must look in the function locals, then in the scope of the
> object in which the function is defined, and then in the scope
> in which that is defined .. and so ontil you reach the bottom
> of the stack (which presumably is 'builtins').

I claim different, and above I have explained why I think this is
necessary.  I'd agree if you said "the scope of the *function* in
which it is defined."

> 	There's no real issue of keeping the class of the object
> of a bound method 'out of the way' when executing the function
> of the bound method. The scope of that class should be searched
> if, and only if, the function happens to have been defined
> inside that class. Note there is no such thing as a method
> in python: executing a bound (or unbound) method results
> in executing a plain old function.

If the code generator keeps track of the distinction between class and
function scopes, my rules can be implemented easily without
introducing a distinction between functions and methods.

> 	You _could_ exclude class scopes from the search stack
> associated with a function, and just search locals, then
> enclosing functions, then the enclosing module and finally
> builtins. But it would be highly inconsistent, and not what
> people used to lexical scoping would expect.


Yet, this is what I want.  It's impossible to design things so that
people coming from very different backgrounds will never encounter a
surprise.  Since Python is primarily an OO language, and there's ten
years of established Python rules, I prefer to give the OO people and
existing Python users an easier time (note that the OO people at least
can count on *consistently* not seeing the class scope in the
"methods") and tell the lexical-scoping people about the exceptions.

> It will be hard
> enough to explain the _consistent_ rule that only local
> bindings can be changed by assignments -- unless a global
> directive changes that. But that rule IS consistent (and the
> global directive much less so!)

> 	I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't found one yet, but I've seen plenty of examples where a
class defines a method with the same name as a global function
(including lots in the standard library -- it really is a common
idiom).  I think this comes too close to risk breakage.

But even if we find that this doesn't break anything, I still think
it's unwise to allow unqualified access to class attributes from
within methods.  People would discover that in this example:

    class C:
	def m1(self):
	    pass
	def m2a(self):
	    m1(self)
	def m2b(self):
	    self.m1()

m2a() and m2b() would be equivalent, i.e. m1(self) and self.m1() are
equivalent.  That's evil, because it opens up a meaningless choice
between alternative (and the m1(self) notation is inferior, because it
doesn't look in base classes).

But I'm beginning to repeat myself, so I think I've said all that can
be said about this.  Sorry the scope rules in Vyper are wrong.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at pythonware.com  Tue Nov 14 15:09:37 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2000 15:09:37 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>              <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <013701c04e44$871496b0$0900a8c0@SPIFF>

John Max Skaller wrote:
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't been following this thread closely, but are we talking
about a scoping model where a class can override any built-in
function or method by defining a method (or a class attribute)
with the same name?

like in urllib.py, for example:

class URLopener:
    ...
    def open(self, fullurl, data=None):
        fullurl = unwrap(fullurl)
        if self.tempcache and self.tempcache.has_key(fullurl):
            filename, headers = self.tempcache[fullurl]
            fp = open(filename, 'rb')
            return addinfourl(fp, headers, fullurl)
    ...

this won't fly.

</F>




From mal at lemburg.com  Tue Nov 14 15:23:50 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 14 Nov 2000 15:23:50 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
		            <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <3A114AF6.EC584A17@lemburg.com>

John Max Skaller wrote:
> 
>         I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

class Object:

   type = 'typename'

   def __init__(self, obj):

       if type(obj) == types.IntType:
           ...

Basically all builtins which are in common use like open(), id()
type(), etc. would no longer be accessible.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From klm at digicool.com  Tue Nov 14 16:53:47 2000
From: klm at digicool.com (Ken Manheimer)
Date: Tue, 14 Nov 2000 10:53:47 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

There's a seminal steele and sussman paper, _The Art of the Interpreter,
or the Modularity Complex (Parts Zero, One, and Two)_, which, among other
things, introduces closures and describes why dynamic scoping is evil.  
(The paper sketches out an elementary, lexically scoped lisp interpreter
which i think is the precursor to scheme.)  It's been over 20 years since
i read it, but i recall something known as the funarg problem, where
dynamic scoping + recursion (maybe, + functions as first class objects)
inherently breaks certain parameter passing arrangements.

If you're interested, see:

  ftp://publications.ai.mit.edu/ai-publications/0-499/AIM-453.ps

(I'm working at home today, so can't even glance at it to refresh my
memory, and oughtn't at the moment, anyway.)

In any case, dynamic scoping breaks locality of reference for just about
any definition of "locality".

With lexical scoping, you can trace through the text of the program to
track down implicit references.  Even with acquisition, the containment
connections are made explicitly (though it seems magic with web
programming, because the zope publisher applies .__of__() for you), and
can be nailed down by introspection on the objects (.aq_base, .aq_parent,
.aq_inner...).

With dynamic scoping, variable references go up the stack to the context
of *any* intervening call frame, with no way for the programmer to know
where it's being resolved.  In practice, even without the funarg breakage,
it doesn't scale up - it's too promiscuous a state-communication
mechanism.  You can work around it - a lot of emacs lisp code does so -
but it's a terrible burden, not a benefit.

Ken




From esr at golux.thyrsus.com  Tue Nov 14 17:59:24 2000
From: esr at golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 11:59:24 -0500
Subject: [Python-Dev] Making popen2 actually work
Message-ID: <200011141659.LAA01557@golux.thyrsus.com>

OK, so Guido told me at one point that 2.0 was supposed to have
included changes to the way buffering of I/O to subprocesses is
handled.  I've installed 2.0, and I need to know how to make the
following program work:

import os, sys

(child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
print "* Child process spawned"

while 1:
    line = raw_input(">");
    child_stdin.write(line); child_stdin.flush()
    print "* Line written to child process"
    transformed = child_stdout.readline(); child_stdout.flush()
    print "* Line read back from child process."
    sys.stdout.write(transformed)

If anybody can tell me what the right magic is here, I'll write up 
a patch for the library docs explaining it and including the corrected
example.  This has to be a FAQ.
--
						>>esr>>



From esr at golux.thyrsus.com  Tue Nov 14 18:08:28 2000
From: esr at golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 12:08:28 -0500
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
Message-ID: <200011141708.MAA01643@golux.thyrsus.com>

I installed from the Beopen-Python-2.0 RPM at Source Forge.  Now:

golux:~/WWW/fetchmail$ fetchmailconf
Traceback (most recent call last):
  File "./fetchmailconf", line 9, in ?
    from Tkinter import *
  File "/usr/local/lib/python2.0/lib-tk/Tkinter.py", line 35, in ?
    import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter

This is a serious problem.  Is there a non-broken RPM out there?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The right of self-defense is the first law of nature: in most
governments it has been the study of rulers to confine this right
within the narrowest limits possible.  Wherever standing armies
are kept up, and when the right of the people to keep and bear
arms is, under any color or pretext whatsoever, prohibited,
liberty, if not already annihilated, is on the brink of
destruction." 
	-- Henry St. George Tucker (in Blackstone's Commentaries)



From jeremy at alum.mit.edu  Tue Nov 14 18:38:09 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 12:38:09 -0500 (EST)
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
In-Reply-To: <200011141708.MAA01643@golux.thyrsus.com>
References: <200011141708.MAA01643@golux.thyrsus.com>
Message-ID: <14865.30849.276890.227441@bitdiddle.concentric.net>

The RPM isn't "broken."  It just doesn't include _tkinter.  You'll
need to get the separate Tkinter RPM for that.

The separation is intentional, because different Linux distriubtions
have different Tk versions installed.  Rather than have the main RPM
depend on a single Tk version, they are packaged separately.  You can
also use the distutils package to install _tkinter.so.  It will work
with any version of Tk, while the RPMs will only work with the version
of Tk installed on my machine.

Jeremy



From guido at python.org  Wed Nov 15 04:48:14 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 22:48:14 -0500
Subject: [Python-Dev] Making popen2 actually work
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:59:24 EST."
             <200011141659.LAA01557@golux.thyrsus.com> 
References: <200011141659.LAA01557@golux.thyrsus.com> 
Message-ID: <200011150348.WAA05535@cj20424-a.reston1.va.home.com>

> OK, so Guido told me at one point that 2.0 was supposed to have
> included changes to the way buffering of I/O to subprocesses is
> handled.  I've installed 2.0, and I need to know how to make the
> following program work:
> 
> import os, sys
> 
> (child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
> print "* Child process spawned"
> 
> while 1:
>     line = raw_input(">");
>     child_stdin.write(line); child_stdin.flush()
>     print "* Line written to child process"
>     transformed = child_stdout.readline(); child_stdout.flush()
>     print "* Line read back from child process."
>     sys.stdout.write(transformed)
> 
> If anybody can tell me what the right magic is here, I'll write up 
> a patch for the library docs explaining it and including the corrected
> example.  This has to be a FAQ.

First, raw_input() strips the newline, so you should change the second
line of the loop body to

     child_stdin.write(line+'\n'); child_stdin.flush()

Second, with "cat" instead of "/usr/bin/rev", this works fine for me,
so your logic is fine.

The trick seems to be to get the subprocess not to buffer its output
(or input).  There's nothing that Python can do for you here!

Unfortunately, most programs use stdio for all their console I/O, and
stdio automatically buffers its output when connected to a pipe... :-(

If you absolutely, positively have to do this with a program that
won't unbuffer its output, your only resort would be to use pseudo
ttys -- but that's quite a different story... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Wed Nov 15 03:13:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 21:13:36 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>
References: <14864.48257.324726.976892@bitdiddle.concentric.net>
	<E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>
	<200011141329.IAA28404@cj20424-a.reston1.va.home.com>
Message-ID: <14865.61776.50840.609922@bitdiddle.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

  >> > I don't know how to package up idle cleanly, though.  I need to
  >> > decide what to install and where to install it.  The Tools/idle
  >> > directory is nominally a package, but it doesn't seem like that
  >> > will help much for installing it.  If we want to call the
  >> > executable script idle.py, it will clash with the idle package
  >> > name.  Any advice here?  It doesn't seem like there is any
  >> > obvious way to "install" IDLE; it looks like it is intended to
  >> > be run out of the source directory.
  >>

  GvR> For Windows, the IDLE problem is solved: it's installed in
  GvR> directory <topdir>/Tools/idle/ and there's a link to it in the
  GvR> start menu.  It's not directly accessible from the command line
  GvR> -- but on Windows, nobody cares about the command line.

  GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
  GvR> install IDLE as
  GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
  GvR> place a symlink or 2-line bootstrap script in
  GvR> <prefix>/bin/idle.

How about
<prefix>/lib/python<version>/Tools/idle
instead.  I suggest the change to support an until-now-implicit policy
that anything in site-packages actually be a package.  If things in
site-packages aren't supposed to be packages, it ought to be called
something like site-tools <0.2 wink>.

  GvR> The introduction of the extra directory level "Tools" implies
  GvR> that idle is not importable as a package from regular Python.
  GvR> This is how I want it -- otherwise I could/would have made IDLE
  GvR> a package in the Lib/lib-tk directory instead of a directory in
  GvR> Tools.

I think the extra Tools directory still implies this, even if it isn't
in site-packages.

  GvR> I'd gladly accept a contribution of a setup.py file that does
  GvR> this for me.  (I looked at the distutils manual for a bit but
  GvR> decided that I'd have to read it cover-to-cover to figure out
  GvR> whether what I want is even possible...)

Is there a reason why IDLE has this peculiar installation process?
Might there be a better way to do it now that we have distutils.  If
there is, we could put it in PEP 42.

Jeremy



From guido at python.org  Wed Nov 15 05:10:06 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 23:10:06 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 21:13:36 EST."
             <14865.61776.50840.609922@bitdiddle.concentric.net> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com>  
            <14865.61776.50840.609922@bitdiddle.concentric.net> 
Message-ID: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>

>   GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
>   GvR> install IDLE as
>   GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
>   GvR> place a symlink or 2-line bootstrap script in
>   GvR> <prefix>/bin/idle.
> 
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Fine.

>   GvR> The introduction of the extra directory level "Tools" implies
>   GvR> that idle is not importable as a package from regular Python.
>   GvR> This is how I want it -- otherwise I could/would have made IDLE
>   GvR> a package in the Lib/lib-tk directory instead of a directory in
>   GvR> Tools.
> 
> I think the extra Tools directory still implies this, even if it isn't
> in site-packages.

Sure.

>   GvR> I'd gladly accept a contribution of a setup.py file that does
>   GvR> this for me.  (I looked at the distutils manual for a bit but
>   GvR> decided that I'd have to read it cover-to-cover to figure out
>   GvR> whether what I want is even possible...)
> 
> Is there a reason why IDLE has this peculiar installation process?
> Might there be a better way to do it now that we have distutils.  If
> there is, we could put it in PEP 42.

IDLE doesn't have a peculiar installation process, but it may not be
what distutils is currently good at.  IDLE is an application that
needs its files installed somewhere but not on the module search path
for other apps.  I don't know how to do that!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From klm at digicool.com  Wed Nov 15 06:05:10 2000
From: klm at digicool.com (Ken Manheimer)
Date: Wed, 15 Nov 2000 00:05:10 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>
Message-ID: <Pine.LNX.4.21.0011142355310.8955-100000@serenade.digicool.com>

On Tue, 14 Nov 2000, Ken Manheimer wrote:

> In any case, dynamic scoping breaks locality of reference for just about
> any definition of "locality".

(Huh - i realized that "locality of reference" isn't the right phrase, not
sure what is.  I was trying to refer to something along the lines of
modularity - basically, the virtue of lexical scoping is that you can,
eventually, trace the connections by reading the program text.  Dynamic
scoping is not so well behaved - any routine that calls your routine may
contain a variable whose value you're expecting to get from elsewhere.  
It's too promiscuous...)

Ken




From moshez at math.huji.ac.il  Wed Nov 15 08:11:29 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 15 Nov 2000 09:11:29 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011150904510.16783-100000@sundial>

On Tue, 14 Nov 2000, Guido van Rossum wrote:

> > How about
> > <prefix>/lib/python<version>/Tools/idle
> > instead.  I suggest the change to support an until-now-implicit policy
> > that anything in site-packages actually be a package.  If things in
> > site-packages aren't supposed to be packages, it ought to be called
> > something like site-tools <0.2 wink>.
> 
> Fine.

Sorry, just noticed a problem with that:

Say you put idle in .../Tools/idle
And compiler in .../Tools/compiler

Then the fact that some script does

sys.path.insert(0, ".../Tools") means that it would get all the tools
as packages to import, instead of only those it needs (say, for
idle-callers, idle). That means a lot of backwards compat. issues.
Why not have each tool be in things like ..../idle-support/idle/
And .../compiler/compiler-support/compiler, etc.?

Actually, once we do that, why bring the Python path into that at all?
"idle" is a program. The fact it is written in Python is incidental --
it might as well have been written in C, for all most users care. The
FHS policy would seem to indicate that the support files would need
to be in things like /usr/lib and /usr/local/lib
Say, the default installation would put it in /usr/local/lib/idle-support/
and would add a line to sys.path.insert(), and put "idle" (the script)
under autoconf control too.

sorry-for-challenging-the-consensus-again-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From pf at artcom-gmbh.de  Wed Nov 15 08:32:59 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 15 Nov 2000 08:32:59 +0100 (MET)
Subject: Using telnetlib instead of 'popen2' (was: Re: [Python-Dev] Making popen2 actually work)
In-Reply-To: <200011150348.WAA05535@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 14, 2000 10:48:14 pm"
Message-ID: <m13vx3z-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum wrote:
> The trick seems to be to get the subprocess not to buffer its output
> (or input).  There's nothing that Python can do for you here!
> 
> Unfortunately, most programs use stdio for all their console I/O, and
> stdio automatically buffers its output when connected to a pipe... :-(
> 
> If you absolutely, positively have to do this with a program that
> won't unbuffer its output, your only resort would be to use pseudo
> ttys -- but that's quite a different story... :-(

Yes.  But it should be noted here, that sometimes you can avoid the rather
complicated and platform depending handling of pseudo ttys, if you use
the 'telnetlib' module from the Python standard library.  The fine lib
documentation contains a good example ready for cut'n'paste.  Disadvantage:
This only works, if the telnet service is not disabled due to security
reasons.

Regards, Peter



From gward at mems-exchange.org  Wed Nov 15 19:11:30 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Wed, 15 Nov 2000 13:11:30 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <14865.61776.50840.609922@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 14, 2000 at 09:13:36PM -0500
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>
Message-ID: <20001115131130.A22029@ludwig.cnri.reston.va.us>

On 14 November 2000, Jeremy Hylton said:
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Actually, lots of non-packagized modules get installed to site-packages
-- they just wind up right in site-packages, rather than in a
subdirectory of it.

I've always thought this directory was misnamed -- should have been
"site-lib" or something.  Oh well.

For the record, the Distutils don't currently support the "application"
model of installation that Guido wants.  It's on the (unofficial,
notional, blue-sky) to-do list.

        Greg



From nas at arctrix.com  Wed Nov 15 18:35:59 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 15 Nov 2000 09:35:59 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001115093559.A19286@glacier.fnational.com>

[The python-dev list server seems to have gotten hungry and eaten my
original message.  I'm trying again.]


I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil




From moshez at math.huji.ac.il  Thu Nov 16 08:06:14 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 16 Nov 2000 09:06:14 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <20001115131130.A22029@ludwig.cnri.reston.va.us>
Message-ID: <Pine.GSO.4.10.10011160905260.24460-100000@sundial>

On Wed, 15 Nov 2000, Greg Ward wrote:

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.
> 
> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

On a similar note, there's a discussion going on about this in 
debian-python at lists.debian.org (there are archives on the Debian
site)

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov 16 17:26:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 11:26:16 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Wed, 15 Nov 2000 13:11:30 EST."
             <20001115131130.A22029@ludwig.cnri.reston.va.us> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>  
            <20001115131130.A22029@ludwig.cnri.reston.va.us> 
Message-ID: <200011161626.LAA10243@cj20424-a.reston1.va.home.com>

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.

The name doesn't matter that much.  I think this is a fine use of
site-packages.

> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

At least this admission saves me wasting time trying to force
distutils into submssion. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Thu Nov 16 20:44:59 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 14:44:59 -0500
Subject: [Python-Dev] curses demos
Message-ID: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us>

I'd like to add a Demo/curses directory to the CVS tree.  If anyone
has reasonably small programs that use the curses module, please let
me know.

Right now I have two programs: a Life game that I wrote, and Alexei
Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
manager.  I also want to write scripts to demonstrate reading the
mouse, and writing a threaded curses program.

--amk



From guido at python.org  Thu Nov 16 21:03:47 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 15:03:47 -0500
Subject: [Python-Dev] curses demos
In-Reply-To: Your message of "Thu, 16 Nov 2000 14:44:59 EST."
             <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us> 
References: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us> 
Message-ID: <200011162003.PAA14322@cj20424-a.reston1.va.home.com>

> I'd like to add a Demo/curses directory to the CVS tree.  If anyone
> has reasonably small programs that use the curses module, please let
> me know.
> 
> Right now I have two programs: a Life game that I wrote, and Alexei
> Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
> manager.  I also want to write scripts to demonstrate reading the
> mouse, and writing a threaded curses program.

Cool.  Go for it!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Fri Nov 17 01:42:56 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 19:42:56 -0500
Subject: [Python-Dev] coroutines and microthreads
Message-ID: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>

(I would cc this to the Stackless list as well, except that starship
mail is still down it seems.  I *am* posting to the python-coro list,
which seems to be appropriate here.  Will Ware, are you listening?)

Coroutines!
-----------

Inspired by Gordon McMillan's Stackless tutorial
(http://www.mcmillan-inc.com/stackless.html), I've been thinking more
about a coroutine API for Python.

I proposed one before (see the thread "uthread strawman" in
python-dev).  I've been thinking a bit more and am now revising my
proposal a bit.

I'll first give an example.  Tim Peters gave a standard example to
return the fringe of a list (checked in as Demo/threads/fcmp.py).

Using my proposed API this example could be rewritten a bit
cleaner, as follows:

    from coro import coroutine, suspend, EarlyExit  # Provisional module name

    def fringe(L):
	for x in L:
	    if type(x) is type(L):
		fringe(x)
	    else:
		suspend(x)

    def printinorder(L):
	c = coroutine(f, L)
	try:
	    while 1:
		print c(),
	except EarlyExit:
	    pass
	print

    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
    printinorder(x) # prints "0 1 2 3 4 5 6"

This is about as simple as it gets.  It supports both generators (like
the example here) and coroutines (like Tim's Demo/threads/squasher.py
example).

Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
propose a function current() which returns the current coroutine
object.  There should also be a way to kill a coroutine (or at least
to send an exception).  When a coroutine falls through at its end,
*some* other coroutine needs to be resumed.  

I believe this can be implemented with a much simplified stackless
approach, that doesn't cater towards continuations (but still borrows
a lot of wisdom from Christian's Stackless).  It can also be
implemented using threads, which should give some hope for getting the
same API supported in JPython, making it more attractive.  I am hoping
to create an implementation on top of Stackless, just to experiment
with the idiom.

Microthreads?
-------------

I'm much less clear about microthreads (uthreads).  Last time, I
thought this was a fancy name for coroutines.  It isn't!  Microthreads
are "almost real" threads, with round-robin scheduling.  What makes
them attractive to some is the fact that they don't use operating
system resources: there's no OS-level stack or kernel process table
entry associated with them.  This also accounts for their biggest
weakness: when a microthread is suspended for I/O, *all* microthreads
are suspended.  In limited contexts, such as a network server, this
can be solved by an approach similar to that in Gordon's
SelectDispatcher.  (It would be a titanic effort to do this for every
potentially blocking I/O operation, and it probably wouldn't work well
with C extensions.)

I'm not sure what applications require the round-robin scheduling
property of uthreads -- certainly Gordon's application would seem to
be doing just fine without it (I'm not sure if it is being scheduled
round-robin or not).

Proper round-robin scheduling for uthreads requires explicit switching
code in the Python interpreter.  Stackless provides this, at the same
place where in regular Python the global interpreter lock is released
and re-acquired to give other threads a chance to run.  Is this
needed?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gmcm at hypernet.com  Fri Nov 17 03:29:52 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Thu, 16 Nov 2000 21:29:52 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <3A1451D0.12627.17914E75@localhost>

[Guido]
> I'll first give an example.  Tim Peters gave a standard example to
> return the fringe of a list (checked in as Demo/threads/fcmp.py).
> 
> Using my proposed API this example could be rewritten a bit
> cleaner, as follows:
> 
>     from coro import coroutine, suspend, EarlyExit  # Provisional module name
> 
>     def fringe(L):
> 	for x in L:
> 	    if type(x) is type(L):
> 		fringe(x)
> 	    else:
> 		suspend(x)
> 
>     def printinorder(L):
> 	c = coroutine(f, L)
> 	try:
> 	    while 1:
> 		print c(),
> 	except EarlyExit:
> 	    pass
> 	print
> 
>     x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>     printinorder(x) # prints "0 1 2 3 4 5 6"
> 
> This is about as simple as it gets.  It supports both generators (like
> the example here) and coroutines (like Tim's Demo/threads/squasher.py
> example).

This looks very nice and clean. It looks sufficient for the type 
of thing I'm doing (and planning to do), but is it really a full 
coroutine API? That is, doesn't the fact that you always 
suspend to the guy who just activated you make this a 
generator API? (OTOH, if I want A and B to talk to each other 
as "coroutines", is it sufficient to make them both "generators" 
and then glue them together with another routine that just 
swaps results?)
 
> Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> propose a function current() which returns the current coroutine
> object.  There should also be a way to kill a coroutine (or at least
> to send an exception).  When a coroutine falls through at its end,
> *some* other coroutine needs to be resumed.  

Random thoughts:
 - I've found it handy that Christian's stuff lets you grab a 
coroutine as well as return one (that is, either side can 
instigate it). Not sure if that's necessary.
 - What do you mean by "kill a coroutine"? You can't interrupt 
one, so isn't it sufficient that when it goes out of scope it gets 
GC'd somehow?
 - It appears from your example that falling off the end 
automatically raises an EarlyExit. I think I see more 
arguments for that than against it :-).
 
> Microthreads?
> -------------
> 
> I'm much less clear about microthreads (uthreads).  Last time, I
> thought this was a fancy name for coroutines.  It isn't!  Microthreads
> are "almost real" threads, with round-robin scheduling.  What makes
> them attractive to some is the fact that they don't use operating
> system resources: there's no OS-level stack or kernel process table
> entry associated with them.  This also accounts for their biggest
> weakness: when a microthread is suspended for I/O, *all* microthreads
> are suspended.  In limited contexts, such as a network server, this
> can be solved by an approach similar to that in Gordon's
> SelectDispatcher.  (It would be a titanic effort to do this for every
> potentially blocking I/O operation, and it probably wouldn't work well
> with C extensions.)

Using the raw Win32 API, I think you could come pretty close. 
I've been wondering if it's possible to do something that would 
get Cameron to quit raving about Tcl's event loop ;-).
 
> I'm not sure what applications require the round-robin scheduling
> property of uthreads -- certainly Gordon's application would seem to
> be doing just fine without it (I'm not sure if it is being scheduled
> round-robin or not).

Each coroutine (or whatever they are) runs until it calls one of 
the SelectDispatcher methods that suspends it.  The socket 
methods suspend it until select says the socket is ready; 
yield suspends it till the next time round the select loop 
(which has a timeout). So piggish routines are encouraged to 
yield once in a while.
 
(While I'm doing things the other way 'round, I believe I could 
use your API without changing SelectDispatcher's API at all.)

> Proper round-robin scheduling for uthreads requires explicit switching
> code in the Python interpreter.  Stackless provides this, at the same
> place where in regular Python the global interpreter lock is released
> and re-acquired to give other threads a chance to run.  Is this
> needed?

While I'm not really a uthread user, I think they would give you 
an unqualified "yes". The advantage to explicitly yeilding is 
that (with proper thought) you don't need nasty things like 
locks; the disadvantage (as demonstrated by a particular OS 
with a rabidly fanatical following) is that one jerk can ruin it for 
everybody.

- Gordon



From guido at python.org  Fri Nov 17 04:15:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 22:15:06 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: Your message of "Thu, 16 Nov 2000 21:29:52 EST."
             <3A1451D0.12627.17914E75@localhost> 
References: <3A1451D0.12627.17914E75@localhost> 
Message-ID: <200011170315.WAA21528@cj20424-a.reston1.va.home.com>

> > This is about as simple as it gets.  It supports both generators (like
> > the example here) and coroutines (like Tim's Demo/threads/squasher.py
> > example).
> 
> This looks very nice and clean. It looks sufficient for the type 
> of thing I'm doing (and planning to do), but is it really a full 
> coroutine API? That is, doesn't the fact that you always 
> suspend to the guy who just activated you make this a 
> generator API? (OTOH, if I want A and B to talk to each other 
> as "coroutines", is it sufficient to make them both "generators" 
> and then glue them together with another routine that just 
> swaps results?)

Yes, it is a full coroutine API -- you don't have to use suspend() at
all.  Appended is my version of squasher.py, an almost mechanical
translation from Tim's version (co.tran(cofoobar, arg) => cofoobar(arg)).
That's built out of a bunch of real coroutines with only explicit
transfers!

> > Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> > propose a function current() which returns the current coroutine
> > object.  There should also be a way to kill a coroutine (or at least
> > to send an exception).  When a coroutine falls through at its end,
> > *some* other coroutine needs to be resumed.  
> 
> Random thoughts:
>  - I've found it handy that Christian's stuff lets you grab a 
> coroutine as well as return one (that is, either side can 
> instigate it). Not sure if that's necessary.

Sorry, not sure what this means.

>  - What do you mean by "kill a coroutine"? You can't interrupt 
> one, so isn't it sufficient that when it goes out of scope it gets 
> GC'd somehow?

That's one solution.  You can't interrupt threads either, and most
people are happy with that (although there's a request for thread
interrupts in PEP 42).

>  - It appears from your example that falling off the end 
> automatically raises an EarlyExit. I think I see more 
> arguments for that than against it :-).

Yes, this follows Tim's lead.  What would be an argument against it?

> > Microthreads?
> > -------------
> > 
> > I'm much less clear about microthreads (uthreads).  Last time, I
> > thought this was a fancy name for coroutines.  It isn't!  Microthreads
> > are "almost real" threads, with round-robin scheduling.  What makes
> > them attractive to some is the fact that they don't use operating
> > system resources: there's no OS-level stack or kernel process table
> > entry associated with them.  This also accounts for their biggest
> > weakness: when a microthread is suspended for I/O, *all* microthreads
> > are suspended.  In limited contexts, such as a network server, this
> > can be solved by an approach similar to that in Gordon's
> > SelectDispatcher.  (It would be a titanic effort to do this for every
> > potentially blocking I/O operation, and it probably wouldn't work well
> > with C extensions.)
> 
> Using the raw Win32 API, I think you could come pretty close. 

You can do it on Unix too, by trapping open() to always set files in
nonblocking mode.  It's just a lot of work because there are so many
calls to trap (cf. your dispatcher, which just traps read, write,
accept, connect).  Long ago, there was a user-mode thread package on
Solaris that did this; it's still immortalized in Python/thread_lwp.h
although I doubt that it still works.  GNU Pth uses the same approach
for user-mode threads.  Hm...  This makes me wonder...  Maybe instead
of uthreads we could promote Python linked with Pth?

> I've been wondering if it's possible to do something that would 
> get Cameron to quit raving about Tcl's event loop ;-).

Cameron is now lobbying for a separation between Tcl and Tk.  I guess
that will mean that the event loop will have to be moved back to Tk,
where it started many years ago. :-)

> > I'm not sure what applications require the round-robin scheduling
> > property of uthreads -- certainly Gordon's application would seem to
> > be doing just fine without it (I'm not sure if it is being scheduled
> > round-robin or not).
> 
> Each coroutine (or whatever they are) runs until it calls one of 
> the SelectDispatcher methods that suspends it.  The socket 
> methods suspend it until select says the socket is ready; 
> yield suspends it till the next time round the select loop 
> (which has a timeout). So piggish routines are encouraged to 
> yield once in a while.

That's what I figured.

> (While I'm doing things the other way 'round, I believe I could 
> use your API without changing SelectDispatcher's API at all.)

I think so too.  Maybe that'll be my next exercise.  I've already done
8 queens with coroutines (slower than recursion, alas -- should be
faster when coroutines are implemented in C though because you save on
number of function calls).  I've also made tho working implementations of
my proposed coro API: one using threads, one using Stackless and the
continuation module.  (Found a few quirks in the examples in your
tutorial.  Interested in details?)

> > Proper round-robin scheduling for uthreads requires explicit switching
> > code in the Python interpreter.  Stackless provides this, at the same
> > place where in regular Python the global interpreter lock is released
> > and re-acquired to give other threads a chance to run.  Is this
> > needed?
> 
> While I'm not really a uthread user, I think they would give you 
> an unqualified "yes". The advantage to explicitly yeilding is 
> that (with proper thought) you don't need nasty things like 
> locks; the disadvantage (as demonstrated by a particular OS 
> with a rabidly fanatical following) is that one jerk can ruin it for 
> everybody.

Let me guess -- you don't like the Mac much, do you? :-)

And yes, I noticed that the first thing the uthread API does is define
a bunch of synchronization primitives. :-(  I think it wouldn't be
hard to implement the round-robin scheduling, but it's yet more added
complexity...

BTW, how's PEP 219 coming along? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

# Coroutine example:  general coroutine transfers
# (After Tim Peters)
#
# The program is a variation of a Simula 67 program due to Dahl & Hoare,
# (Dahl/Dijkstra/Hoare, Structured Programming; Academic Press, 1972)
# who in turn credit the original example to Conway.
#
# We have a number of input lines, terminated by a 0 byte.  The problem
# is to squash them together into output lines containing 72 characters
# each.  A semicolon must be added between input lines.  Runs of blanks
# and tabs in input lines must be squashed into single blanks.
# Occurrences of "**" in input lines must be replaced by "^".
#
# Here's a test case:

test = """\
   d    =   sqrt(b**2  -  4*a*c)
twoa    =   2*a
   L    =   -b/twoa
   R    =   d/twoa
  A1    =   L + R
  A2    =   L - R\0
"""

# The program should print:

# d = sqrt(b^2 - 4*a*c);twoa = 2*a; L = -b/twoa; R = d/twoa; A1 = L + R;
#A2 = L - R
#done

# getline: delivers the next input line to its invoker
# disassembler: grabs input lines from getline, and delivers them one
#    character at a time to squasher, also inserting a semicolon into
#    the stream between lines
# squasher:  grabs characters from disassembler and passes them on to
#    assembler, first replacing "**" with "^" and squashing runs of
#    whitespace
# assembler: grabs characters from squasher and packs them into lines
#    with 72 character each, delivering each such line to putline;
#    when it sees a null byte, passes the last line to putline and
#    then kills all the coroutines
# putline: grabs lines from assembler, and just prints them

from coro import coroutine, suspend, main, EarlyExit

def getline(text):
    for line in string.splitfields(text, '\n'):
        codisassembler(line)

def disassembler():
    while 1:
        card = cogetline()
        for i in range(len(card)):
            cosquasher(card[i])
        cosquasher(';')

def squasher():
    while 1:
        ch = codisassembler()
        if ch == '*':
            ch2 = codisassembler()
            if ch2 == '*':
                ch = '^'
            else:
                coassembler(ch)
                ch = ch2
        if ch in ' \t':
            while 1:
                ch2 = codisassembler()
                if ch2 not in ' \t':
                    break
            coassembler(' ')
            ch = ch2
        coassembler(ch)

def assembler():
    line = ''
    while 1:
        ch = cosquasher()
        if ch == '\0':
            break
        if len(line) == 72:
            coputline(line)
            line = ''
        line = line + ch
    line = line + ' ' * (72 - len(line))
    coputline(line)

def putline():
    try:
        while 1:
            line = coassembler()
            print line
    except EarlyExit:
        main() # Hack -- resume main coroutine to exit

import string
cogetline = coroutine(getline, test)
coputline = coroutine(putline)
coassembler = coroutine(assembler)
codisassembler = coroutine(disassembler)
cosquasher = coroutine(squasher)

coputline()
print "done"

raw_input("Exit?")

# end of example



From akuchlin at mems-exchange.org  Fri Nov 17 05:17:15 2000
From: akuchlin at mems-exchange.org (A.M. Kuchling)
Date: Thu, 16 Nov 2000 23:17:15 -0500
Subject: [Python-Dev] Python/Distutil draft PEP
Message-ID: <200011170417.XAA00509@207-172-146-153.s153.tnt3.ann.va.dialup.rcn.com>

PEP: XXX
Title: Using Distutils to Build Python
Version: $Revision:  $
Author: akuchlin at mems-exchange.org (A.M. Kuchling)
Status: Draft
Type: Standards
Created: 16-Nov-2000

Introduction

    The Modules/Setup mechanism has some flaws:

    * People have to remember to uncomment bits of Modules/Setup in
      order to get all the possible modules.

    * Moving Setup to a new version of Python is tedious; new modules
      have been added, so you can't just copy the older
      version, but have to reconcile the two versions.

    * Users have to figure out where the needed libraries, such as
      zlib, are installed.


Proposal

    Use the Distutils to build the modules that come with Python.

    The changes can be broken up into several pieces:

    1. The Distutils needs some Python modules to be able to build
       modules.  Currently I believe the minimal list is posix, _sre,
       and string.

       These modules will have to be built before the Distutils can be
       used, so they'll simply be hardwired into Modules/Makefile and 
       be automatically built.  

    2. A top-level setup.py script will be written that checks the
       libraries installed on the system and compiles 
       as many modules as possible.       

    3. Modules/Setup will be kept and settings in it will override
       setup.py's usual behaviour, so you can disable a module known
       to be buggy, or specify particular compilation or linker flags.
       However, in the common case where setup.py works correctly, 
       everything in Setup will remain commented out.  The other
       Setup.*  become unnecessary, since nothing will be generating 
       Setup automatically.


Unresolved Issues
 
    Do we need to make it possible to disable the 3 hard-wired modules
    without manually hacking the Makefiles?  If yes, perhaps a
    configure switch is sufficient.

    The Distutils always compile modules as shared libraries.  How do
    we support compiling them statically into the resulting Python
    binary?
   
    makesetup and the other contents of $(LIBPL)/config need to be
    preserved for compatibility with existing modules; for how many
    versions do we need to keep them around?


Copyright

    This document has been placed in the public domain.



From moshez at zadka.site.co.il  Fri Nov 17 15:04:53 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 16:04:53 +0200
Subject: [Python-Dev] PEP-228 (fw)
Message-ID: <E13wm8M-00085z-00@darjeeling.zadka.site.co.il>

I'm forwarding this message about PEP-228, but I haven't fully
understood it. In any case, since PEP-228 is apparently not
a 2.1 candidate, I've told Michael that his remarks will be considered,
but they are not relevant yet.

------- Forwarded Message

From: Michael McLay <mclay at nist.gov>
Date: Fri, 17 Nov 2000 04:20:45 -0500 (EST)
To: pep at zadka.site.co.il
Subject: PEP:228

I've dropped off the Python mailing lists for the last couple years
because the volume is too high and I don't have time to contribute to
the discussion.  I do read the summaries and PEP:228 was mentioned in
reference to the 2.1 release.  I'd like to see this problem solve in
the 2.1 release and I have a few thoughts on the matter that may be of
use to this task.  Is there a mailing list specific to PEP:228 or
would I have to join python-dev to participate in the discussion?
Please feel free to forward my message to the python-dev list or use
my comments in PEP:228.

Python's use in the XML community is growing and broad-based XML
support in Python may be key to widespread adoption of Python.  There
is some support for XML Schema, the XML Schema Validation software is
written in Python, but there is no direct support in the core Python
XML library.  One problem area for an XML Schema implementation is
related to the numerical types in Python.  The ability to implement
XML Schema types could be improved if the requirements are considered
when creating the PEP:228 implementation.

And from the Python language perspective a review of the numerical
model used in XML Schema may be useful to the discussion of the
revised numerical model for Python.  The authors of the XML Schema
specification put a good deal of effort into defining a modern,
portable numerical model and I think it might be a good fit with the
Python object model.  I might also be possible that the XML Schema
type and entity definition mechanism would be helpful in the
discussion of defining type interfaces for Python.

A table of simple types built into XML Schema are defined in:

   http://www.w3.org/TR/xmlschema-0/#simpleTypesTable

These are defined more fully in "XML Schema Part 2: DataTypes" at:

    http://www.w3.org/TR/2000/CR-xmlschema-2-20001024/

This includes a discussion of the difference between built-inprimitive
and built-inderived datatypes.

The derived types are defined in the XML Schema at: 

    http://www.w3.org/2000/10/datatypes.xsd

In this document, for instance, an integer is derived from a decimal
type in XML Schema. 

 <simpleType name="integer">
   <restriction base="decimal">
      <scale value="0" fixed="true"/>
   </restriction>
 </simpleType>

The use of XML Schema in conjunction with Python would be very helpful
to a soon to be large domain of applications.  I am working with a
group that is currently defining 20 standards using XML Schema and
this is probably just the tip of the iceburg.  I need to be able to
define constrained numerical types as defined in XML Schma.  In Python
this might look like:

  size = ConstrainedDecimal(initial=2, minInclusive=2, 
				       maxInclusive=3000, scale=0)

  for i in range(3000-2):
    size += 1

  print size

The  ConstrainedDecimal would define a restriction range on the value
of "size".  





------- End of Forwarded Message

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From just at letterror.com  Fri Nov 17 09:55:48 2000
From: just at letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 09:55:48 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <l03102803b63a9e816862@[193.78.237.158]>

At 7:42 PM -0500 16-11-2000, Guido van Rossum wrote:
[ snipp ]
>Using my proposed API this example could be rewritten a bit
>cleaner, as follows:
>
>    from coro import coroutine, suspend, EarlyExit  # Provisional module name
>
>    def fringe(L):
>	for x in L:
>	    if type(x) is type(L):
>		fringe(x)
>	    else:
>		suspend(x)
>
>    def printinorder(L):
>	c = coroutine(f, L)
>	try:
>	    while 1:
>		print c(),
>	except EarlyExit:
>	    pass
>	print
>
>    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>    printinorder(x) # prints "0 1 2 3 4 5 6"
>
>This is about as simple as it gets.  It supports both generators (like
>the example here) and coroutines (like Tim's Demo/threads/squasher.py
>example).

Neat! (It's _very_ close to what I proposed myself, so how can I not like
it? ;-)

One question: I assume values can also be passed when explicitly resuming a
coroutine. However, the _first_ time a coroutine is "resumed" (started,
really), the stored init arguments get passed to the function. What to do
with a value passed to the resume call? Two possible solutions:
1) don't not allow passing a value the first time
2) add the value(s) to the init arguments

As code:

  def fribble(initarg):
     ...

  c = coroutine(fribble, initarg)
  c(x)  # <-- where does x go? or is it not allowed?

>Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
>propose a function current() which returns the current coroutine
>object.

Starts looking more and more like what I proposed myself...

>There should also be a way to kill a coroutine (or at least
>to send an exception).  When a coroutine falls through at its end,
>*some* other coroutine needs to be resumed.

Here's what I did in my own strawman:
- resume the guy that resumed me, if any (making it equivalent to suspend())
- if we were resumed by a suspend() call (which in my proposal means we
don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
resume the "main" coroutine, which is the (non-coro) code that started the
first coroutine.

Maybe coroutines should have a kill() method, which would post the
EarlyExit exception. Maybe even a postException() method (like uthreads
have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

>I believe this can be implemented with a much simplified stackless
>approach, that doesn't cater towards continuations (but still borrows
>a lot of wisdom from Christian's Stackless).  It can also be
>implemented using threads, which should give some hope for getting the
>same API supported in JPython, making it more attractive.  I am hoping
>to create an implementation on top of Stackless, just to experiment
>with the idiom.

Here's my own code, based on stackless 1.1:
   http://www.petr.com/just/corojust.tar.gz
or
   http://www.petr.com/just/corojust/
for the individual files.

Apart from the initialisation it's pretty much what you propose.

Just





From guido at python.org  Fri Nov 17 13:51:09 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 07:51:09 -0500
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."
             <l03102803b63a9e816862@[193.78.237.158]> 
References: <l03102803b63a9e816862@[193.78.237.158]> 
Message-ID: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>

(A typically brotherly dialog. :-)

[Guido]
> >This is about as simple as it gets.  It supports both generators (like
> >the example here) and coroutines (like Tim's Demo/threads/squasher.py
> >example).

[Just]
> Neat! (It's _very_ close to what I proposed myself, so how can I not like
> it? ;-)

I looked at what you did, and probably a dozen other examples.

> One question: I assume values can also be passed when explicitly resuming a
> coroutine. However, the _first_ time a coroutine is "resumed" (started,
> really), the stored init arguments get passed to the function. What to do
> with a value passed to the resume call? Two possible solutions:
> 1) don't not allow passing a value the first time
> 2) add the value(s) to the init arguments
> 
> As code:
> 
>   def fribble(initarg):
>      ...
> 
>   c = coroutine(fribble, initarg)
>   c(x)  # <-- where does x go? or is it not allowed?

According to Tim, it's a standard issue that every coroutine API has
to deal with.  I lay awake worrying about this for a long time.  I
read the thread about this in the python-coro list.  I even started
thinking about a very different API, that would disinguish between
get() and put() operations.  But there was nothing that satisfied all
constraints.  My current solution: for the initial transfer, *no
argument is allowed*.  (Except None. :-)  I've looked at several
examples (8 queens, squasher, fringe) and they all work fine under
this assumption.

> >Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> >propose a function current() which returns the current coroutine
> >object.
> 
> Starts looking more and more like what I proposed myself...

Great minds and all that... :-)

I've been thinking about names a bit more, and 'main' is a truly lousy
name.  I now propose a function initial() to get at the main
coroutine.  (A function out of symmetry with current(), but also
because in an environment that also supports OS-level threads, there
is a different main coroutine per OS thread.)

> >There should also be a way to kill a coroutine (or at least
> >to send an exception).  When a coroutine falls through at its end,
> >*some* other coroutine needs to be resumed.
> 
> Here's what I did in my own strawman:
> - resume the guy that resumed me, if any (making it equivalent to suspend())

But there could be a cycle here!  What if A resumed B resumed C
resumed A?

> - if we were resumed by a suspend() call (which in my proposal means we
> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
> resume the "main" coroutine, which is the (non-coro) code that started the
> first coroutine.

But the main coroutine may not be expecting this -- e.g. in the
fringe() example, the main coroutine is expecting the next fringe
item or an EarlyExit exception.

> Maybe coroutines should have a kill() method, which would post the
> EarlyExit exception. Maybe even a postException() method (like uthreads
> have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

Indeed.  This needs a bit more thinking (which so far I've
successfully managed to avoid :-).

> >I believe this can be implemented with a much simplified stackless
> >approach, that doesn't cater towards continuations (but still borrows
> >a lot of wisdom from Christian's Stackless).  It can also be
> >implemented using threads, which should give some hope for getting the
> >same API supported in JPython, making it more attractive.  I am hoping
> >to create an implementation on top of Stackless, just to experiment
> >with the idiom.
> 
> Here's my own code, based on stackless 1.1:
>    http://www.petr.com/just/corojust.tar.gz
> or
>    http://www.petr.com/just/corojust/
> for the individual files.
> 
> Apart from the initialisation it's pretty much what you propose.

I saw that, and even peeked a bit at it.  It seems you must have
struggled a bit with the initialization sequence in the squasher
example, gmcm_coro.py (which BTW is due to Tim Peters in its current
-- Python -- form).  That's alright -- coroutine initialization *is* a
problem.

I've found that finalization is a problem, too: I'm not at all sure
that my version of that example correctly frees all the coroutines
when it is done, since several of them are still suspended (waiting
for more input) when the decision to exit is made.  Also, I had to
invent the 'main' API in order to make it work.  Tim's original
(Demo/threads/squasher.py in the CVS tree) has a kill operation that
makes this easier.  His kill() kills the entire *group* of related
coroutines -- a concept that we don't have and probably don't need: it
just makes using the API more complicated, and Tim has said he'd do
away with it if he had to do it again.

BTW, I've got an almost-working implementation of the coro API using
Stackless continuations!  Problem: I don't know how to raise an
exception for a continuation -- which I need to implement EarlyExit.
Can anyone help?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From just at letterror.com  Fri Nov 17 14:44:51 2000
From: just at letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 14:44:51 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."            
 <l03102803b63a9e816862@[193.78.237.158]>
 <l03102803b63a9e816862@[193.78.237.158]>
Message-ID: <l03102807b63ae45cd9f0@[193.78.237.158]>

At 7:51 AM -0500 17-11-2000, Guido van Rossum wrote:
[ initialisation ]
>My current solution: for the initial transfer, *no
>argument is allowed*.

In retrospect, that indeed seems the best solution. I wonder why we (as in
the python-coro at egroups clan) didn't think of that before... Glad to have
you here ;-)


[ "main" coro ]
>I've been thinking about names a bit more, and 'main' is a truly lousy
>name.  I now propose a function initial() to get at the main
>coroutine.  (A function out of symmetry with current(), but also
>because in an environment that also supports OS-level threads, there
>is a different main coroutine per OS thread.)

Fine with me.

>> Here's what I did in my own strawman:
>> - resume the guy that resumed me, if any (making it equivalent to suspend())
>
>But there could be a cycle here!  What if A resumed B resumed C
>resumed A?

Here's what I did: if A does suspend(), it resumes C, but at the same time
A _loses_ it's "resumer". So if it does do a suspend() again, it'll resume
the "initial" coro.

>> - if we were resumed by a suspend() call (which in my proposal means we
>> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
>> resume the "main" coroutine, which is the (non-coro) code that started the
>> first coroutine.
>
>But the main coroutine may not be expecting this -- e.g. in the
>fringe() example, the main coroutine is expecting the next fringe
>item or an EarlyExit exception.

Sure, but you need to be able to go _somewhere_, and this seems more useful
than, say, terminating the app ;-)

[ my own implementiation ]
>> Apart from the initialisation it's pretty much what you propose.
>
>I saw that, and even peeked a bit at it.  It seems you must have
>struggled a bit with the initialization sequence in the squasher
>example, gmcm_coro.py (which BTW is due to Tim Peters in its current
>-- Python -- form).  That's alright -- coroutine initialization *is* a
>problem.

Still, the first-time-no-arguments-allowed rule solves it nicely.

>I've found that finalization is a problem, too: I'm not at all sure
>that my version of that example correctly frees all the coroutines
>when it is done, since several of them are still suspended (waiting
>for more input) when the decision to exit is made.

Are you talking of freeing all data associated with the coroutine, or about
properly finalizing it (so that any finally clauses get executed)? If the
former: don't even try, it's a nightmare with continuations. If the latter:
good question ;-).

>BTW, I've got an almost-working implementation of the coro API using
>Stackless continuations!  Problem: I don't know how to raise an
>exception for a continuation -- which I need to implement EarlyExit.
>Can anyone help?

def hellraiser():
    continuation.return_current()
    raise <whatever>

k = hellraiser()
k.caller = your_co
k()

Just





From tismer at tismer.com  Fri Nov 17 14:47:53 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 15:47:53 +0200
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."             <l03102803b63a9e816862@[193.78.237.158]> <l03102803b63a9e816862@[193.78.237.158]> <l03102807b63ae45cd9f0@[193.78.237.158]>
Message-ID: <3A153709.A56845AC@tismer.com>


Just van Rossum wrote:
...
> >BTW, I've got an almost-working implementation of the coro API using
> >Stackless continuations!  Problem: I don't know how to raise an
> >exception for a continuation -- which I need to implement EarlyExit.
> >Can anyone help?
> 
> def hellraiser():
>     continuation.return_current()
>     raise <whatever>
> 
> k = hellraiser()
> k.caller = your_co
> k()
> 
> Just

Hey, there is it again, the little trick. I love it :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From thomas.heller at ion-tof.com  Fri Nov 17 17:21:50 2000
From: thomas.heller at ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 17:21:50 +0100
Subject: [Python-Dev] imputil
Message-ID: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>

imputil, which is now an official part of
python, changes the semantics of sys.path
as soon as importers are installed.

On the other hand, a quick and dirty search
finds at least these files in the standard library
where it is assumed that sys.path is a list of strings:

linecache.py, profile.py, pdb.py, pyclbr.py

Should imputil be fixed to install the ImportManager
in a different way (not installing the importers
into sys.path), or should the library be fixed?

Thomas





From jeremy at alum.mit.edu  Fri Nov 17 05:31:23 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 16 Nov 2000 23:31:23 -0500 (EST)
Subject: [Python-Dev] screwup on the patches & bugs notifications
Message-ID: <14868.46235.22940.869048@bitdiddle.concentric.net>

I hurried to get notifications of outstanding patches and bugs last
night and botched both of them.  Sorry about that.  If you can a
message about patches, you can disregard it entirely.  It was made from
an old snapshot of the SF database.  The bug mail contains current
bugs, but the formatting of the mail headers was off.  I've fixed that
for the future.

Jeremy



From guido at python.org  Fri Nov 17 17:35:09 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 11:35:09 -0500
Subject: [Python-Dev] imputil
In-Reply-To: Your message of "Fri, 17 Nov 2000 17:21:50 +0100."
             <038701c050b2$7dd00740$e000a8c0@thomasnotebook> 
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook> 
Message-ID: <200011171635.LAA26120@cj20424-a.reston1.va.home.com>

> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

Sorry to burst your bubble.  Imputil is still experimental.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Fri Nov 17 17:45:58 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 17:45:58 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>
Message-ID: <3A1560C6.B872D948@lemburg.com>

Thomas Heller wrote:
> 
> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

My understanding was that Importers need to provide
a __str__ method which is then used... haven't looked
at imputil.py in ages though, so I can't really comment.

Perhaps imputil.py should leave sys.path alone (or maybe just
wipe it from unneeded entries) and use a new sys.importers
object for the import management ?!

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fdrake at acm.org  Fri Nov 17 17:58:47 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri, 17 Nov 2000 11:58:47 -0500 (EST)
Subject: [Python-Dev] forwarded message from nimoy
Message-ID: <14869.25543.871007.185471@cj42289-a.reston1.va.home.com>


  It sounds like people are happy with Python.  ;)  Just thought I'd
pass this bit of praise araound, since it seems to apply to everyone's
efforts.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations

-------------- next part --------------
An embedded message was scrubbed...
From: "nimoy" <jtnimoy at ucla.edu>
Subject: feedback
Date: Thu, 16 Nov 2000 21:26:26 -0800
Size: 2337
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001117/f3c6904b/attachment.eml>

From thomas.heller at ion-tof.com  Fri Nov 17 18:24:17 2000
From: thomas.heller at ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 18:24:17 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook> <3A1560C6.B872D948@lemburg.com>
Message-ID: <04a501c050bb$372df8c0$e000a8c0@thomasnotebook>

> Thomas Heller wrote:
> > 
> > imputil, which is now an official part of
> > python, changes the semantics of sys.path
> > as soon as importers are installed.
> > 
> > On the other hand, a quick and dirty search
> > finds at least these files in the standard library
> > where it is assumed that sys.path is a list of strings:
> > 
> > linecache.py, profile.py, pdb.py, pyclbr.py
> > 
> > Should imputil be fixed to install the ImportManager
> > in a different way (not installing the importers
> > into sys.path), or should the library be fixed?
[MAL]
> 
> My understanding was that Importers need to provide
> a __str__ method which is then used... haven't looked
> at imputil.py in ages though, so I can't really comment.
> 
No importer currently does define __str__, neither is it used
in the standard libary. The pattern is (in linecache):

        for dirname in sys.path:
            fullname = os.path.join(dirname, basename)

> Perhaps imputil.py should leave sys.path alone (or maybe just
> wipe it from unneeded entries) and use a new sys.importers
> object for the import management ?!
This sounds better.

Thomas





From mal at lemburg.com  Fri Nov 17 18:13:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 18:13:42 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
Message-ID: <3A156746.D2B59356@lemburg.com>

Hi,

sorry to bother you again with this, but the current state
of Python's community elements is in such a bad shape, that
I'm getting more and more frustrated...

First, starship is taken off the net due to a mailing problem.
After a few days starship resurfaces: but apparently using
some really old backup which didn't even contain all files
-- at least the .ssh dirs seem to be lost causing RSA logins
using SSH to fail (don't know about normal logins -- my old
passwords don't work anymore either, but that could have been
caused by the harddisk crash earlier this year).

As consequence, I moved all my Python Pages to a new site
which should provide more stability. Next, I tried to get this
message across to the comp.lang.python.* crowd: I posted
messages to the python.org mailing lists which should have
been gatewayed to the newsgroups -- nothing happened. The
messages seem to be stuck somewhere half-way across the 
Atlantic.

The result: even though I tried to do some good, by moving my
site, noone will notice and I still get complaints about
my extensions not working with 2.0 -- the announcement about
the new versions *did* make it to comp.lang.python and the
announcement list: but it still had the starship URL on it,
so now people find the announcement and there's no way
to point them to the new URL (I can't login to starship to
post a notice there...).

Help someone, please.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From akuchlin at mems-exchange.org  Fri Nov 17 18:45:07 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 12:45:07 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <3A156746.D2B59356@lemburg.com>; from mal@lemburg.com on Fri, Nov 17, 2000 at 06:13:42PM +0100
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <20001117124507.G25735@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 06:13:42PM +0100, M.-A. Lemburg wrote:
>As consequence, I moved all my Python Pages to a new site
>which should provide more stability. Next, I tried to get this

Frankly I think this is the right move, and my pages are now at amk.ca
for pretty much the same reason.  

>message across to the comp.lang.python.* crowd: I posted
>messages to the python.org mailing lists which should have
>been gatewayed to the newsgroups -- nothing happened. The

News/mail gatewaying seems fairly seriously broken at the moment; I've
noticed many followups to posts that I've never seen.  I read c.l.p on
news.cnri.reston.va.us, presumably the same machine used for NNTP
posting by python.org's Mailman, which would imply that Mailman is at
fault.  

--amk



From tismer at tismer.com  Fri Nov 17 17:59:57 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 18:59:57 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A15640D.C2412AB1@tismer.com>

Marc,

"M.-A. Lemburg" wrote:

[justified complaints about Starship]

I'd like to apologize for this mess, but I'd be happier
if I were guilty. Instead, I'm unable to do anything.
Since the Starship has begun to move
again, it got out of my control. I still don't have root
access to the new machine via ssh. This might be by chance,
at least I hope so!
This situation is absolutely what I do not want.
If things don't improve quickly, I might consider
to move the domain to a better server. Moving to
BeOpen should have been a good thing. Finally it
wasn't, but nobody could predict this. Then Starship
went into some Zombie state and was unaccessible
when the DC site was built. Meanwhile the old ship
re-appeared under a different IP, and Jim Tittsler
offered to do a transfer of the current data, but he
also seems to have no super user for the new site.

BTW, you might have ssh access via port 10022, at least
for me this works just fine:

ssh -v -l pirx -p 10022 starship.python.net

Marc, you are the first Starship user! If you leave,
I consider the project dead. Please don't.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From barry at digicool.com  Fri Nov 17 18:59:36 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 12:59:36 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
	<20001117124507.G25735@kronos.cnri.reston.va.us>
Message-ID: <14869.29192.697233.746198@anthem.concentric.net>

>>>>> "AK" == Andrew Kuchling <akuchlin at mems-exchange.org> writes:

    AK> News/mail gatewaying seems fairly seriously broken at the
    AK> moment; I've noticed many followups to posts that I've never
    AK> seen.  I read c.l.p on news.cnri.reston.va.us, presumably the
    AK> same machine used for NNTP posting by python.org's Mailman,
    AK> which would imply that Mailman is at fault.

Not necessarily.

First, there's certainly nothing in the Mailman log files indicating
any problems writing to nntp.  Second, Mailman on python.org actually
gates messages to ournews.cnri.reston.va.us which is an internal
machine on the CNRI network, and should be one hop downstream from
news.cnri.reston.va.us (a.k.a. news.alterdial.uu.net).

My prediction is that the news is piling up on ournews and there are
problems feeding those messages upstream to alterdial.  This is not a
rare occurance unfortunately.  A while back I contacted the CNRI
admins about this, but I don't remember hearing a positive resolution.

I'm reluctant to point python-list's newserver directly at the
alterdial machine because I remember there were worse problems with
that approach (like postings actually getting lost as opposed to just
sitting behind a log jam -- ournews never expires c.l.py.* postings).

Andrew, maybe you can ping the folks at CNRI more directly to at least
make sure that there are no problems between ournews<->news?

-Barry



From akuchlin at mems-exchange.org  Fri Nov 17 19:08:56 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 13:08:56 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <14869.29192.697233.746198@anthem.concentric.net>; from barry@digicool.com on Fri, Nov 17, 2000 at 12:59:36PM -0500
References: <3A156746.D2B59356@lemburg.com> <20001117124507.G25735@kronos.cnri.reston.va.us> <14869.29192.697233.746198@anthem.concentric.net>
Message-ID: <20001117130856.C7763@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 12:59:36PM -0500, Barry A. Warsaw wrote:
>My prediction is that the news is piling up on ournews and there are
>problems feeding those messages upstream to alterdial.  This is not a

And I believe you're correct; pointing a newsreader at ournews, I can
see postings (such as MAL's "ANN: New URL for the mx-Extensions") that
I never saw when reading news on uunet's server, so this is another
example of CNRI's incompetent sysadmining.  I've sent a note off
asking them to look into it.

--amk



From barry at digicool.com  Fri Nov 17 19:10:03 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 13:10:03 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
	<3A15640D.C2412AB1@tismer.com>
Message-ID: <14869.29819.597320.836410@anthem.concentric.net>

>>>>> "CT" == Christian Tismer <tismer at tismer.com> writes:

    CT> I still don't have root access to the new machine via
    CT> ssh. This might be by chance, at least I hope so!

Gotta be, because /nobody/ had root access via ssh until moments ago
(I have root access to the underlying machine -- outside the chroot
jail).  I just cat'd my authorized_keys file, as well as pirx's and
jwt's to starship's root account, so Chris, Jim Tittsler and I at
least should be able to get in as root at starship.python.net.

This should let Jim do any necessary unpacking of transferred data.

-Barry



From tismer at tismer.com  Fri Nov 17 18:19:07 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 19:19:07 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
		<3A15640D.C2412AB1@tismer.com> <14869.29819.597320.836410@anthem.concentric.net>
Message-ID: <3A15688B.B69C7878@tismer.com>


"Barry A. Warsaw" wrote:
> 
> >>>>> "CT" == Christian Tismer <tismer at tismer.com> writes:
> 
>     CT> I still don't have root access to the new machine via
>     CT> ssh. This might be by chance, at least I hope so!
> 
> Gotta be, because /nobody/ had root access via ssh until moments ago
> (I have root access to the underlying machine -- outside the chroot
> jail).  I just cat'd my authorized_keys file, as well as pirx's and
> jwt's to starship's root account, so Chris, Jim Tittsler and I at
> least should be able to get in as root at starship.python.net.
> 
> This should let Jim do any necessary unpacking of transferred data.

Ahhh, puuuh, that's great. :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From mal at lemburg.com  Fri Nov 17 20:55:13 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 20:55:13 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com> <3A15640D.C2412AB1@tismer.com>
Message-ID: <3A158D21.134A44EE@lemburg.com>

Christian Tismer wrote:

[This trick got me started again... thanks.]
> BTW, you might have ssh access via port 10022, at least
> for me this works just fine:
> 
> ssh -v -l pirx -p 10022 starship.python.net
> 
> Marc, you are the first Starship user! If you leave,
> I consider the project dead. Please don't.

Sorry, Christian, but things got so much out of hand
lately, that I think a new site for my stuff will make
my users happier.

With the help of Andrew I have added redirects for all
my files on starship to the new site, so the move should
be painless (the old URLs will continue to work). I'm
sort of a virtual starship user now ;-)

Anyway, starship is not dead -- it has been a great
community site for the past three years and I hope that
it will now regain the status it had until early this
year when things started to go wrong due to the various
well-known problems.

Let's take this relaunch as a chance to get things moving
again. I'd suggest to restart providing everybody in the Python 
community should with an account on the server to host Python
software in return for the annual fee (what was the PSA-membership
fee until recently). This should by the sysadmins enough
pizza to keep the server running for a looong time ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov 18 13:51:16 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 18 Nov 2000 13:51:16 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A167B44.4D53B796@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Help someone, please.

Thank you to everyone who jumped to the rescue :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From ping at lfw.org  Sat Nov 18 22:57:31 2000
From: ping at lfw.org (Ka-Ping Yee)
Date: Sat, 18 Nov 2000 13:57:31 -0800 (PST)
Subject: [Python-Dev] socket.error should be an IOError?
Message-ID: <Pine.LNX.4.10.10011181352250.504-100000@skuld.kingmanhall.org>

In socketmodule, socket.error is currently just a plain old error
(derived only from Exception).  Looks to me like it should be an IOError.
Do you agree?

 line 2379 of socketmodule.c:

???????d = PyModule_GetDict(m);
???????PySocket_Error = PyErr_NewException("socket.error", NULL, NULL);
???????if (PySocket_Error == NULL)

...just change that first NULL to PyExc_IOError.



-- ?!ng

"All models are wrong; some models are useful."
    -- George Box




From jack at oratrix.nl  Sun Nov 19 23:27:14 2000
From: jack at oratrix.nl (Jack Jansen)
Date: Sun, 19 Nov 2000 23:27:14 +0100
Subject: [Python-Dev] site.py, .pth files and unix-dependencies
Message-ID: <20001119222719.5980A1301E4@oratrix.oratrix.nl>

In the course of getting distutils to behave on the Mac I came across
.pth files for the first time (all packages I ever tried until now
used the __init__.py method of advertising themselves) and it turns
out they don't work on the Mac.

There's some funny code in site.py that triggers on "os.sep ==
'/'". I've added an "elif os.sep == ':'" for the mac-specific location 
where site-python is (sys.prefix + :lib:site-python), but checking for 
os.sep in stead of simply checking that sys.platform==posix makes me
wonder whether something infinitely more deep than my understanding
allows me to grasp is going on.

Can someone enlighten me? Is my fix okay and should I check it in?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen at oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++



From akuchlin at mems-exchange.org  Mon Nov 20 23:32:30 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Mon, 20 Nov 2000 17:32:30 -0500
Subject: [Python-Dev] Too much silence
Message-ID: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>

All the PEPs for 2.1 are supposed to be complete for Dec. 16, and some
of those PEPs are pretty complicated.  I'm a bit worried that it's
been so quiet on python-dev lately, especially after the previous two
weeks of lively discussion.  (And no comments on my Python/Distutils
PEP at all -- not even an assigned number... <sniffle>)

--amk



From nas at arctrix.com  Mon Nov 20 17:52:16 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 08:52:16 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001120085216.A30685@glacier.fnational.com>

No doubt.  I'm still waiting for some feedback on my coercion
patch.  I'd write a PEP but I don't think it would help.  Marc's
web page already explains things in detail.  Also, I don't want
to spend too much time going down a path that Guido and the other
developers are not going to accept.

The patch makes some pretty major changes to Python's internals
(for the better I think).  If something is going to be done for
2.1 then it should happen sooner rather than later.

  Neil



From jeremy at alum.mit.edu  Tue Nov 21 00:39:25 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 20 Nov 2000 18:39:25 -0500 (EST)
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120085216.A30685@glacier.fnational.com>
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
	<20001120085216.A30685@glacier.fnational.com>
Message-ID: <14873.46637.76321.845888@bitdiddle.concentric.net>

Should Marc's Web page be turned into the PEP then?  I don't have time
to read a patch, but I do have time to read a PEP.

Jeremy



From mal at lemburg.com  Tue Nov 21 00:58:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 00:58:10 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
		<20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net>
Message-ID: <3A19BA92.1A555299@lemburg.com>

Jeremy Hylton wrote:
> 
> Should Marc's Web page be turned into the PEP then?  I don't have time
> to read a patch, but I do have time to read a PEP.

If someone could do this, I'd appreciate it. I like Neil's
idea of using a type flag to signal "new style number", but apart
from that, I think that the web page pretty much makes things
clear.

The patch would need to be updated to Python 2.0, though -- don't
have time for this myself, so it's up to you.

I completely agree with Neil, that these things should be
done now rather than later: the current coercion mechanism is
a major bottleneck when it comes to implementing new types in C
that are supposed to interoperate with existing builtin types.

What the patch basically does is add Python's instance coercion
support to the C level -- in a backward compatible way.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From nas at arctrix.com  Mon Nov 20 18:54:45 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 09:54:45 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <3A19BA92.1A555299@lemburg.com>; from mal@lemburg.com on Tue, Nov 21, 2000 at 12:58:10AM +0100
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com>
Message-ID: <20001120095445.A30774@glacier.fnational.com>

On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> Jeremy Hylton wrote:
> > Should Marc's Web page be turned into the PEP then?  I don't have time
> > to read a patch, but I do have time to read a PEP.

> The patch would need to be updated to Python 2.0, though -- don't
> have time for this myself, so it's up to you.

My patch _is_ up to date.  My original message also included a
brief description of what the patch does.  For more details look
at the URL to Marc's page that I provided.  There is also PEP 208
(which should probably be updated).

Marc and Moshe: who is currently championing PEP 207 and 208?

Barry or Jeremy: the formatting of 208 on sourceforge seems to be
screwy.  Part of the extended print PEP seems to be included.

I'll increase the traffic on this list yet. :)

  Neil



From tim.one at home.com  Tue Nov 21 02:21:15 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 20 Nov 2000 20:21:15 -0500
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120095445.A30774@glacier.fnational.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOFIAAA.tim.one@home.com>

[Neil Schemenauer]
> My patch _is_ up to date.  My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).

Yes; PEP 208 should be self-contained.

> Marc and Moshe: who is currently championing PEP 207 and 208?
>
> Barry or Jeremy: the formatting of 208 on sourceforge seems to be
> screwy.  Part of the extended print PEP seems to be included.

PEP 208 is in a severely broken state.  This is the full content of
pep-0208.txt:

====================================
PEP: 208
Title: Reworking the Coercion Model
Version: $Revision: 1.1 $
Owner: davida at activestate.com (David Ascher)
Python-Version: 2.1
Status: Incomplete



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
====================================

No change has been checked in since this original skeleton was added in the
middle of July.  So I guess somebody has been uploading damaged HTML for 208
without going thru CVS and the HTML generation process.  Nothing Barry or
Jeremy can do about that:  somebody has to update the .txt file.

> I'll increase the traffic on this list yet. :)

Guido will eventually chime in on the coercions patch, but is on vacation
until next Monday (however, AFAIK, he didn't release anything first <wink>).
I'll chime in too, but am almost in the same boat.

here-yet-absent-ly y'rs  - tim




From moshez at zadka.site.co.il  Tue Nov 21 14:06:31 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 15:06:31 +0200
Subject: [Python-Dev] Too much silence 
In-Reply-To: Message from Neil Schemenauer <nas@arctrix.com> 
   of "Mon, 20 Nov 2000 08:52:16 PST." <20001120085216.A30685@glacier.fnational.com> 
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>  <20001120085216.A30685@glacier.fnational.com> 
Message-ID: <E13yD84-0004hI-00@darjeeling.zadka.site.co.il>

> No doubt.  I'm still waiting for some feedback on my coercion
> patch.  I'd write a PEP but I don't think it would help.  Marc's
> web page already explains things in detail.  Also, I don't want
> to spend too much time going down a path that Guido and the other
> developers are not going to accept.

Neil, you can take the coercion patch from me. I thought I'd
have more time to deal with it then I do. Please let me know.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From thomas at xs4all.net  Tue Nov 21 09:47:44 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 21 Nov 2000 09:47:44 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001121094744.D540@xs4all.nl>

On Mon, Nov 20, 2000 at 05:32:30PM -0500, Andrew Kuchling wrote:

> I'm a bit worried that it's been so quiet on python-dev lately, especially
> after the previous two weeks of lively discussion.

I'll appologize for my silence, but I can't speak for the rest of -dev ;)
I'm going to be silent for a few months more, though, as I'm swamped in work
and supposed to replace my boss during his 2-month vacation. I'll spend what
little time I have on catalog-sig rather than -dev issues, though I'll try
to keep up with the messages on -dev at least ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From mal at lemburg.com  Tue Nov 21 10:22:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 10:22:56 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>
Message-ID: <3A1A3EF0.3584B425@lemburg.com>

Neil Schemenauer wrote:
> 
> On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> > Jeremy Hylton wrote:
> > > Should Marc's Web page be turned into the PEP then?  I don't have time
> > > to read a patch, but I do have time to read a PEP.
> 
> > The patch would need to be updated to Python 2.0, though -- don't
> > have time for this myself, so it's up to you.
> 
> My patch _is_ up to date. 

Cool... sorry, but I didn't have time to look closely.

> My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).
> 
> Marc and Moshe: who is currently championing PEP 207 and 208?

Not me -- got no time to spare.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Tue Nov 21 12:31:34 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 13:31:34 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
Message-ID: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>

I've looked at the people bugs can be assigned to, and Greg Stein is
missing. Is this purposeful or just an oversight?

Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
would look at it -- it's his code originally, so maybe he does not think
it's a bug.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From gstein at lyra.org  Tue Nov 21 13:34:44 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 04:34:44 -0800
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 21, 2000 at 01:31:34PM +0200
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
Message-ID: <20001121043444.K21426@lyra.org>

On Tue, Nov 21, 2000 at 01:31:34PM +0200, Moshe Zadka wrote:
> I've looked at the people bugs can be assigned to, and Greg Stein is
> missing. Is this purposeful or just an oversight?
> 
> Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
> would look at it -- it's his code originally, so maybe he does not think
> it's a bug.

Probably an oversight. I've had bugs assigned before, but a month or so ago
I accidentally removed myself from the Python project (damn trash can on the
"my" page is located in a horrible place, with no conformation). Tim put me
back, but there is probably a magic switch somewhere to enable bug
assignments.

I'm going to mail Roy Fielding to see if servers or clients are allowed to
send lower-cased HTTP version strings. Looking at the Apache code, it will
assume HTTP/1.0 if a client sends a lower-cased version. I'm suspicious that
the RFC didn't clarify the case-sensitivity on that string (yes, the bug
submitter is correct that the RFC allows a lower-cased version, but that
just seems wrong/mis-intended).

In any case... pending the input from Roy, I'll deal with the bug.

But one of the SF Python Admins will need to enable my bug db privs to work
on that bug properly.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From fdrake at acm.org  Tue Nov 21 15:41:31 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 09:41:31 -0500 (EST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <20001121043444.K21426@lyra.org>
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
	<20001121043444.K21426@lyra.org>
Message-ID: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > Probably an oversight. I've had bugs assigned before, but a month or so ago
 > I accidentally removed myself from the Python project (damn trash can on the
 > "my" page is located in a horrible place, with no conformation). Tim put me
 > back, but there is probably a magic switch somewhere to enable bug
 > assignments.

  I've fixed the permissions; you're back!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at math.huji.ac.il  Tue Nov 21 22:35:49 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 23:35:49 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011212334230.10616-100000@sundial>

On Tue, 21 Nov 2000, Fred L. Drake, Jr. wrote:

>   I've fixed [Greg Stein's] permissions; you're back!  ;-)

Well, the bright side is that it generated some noise on Python-Dev
<wink>.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Tue Nov 21 23:06:04 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:06:04 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121170604.B15925@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
>Update of /cvsroot/python/python/dist/src/Lib/xml/dom
>In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
>
>Modified Files:
>	minidom.py 
...

Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
has wrong semantics".

Does the patch check for illegal children (bug #116678).

--amk



From fdrake at acm.org  Tue Nov 21 23:16:52 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:16:52 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121170604.B15925@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
Message-ID: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
 > has wrong semantics".

  Sorry; that would have taken just enough more that I didn't want to
get into that today, but it shouldn't be hard.  See my comments on the
bug page (just added).

 > Does the patch check for illegal children (bug #116678).

  No.  There's still seriously little parameter checking in minidom,
and I'm not sure I want to add that.  One of the problems people had
with PyOM was the general weight of the code, and adding all the
checks contributes to that (though I suspect the proxies in PyDOM were
a far more serious contributor).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From akuchlin at mems-exchange.org  Tue Nov 21 23:22:33 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:22:33 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:16:52PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172233.A16224@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:16:52PM -0500, Fred L. Drake, Jr. wrote:
>and I'm not sure I want to add that.  One of the problems people had
>with PyDOM was the general weight of the code, and adding all the
>checks contributes to that (though I suspect the proxies in PyDOM were
>a far more serious contributor).

Indeed; I think the proxies really obfuscated the code.  Some simple
parameter checking, though, shouldn't add too much of a burden, and
will protect users from common mistakes that will result in invalid
trees.

--amk




From fdrake at acm.org  Tue Nov 21 23:20:47 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:20:47 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121172233.A16224@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Indeed; I think the proxies really obfuscated the code.  Some simple
 > parameter checking, though, shouldn't add too much of a burden, and
 > will protect users from common mistakes that will result in invalid
 > trees.

  So when should we be expecting your patch?  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From akuchlin at mems-exchange.org  Tue Nov 21 23:25:08 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:25:08 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:20:47PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172508.A16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:20:47PM -0500, Fred L. Drake, Jr. wrote:
>  So when should we be expecting your patch?  ;-)

Expect it when you see it. 

--amk



From paulp at ActiveState.com  Tue Nov 21 23:34:29 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 14:34:29 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <3A1AF875.4110F856@activestate.com>

Andrew Kuchling wrote:
> 
> ...
> 
> Indeed; I think the proxies really obfuscated the code.  Some simple
> parameter checking, though, shouldn't add too much of a burden, and
> will protect users from common mistakes that will result in invalid
> trees.

Those checks would slow down the original tree building unless we split
the interface into "internal" methods that we use ourself and "external
methods" that do the extra checking. Worth the effort and extra code
complexity? Maybe...maybe not.

 Paul Prescod



From jeremy at alum.mit.edu  Tue Nov 21 23:26:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:26:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <3A1AF875.4110F856@activestate.com>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
Message-ID: <14874.63110.889245.738951@bitdiddle.concentric.net>

>>>>> "PP" == Paul Prescod <paulp at ActiveState.com> writes:

  PP> Andrew Kuchling wrote:
  >>
  >> ...
  >>
  >> Indeed; I think the proxies really obfuscated the code.  Some
  >> simple parameter checking, though, shouldn't add too much of a
  >> burden, and will protect users from common mistakes that will
  >> result in invalid trees.

  PP> Those checks would slow down the original tree building unless
  PP> we split the interface into "internal" methods that we use
  PP> ourself and "external methods" that do the extra checking. Worth
  PP> the effort and extra code complexity? Maybe...maybe not.

Could those checks be implemented as assertions?  If so, people who
care about speed can use "python -O"

Jeremy




From akuchlin at mems-exchange.org  Tue Nov 21 23:43:50 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:43:50 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121174350.B16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
>Could those checks be implemented as assertions?  If so, people who
>care about speed can use "python -O"

Making error checks optional and leaving it up to the user to avoid
problems...  did I get subscribed to the perl5-porters list by
mistake?

--amk



From fdrake at acm.org  Tue Nov 21 23:41:32 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:41:32 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
	<14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <14874.64028.658410.793457@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Could those checks be implemented as assertions?  If so, people who
 > care about speed can use "python -O"

  Yes, but it is not clear that the checks are expensive.
  Another issue is compliance with the spec -- DOM level 1 states that
certain exceptions will be raised for various conditions, and using
assertions to check those would mean that exceptions would *not* be
raised in those cases.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From jeremy at alum.mit.edu  Tue Nov 21 23:38:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:38:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121174350.B16259@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
	<14874.63110.889245.738951@bitdiddle.concentric.net>
	<20001121174350.B16259@kronos.cnri.reston.va.us>
Message-ID: <14874.63830.803853.153636@bitdiddle.concentric.net>

>>>>> "AMK" == Andrew Kuchling <akuchlin at mems-exchange.org> writes:

  AMK> On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
  >> Could those checks be implemented as assertions?  If so, people
  >> who care about speed can use "python -O"

  AMK> Making error checks optional and leaving it up to the user to
  AMK> avoid problems...  did I get subscribed to the perl5-porters
  AMK> list by mistake?

Not quite what I asked about: Enabling error checks by default, but
allowing users to turn them off in optmized mode.  If the checks are
expensive, which Fred subsequently said he wasn't sure about, this
might not be unreasonable.

Perhaps I'm just odd when it comes to -O.  I've never used it.

Jeremy




From gstein at lyra.org  Tue Nov 21 23:55:39 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:55:39 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121145539.B21426@lyra.org>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
> >>>>> "PP" == Paul Prescod <paulp at ActiveState.com> writes:
>   PP> Andrew Kuchling wrote:
>   >> ...
>   >> Indeed; I think the proxies really obfuscated the code.  Some
>   >> simple parameter checking, though, shouldn't add too much of a
>   >> burden, and will protect users from common mistakes that will
>   >> result in invalid trees.
> 
>   PP> Those checks would slow down the original tree building unless
>   PP> we split the interface into "internal" methods that we use
>   PP> ourself and "external methods" that do the extra checking. Worth
>   PP> the effort and extra code complexity? Maybe...maybe not.
> 
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

+1 ... that would be the way to do it.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From gstein at lyra.org  Tue Nov 21 23:56:52 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:56:52 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121145652.C21426@lyra.org>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
> Update of /cvsroot/python/python/dist/src/Lib/xml/dom
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
> 
> Modified Files:
> 	minidom.py 
> Log Message:
>...
> *** minidom.py	2000/10/23 18:09:50	1.13
> --- minidom.py	2000/11/21 22:02:22	1.14
> ***************
> *** 15,22 ****
>   """
>   
> - import pulldom
>   import string

This "import string" plus numerous uses within minidom seem a bit extraneous
since this is part of the Python 2.0 library...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From tim.one at home.com  Wed Nov 22 00:09:39 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 18:09:39 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145539.B21426@lyra.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>

[Andrew Kuchling]
> Indeed; I think the proxies really obfuscated the code.  Some
> simple parameter checking, though, shouldn't add too much of a
> burden, and will protect users from common mistakes that will
> result in invalid trees.

[Paul Prescod]
> Those checks would slow down the original tree building unless
> we split the interface into "internal" methods that we use
> ourself and "external methods" that do the extra checking. Worth
> the effort and extra code complexity? Maybe...maybe not.

[Jeremy Hylton]
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

[Greg Stein]
> +1 ... that would be the way to do it.

-1.  User input is never trustworthy.  Your and your users' software lives
will be a lot happier if you stick to the rule that an assertion failure
always (always!) announces a bug in the implementation -- assertion failure
is never a user's fault.  This makes assertions *possibly* suitable for
Paul's hypothesized "internal methods", but never for checking that
user-supplied arguments satisfy preconditions.

pinning-the-blame-is-90%-of-debugging-and-"assert"-should-pin-it-
    exactly-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 01:32:50 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 16:32:50 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>
Message-ID: <3A1B1432.E900C248@activestate.com>

Tim Peters wrote:
> 
> ...
> 
> -1.  User input is never trustworthy.  Your and your users' software lives
> will be a lot happier if you stick to the rule that an assertion failure
> always (always!) announces a bug in the implementation -- assertion failure
> is never a user's fault.  

So you prefer

if __debug__ and node.nodeType!=ELEMENT_TYPE: 
    raise TypeError

Unfortunately there's no way to turn that off at "compile time" so you
always incur the __debug__ lookup cost. That would send us back to two
versions of the methods.

Maybe testing would indicate that the performance implications are
minor. If so, I wouldn't mind having the type checks in there.

 Paul Prescod



From tim.one at home.com  Wed Nov 22 02:34:32 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 20:34:32 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B1432.E900C248@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>

[Tim, objects to abusing assertions]

[Paul Prescod]
> So you prefer
>
> if __debug__ and node.nodeType!=ELEMENT_TYPE:
>     raise TypeError

I personally prefer

if node.nodeType != ELEMENT_TYPE:
    raise TypeError

if that is in fact a correct test of whatever user-input precondition it is
you're verifying.  An assert would be appropriate if it were "impossible"
for the test to fail.

> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.

Actually, there is:

if __debug__:
    if node.nodeType != ELEMENT_TYPE:
        raise TypeError

Python produces no code for that block under -O (btw, this is the same
mechanism that makes asserts vanish under -O:  it's __debug__ that's magic,
not asserts).

As a user, though, I don't expect -O to turn off argument verification!
Same as the Python implementation in these respects:  public API functions
*always* check their arguments, while some private API functions check only
in Debug builds (and via the C library's assert() function, as it's a bug in
the implementation if a private API is misused).

do-s-right-thing-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 02:56:35 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 17:56:35 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>
Message-ID: <3A1B27D3.8ECDA690@activestate.com>

Tim Peters wrote:
> 
>...
> 
> As a user, though, I don't expect -O to turn off argument verification!
> Same as the Python implementation in these respects:  public API functions
> *always* check their arguments, while some private API functions check only
> in Debug builds (and via the C library's assert() function, as it's a bug in
> the implementation if a private API is misused).

As a user, I don't expect much argument verification from the Python
library at all! C-level verification makes sense because the alternative
is core dumps. That's not acceptable.

For the rare Python-coded function that DOES do argument verification, I
wouldn't have much of an expectation of the affect of "-O" on it
because, like Jeremy, I hardly ever use -O. So maybe the argument is not
worth having -- if nobody ever uses -O then we should always just
balance safety and performance rather than expecting the user to choose
one or the other.

 Paul Prescod



From tim.one at home.com  Wed Nov 22 04:12:44 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 22:12:44 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B27D3.8ECDA690@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>

[Paul Prescod]
> As a user, I don't expect much argument verification from the Python
> library at all! C-level verification makes sense because the alternative
> is core dumps. That's not acceptable.

Why not?  I don't see any real difference between a core dump and an
uncaught & unexpected Python exception:  in either case the program didn't
get the job done, and left stuff in an unknown state.  Nothing
supernaturally evil about a core dump in that respect; if one is
unacceptable, so is the other.  In the good old days a core dump often
crashed the OS too, but that's rare even on Windows now.

> For the rare Python-coded function that DOES do argument verification, I
> wouldn't have much of an expectation of the affect of "-O" on it
> because, like Jeremy, I hardly ever use -O. So maybe the argument is not
> worth having -- if nobody ever uses -O then we should always just
> balance safety and performance rather than expecting the user to choose
> one or the other.

I don't care about argument verification except to the extent that it
validates preconditions.  If the preconditions for using a public function
aren't stated, the docs are inadequate (what's the user supposed to do then?
guess?).  If the preconditions aren't verified, then a user error may lead
to an incomprehensible error somewhere in the bowels.  Most library
functions have preconditions of the form "x is a sequence" or "y supports
.sort()", and for such builtin types & operations laziness is usually
tolerable because the specific exception raised by accident (in the absence
of checking and the presence of a bad argument) says "not a sequence" or
"can't be .sort()ed" more or less directly.  If you've got fancier
preconditions, an accidental exception likely makes no sense at all to the
user.

Andrew's claim was "some simple parameter checking, though, shouldn't add
too much of a burden, and will protect users from common mistakes that will
result in invalid trees".  I'm arguing both that validation is valuable to
the user and that "-O" is a rotten way to turn it off (even if people *did*
use it -- and I agree that few ever do).  Without any validation first,
though, an argument about how to turn it off-- or whether there's even a
need to --is at best premature.  If Andrew is wrong (that user mistakes
aren't common, or that simple checking couldn't protect users in a useful
way), I haven't heard anyone say so.

if-not-the-rest-is-just-a-question-of-biting-the-bullet-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 05:13:22 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 20:13:22 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <3A1B47E2.E8E57AE0@activestate.com>

Tim Peters wrote:
> 
>...
>
> Why not?  I don't see any real difference between a core dump and an
> uncaught & unexpected Python exception:  in either case the program didn't
> get the job done, and left stuff in an unknown state.  

A core dump would kill Zope, PythonWin, Alice etc. An exception does
not. To me, that's a big difference. Also, Py_Object type checks are
extremely cheap in C code. And once we put in the parameter checks the
user will get an unexpected Python exception. Presumably they are not
building faulty XML trees on purpose!

Anyhow, I am won over despite your unpersuasive argument.

I note that minidom will not always give you an exception for a poorly
formed tree. That means that the programmer may not find her error until
the XML is "out of Python's hands." It should give an exception sooner
or later but not never.

 Paul Prescod



From fdrake at acm.org  Wed Nov 22 06:19:18 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:19:18 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145652.C21426@lyra.org>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121145652.C21426@lyra.org>
Message-ID: <14875.22358.807220.890291@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > This "import string" plus numerous uses within minidom seem a bit extraneous
 > since this is part of the Python 2.0 library...

  An identical copy should be shippable as part of PyXML, which needs
to support Python 1.5.2 as well.  Otherwise, a number of the changes
would be slightly different.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 06:33:38 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:33:38 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
	<3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <14875.23218.734329.350940@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I note that minidom will not always give you an exception for a poorly
 > formed tree. That means that the programmer may not find her error until
 > the XML is "out of Python's hands." It should give an exception sooner
 > or later but not never.

  I'd like to mention again that there's also a matter of compliance
with the specification.  The DOM level 1 recommendation includes
specific documentation about the exceptions that are raised in certain
conditions.  Perhaps we should "map" these to more Pythonic
exceptions, and perhaps not, but I think the exceptions should be
raised when the API specification says they will be.  This is an
important aspect of compliance, and the XML community has demonstrated
substantially more interest in standards compliance than the HTML
community ever did; we should reap the benefits and not end up having
Python discarded because the standard implementation isn't compliant.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 06:35:19 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:35:19 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
References: <3A1B27D3.8ECDA690@activestate.com>
	<LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <14875.23319.386269.278252@cj42289-a.reston1.va.home.com>

Tim Peters writes:
 > need to --is at best premature.  If Andrew is wrong (that user mistakes
 > aren't common, or that simple checking couldn't protect users in a useful
 > way), I haven't heard anyone say so.

  Frankly, I found it very helpful to get exceptions when working with
PyDOM.  Some of them annoyed me, but it made me write better code.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From uche.ogbuji at fourthought.com  Wed Nov 22 07:23:14 2000
From: uche.ogbuji at fourthought.com (uche.ogbuji at fourthought.com)
Date: Tue, 21 Nov 2000 23:23:14 -0700
Subject: [Python-Dev] Assertions 
In-Reply-To: Message from Paul Prescod <paulp@ActiveState.com> 
   of "Tue, 21 Nov 2000 16:32:50 PST." <3A1B1432.E900C248@activestate.com> 
Message-ID: <200011220623.XAA13573@localhost.localdomain>

> Tim Peters wrote:
> > 
> > ...
> > 
> > -1.  User input is never trustworthy.  Your and your users' software lives
> > will be a lot happier if you stick to the rule that an assertion failure
> > always (always!) announces a bug in the implementation -- assertion failure
> > is never a user's fault.  
> 
> So you prefer
> 
> if __debug__ and node.nodeType!=ELEMENT_TYPE: 
>     raise TypeError
> 
> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.
> 
> Maybe testing would indicate that the performance implications are
> minor. If so, I wouldn't mind having the type checks in there.

Of course, continuations would be perfect here, as one could set the 
continuation to resume after the check if he wished to bypass it.

Something tells me I should duck now...

not-very-seriously-mind-you'ly yrs

-- 
Uche Ogbuji                               Principal Consultant
uche.ogbuji at fourthought.com               +1 303 583 9900 x 101
Fourthought, Inc.                         http://Fourthought.com 
4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA
Software-engineering, knowledge-management, XML, CORBA, Linux, Python





From gstein at lyra.org  Wed Nov 22 07:33:03 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 22:33:03 -0800
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>; from paulp@ActiveState.com on Tue, Nov 21, 2000 at 08:13:22PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <20001121223302.L21426@lyra.org>

By definition, isn't minidom supposed to be cheap/easy/quick? The quick
answer to a problem?

If somebody wants an exception-conforming DOM implementation with all the
bells and whistles, then they can go elsewhere. If minidom starts getting
loaded up, then it has kind of defeated its purpose, no?

Cheers,
-g

On Tue, Nov 21, 2000 at 08:13:22PM -0800, Paul Prescod wrote:
> Tim Peters wrote:
> > 
> >...
> >
> > Why not?  I don't see any real difference between a core dump and an
> > uncaught & unexpected Python exception:  in either case the program didn't
> > get the job done, and left stuff in an unknown state.  
> 
> A core dump would kill Zope, PythonWin, Alice etc. An exception does
> not. To me, that's a big difference. Also, Py_Object type checks are
> extremely cheap in C code. And once we put in the parameter checks the
> user will get an unexpected Python exception. Presumably they are not
> building faulty XML trees on purpose!
> 
> Anyhow, I am won over despite your unpersuasive argument.
> 
> I note that minidom will not always give you an exception for a poorly
> formed tree. That means that the programmer may not find her error until
> the XML is "out of Python's hands." It should give an exception sooner
> or later but not never.
> 
>  Paul Prescod
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/



From moshez at zadka.site.co.il  Wed Nov 22 18:33:27 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 19:33:27 +0200
Subject: [Python-Dev] PEP Process Inefficient?
Message-ID: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>

The PEP process, while *miles* better then anything I've seen in any
other development process I've ever seen, has some deficiencies.
I'll try to point them out, and to suggest some partial solution:

1. Users are not sure who to post PEP questions/remarks to: Python dev?
   the original author? The Python devver they know best?
2. It is the responsiblity of the PEP author to add open questions/pertinent
   remarks to the PEP.
3. Mail about the PEP which contains important discussion is lost.

Proposal: (still some shaky areas though):

Why not use Squishdot for PEPs? The PEP would be the "article", and
remarks could be comments. I think (but I'm not sure) Squishdot
has the ability to edit/moderate comments, and retroactively change the
article. If not, then my second best hope is some kind of Wiki system,
with self imposed article/comments-like structure.

Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
has inspired a lot of discussion, and very interesting comments. However
for someone who doesn't remember all the e-mails by heart, wading through
the archives is much too time consuming. In addition, personal mail sent to
me is not available for public discussion. 

Problems with proposed solution: no e-mail notification. I think Zwiki
does have an option for e-mail notification, though. Don't know about
Squishdot.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Wed Nov 22 10:56:23 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 10:56:23 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <3A1B9847.1E6205AC@lemburg.com>

Moshe Zadka wrote:
> 
> The PEP process, while *miles* better then anything I've seen in any
> other development process I've ever seen, has some deficiencies.
> I'll try to point them out, and to suggest some partial solution:
> 
> 1. Users are not sure who to post PEP questions/remarks to: Python dev?
>    the original author? The Python devver they know best?

Perhaps the PEP should specify this by e.g. using a header
Comments-To: email-address

The email address could then be made "usable" on the PEP web pages.

> 2. It is the responsiblity of the PEP author to add open questions/pertinent
>    remarks to the PEP.
> 3. Mail about the PEP which contains important discussion is lost.

The original idea was for PEP authors to include the important
parts of those discussions in summary in the PEP -- AFAIK, this
is not done :-(
 
> Proposal: (still some shaky areas though):
> 
> Why not use Squishdot for PEPs? The PEP would be the "article", and
> remarks could be comments. I think (but I'm not sure) Squishdot
> has the ability to edit/moderate comments, and retroactively change the
> article. If not, then my second best hope is some kind of Wiki system,
> with self imposed article/comments-like structure.
> 
> Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
> has inspired a lot of discussion, and very interesting comments. However
> for someone who doesn't remember all the e-mails by heart, wading through
> the archives is much too time consuming. In addition, personal mail sent to
> me is not available for public discussion.
> 
> Problems with proposed solution: no e-mail notification. I think Zwiki
> does have an option for e-mail notification, though. Don't know about
> Squishdot.

Hmm, wouldn't something like the SF bug manager be the ideal
platform ? Perhaps someone could get the SF guys to add something
like an "idea" manager with similar functionality but different
content to SF ?!

In the meantime, I think PEP discussions ought to taken to the
sig mailing lists using python-dev as fallback solution.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Wed Nov 22 11:27:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 12:27:10 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1B9847.1E6205AC@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>

On Wed, 22 Nov 2000, M.-A. Lemburg wrote:

> The original idea was for PEP authors to include the important
> parts of those discussions in summary in the PEP -- AFAIK, this
> is not done :-(

I'm trying to do this. However, summarising every e-mail sent to me
about this PEP is quite a burden.

> Hmm, wouldn't something like the SF bug manager be the ideal
> platform ? Perhaps someone could get the SF guys to add something
> like an "idea" manager with similar functionality but different
> content to SF ?!

Yeah, that would work to.

> In the meantime, I think PEP discussions ought to taken to the
> sig mailing lists using python-dev as fallback solution.

I think most of the PEPs are not in the charter of any new sig, and
python-dev is invitation only forum. Perhaps having pep-xxx-sig would
solve it too?

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Wed Nov 22 12:16:09 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 12:16:09 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>
Message-ID: <3A1BAAF8.231B3914@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 22 Nov 2000, M.-A. Lemburg wrote:
> 
> > The original idea was for PEP authors to include the important
> > parts of those discussions in summary in the PEP -- AFAIK, this
> > is not done :-(
> 
> I'm trying to do this. However, summarising every e-mail sent to me
> about this PEP is quite a burden.
> 
> > Hmm, wouldn't something like the SF bug manager be the ideal
> > platform ? Perhaps someone could get the SF guys to add something
> > like an "idea" manager with similar functionality but different
> > content to SF ?!
> 
> Yeah, that would work to.
> 
> > In the meantime, I think PEP discussions ought to taken to the
> > sig mailing lists using python-dev as fallback solution.
> 
> I think most of the PEPs are not in the charter of any new sig, and
> python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> solve it too?

I don't think people would sign up for each and every PEP...
and you know how hard it is to even move a discussion from
one forum to another -- this either doesn't work out or 
kills discussion.

I think the simplest solution would be something like the
SF bug manager. I'm pretty sure that other projects on SF
have similar problems, so this might be worthwhile dicussing
with the SF folks. The technology and user interface are there,
it would only take a few new tables in the database.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fdrake at users.sourceforge.net  Wed Nov 22 13:51:00 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Wed, 22 Nov 2000 04:51:00 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011221251.EAA16696@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From jeremy at alum.mit.edu  Wed Nov 22 15:50:22 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 22 Nov 2000 09:50:22 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <14875.56622.102311.641555@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at zadka.site.co.il> writes:

  MZ> The PEP process, while *miles* better then anything I've seen in
  MZ> any other development process I've ever seen, has some

(Not that we've used the PEP process for much of anything yet.
Except, I guess, extended print, which I've grown quite fond of.)

  MZ> deficiencies.  I'll try to point them out, and to suggest some
  MZ> partial solution:

I don't like your partial solution, so I'll try to pick about the
problems with the process <0.8 wink>.  Put another way: I can't
envision Slashdot as a useful design forum, but am interested in
improving the PEP process.  

  MZ> 1. Users are not sure who to post PEP questions/remarks to:
  MZ>    Python dev?  the original author? The Python devver they know
  MZ>    best?

In the absence of a reason to post elsewhere, the comments ought to be
sent to the PEP author.  If the message is intended to provoke wider
discussion, then the user can post it on any relevent forum (and cc
the PEP author).  A Unicode PEP might be discussed on the i18n-sig; a
Web PEP on the python-web-modules list; a change to core Python on
python-dev. 

I don't think there will ever be a rule that says: "Comments on PEP
1812 must be posted to the pep-1812 web forum."  Discussion should
occur in the forum in which it is most relevent.  Perhaps we could all
send our comments to Andrew, and he could write a bi-weekly
pep-comment summary <wink>.

Is the problem that you don't know where to post comments on someone
else's PEP, or that you are having trouble keeping track of
discussions in multiple places?

  MZ> 2. It is the responsiblity of the PEP author to add open
  MZ>    questions/pertinent remarks to the PEP.

What is the deficiency with this approach?  The PEP is supposed to
present a coherent proposal and design for a new language feature.
It is the designer's responsibility to write a good design document;
several people can share responsibility for the design.

Are you saying it's a problem that the PEPs aren't open to
modification by anyone?  (The last thing we need is someone who
doesn't understand what a first-class function is messing with the
scoping PEP <0.2 wink>.)

The designer is responsible for discussing trade-offs and alternatives
to justify her design.  This is where the responsibility to address
questions and comments comes from.

  MZ> 3. Mail about the PEP which contains important discussion is
  MZ>    lost.

How?  I've got all the mail on the nested static scopes PEP.  Is the
problem just the personal burden of keeping track of lots of mail
messages discussing a PEP?

Any discussion that occurs on a mailing list will be preserved in the
mailing list archive.  That's not lost.  So the only problem would be
with discussion that occurs in private email, where everyone deletes
copies of the mail.

I wonder if the key problem is not having an authoritative mail
archive for the PEP.  One part of the Scheme RFI process that we did
not copy was having a mail archive for each document.  Perhaps this
would be sufficient to address your concerns.  If you, the PEP author,
receive a comment by private email, you could bounce it to the
archive.

the-biggest-problem-with-the-PEP-process-is-no-one-writing-PEPs-ly y'rs,
Jeremy



From moshez at math.huji.ac.il  Wed Nov 22 16:12:28 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:12:28 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.56622.102311.641555@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>

The key problem, as Jeremy noted, was that there is no archive of
discussion about the PEP. I had a much more long winded and roundabout
way of saying that. Also, a minor related problem is that when the
only address on the PEP points to me (for example), I get all the mail
about the PEP personally -- and I'm loath to bounce it to a public forum.
*Having* a public forum which is *indicated in the PEP* would solve that.

I could phrase that as a point-by-point answer to Jeremy (and originally
did) but I decided that that would be idiotic.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Wed Nov 22 16:21:01 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:21:01 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011221657310.14321-100000@sundial>
Message-ID: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > only address on the PEP points to me (for example), I get all the mail
 > about the PEP personally -- and I'm loath to bounce it to a public forum.
 > *Having* a public forum which is *indicated in the PEP* would solve that.

  Having a public forum isn't quite what's needed (though useful);
much of the time, it's more important to have a public *archive*.  I'm
sure there's a way to set things up a mail handler that maintains an
mbox file with collected mails on a topic, and a pipermail browsable
version of that should be available, and each PEP should point to the
appropriate archive.  (It should also clearly state that relevant
emails may be added to the archive at the recipient's discretion!)
  Using a mail handler for this would allow the handler to be added to
the CC: on the emails and generally collect replies without any
additional work on the part of the PEP author.  The handler could be
CC'd when the PEP is posted so that replies to the posting go there as
well.
  Now we just need someone interested in writing a handler.  ;-{


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at math.huji.ac.il  Wed Nov 22 16:28:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:28:56 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>

On Wed, 22 Nov 2000, Fred L. Drake, Jr. wrote:

>   Using a mail handler for this would allow the handler to be added to
> the CC: on the emails and generally collect replies without any
> additional work on the part of the PEP author.  The handler could be
> CC'd when the PEP is posted so that replies to the posting go there as
> well.
>   Now we just need someone interested in writing a handler.  ;-{

Ummmmm....couldn't that be implemented as a mailman list with *no*
subscribers (subscription requests auto-bounce)
I probably missed something?
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Wed Nov 22 16:32:45 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:32:45 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>; from gstein@lyra.org on Tue, Nov 21, 2000 at 10:33:03PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com> <20001121223302.L21426@lyra.org>
Message-ID: <20001122103245.A17172@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 10:33:03PM -0800, Greg Stein wrote:
>If somebody wants an exception-conforming DOM implementation with all the
>bells and whistles, then they can go elsewhere. If minidom starts getting
>loaded up, then it has kind of defeated its purpose, no?

Checking for the correct children should be quite fast; in PyDOM it
was basically the line "if newnode.type not in
self._LEGAL_CHILDREN_TYPES: raise ...".  I don't know about the other
minidom bug report, but will try to look into it before too long.

--amk

   Some compilers allow a check during execution that subscripts do not
   exceed array dimensions. This is a help, but not sufficient. First,
   many programmers do not use such compilers because "They're not
   efficient." (Presumably, this means that it is vital to get the wrong
   answers quickly.)
 
   Kernighan and Plauger, in _The Elements of Programming Style_




From fredrik at effbot.org  Wed Nov 22 16:49:38 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Wed, 22 Nov 2000 16:49:38 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <14875.56622.102311.641555@bitdiddle.concentric.net><Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <012f01c0549b$d39cb320$3c6340d5@hagrid>

Fred L. Drake wrote:
>   Having a public forum isn't quite what's needed (though useful);
> much of the time, it's more important to have a public *archive*.  I'm
> sure there's a way to set things up a mail handler that maintains an
> mbox file with collected mails on a topic, and a pipermail browsable
> version of that should be available, and each PEP should point to the
> appropriate archive.  (It should also clearly state that relevant
> emails may be added to the archive at the recipient's discretion!)

http://www.lfw.org/ping/roundup.html

</F>




From akuchlin at mems-exchange.org  Wed Nov 22 16:34:36 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:34:36 -0500
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 22, 2000 at 10:21:01AM -0500
References: <14875.56622.102311.641555@bitdiddle.concentric.net> <Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <20001122103436.B17172@kronos.cnri.reston.va.us>

On Wed, Nov 22, 2000 at 10:21:01AM -0500, Fred L. Drake, Jr. wrote:
>  Using a mail handler for this would allow the handler to be added to
>the CC: on the emails and generally collect replies without any
>additional work on the part of the PEP author.  The handler could be
>CC'd when the PEP is posted so that replies to the posting go there as
>well.

This sounds suspiciously like the nosy lists in ?!ng's Roundup
bug-tracking tool.

--amk



From fdrake at acm.org  Wed Nov 22 16:33:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:33:04 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>
References: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011221727460.14321-100000@sundial>
Message-ID: <14875.59184.806912.900828@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Ummmmm....couldn't that be implemented as a mailman list with *no*
 > subscribers (subscription requests auto-bounce)
 > I probably missed something?

  Definately, but at that point, why disallow subscribers unless
there's already a forum for the PEP?
  I don't object to setting up a mailing list if there isn't one that
fits the bill, but I think Mailman is a fairly heavy way to do the
job, and server resources definately seem scarce these days, so we
don't want to overload what we have!
  (We'd probably also want to add checks that a message hasn't already
been sent along, and deal with forwards a little differently -- we
only want the forwarded message if someone is just archiving a message
received in private email.  Hmmm... this is starting to sound like an
AI problem!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 16:35:19 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:35:19 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122103436.B17172@kronos.cnri.reston.va.us>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011221657310.14321-100000@sundial>
	<14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
	<20001122103436.B17172@kronos.cnri.reston.va.us>
Message-ID: <14875.59319.351125.5491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > This sounds suspiciously like the nosy lists in ?!ng's Roundup
 > bug-tracking tool.

  Yes, this is very suspicious, indeed!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From trentm at ActiveState.com  Wed Nov 22 17:38:51 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:38:51 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1BAAF8.231B3914@lemburg.com>; from mal@lemburg.com on Wed, Nov 22, 2000 at 12:16:09PM +0100
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com>
Message-ID: <20001122083851.H16729@ActiveState.com>

On Wed, Nov 22, 2000 at 12:16:09PM +0100, M . -A . Lemburg wrote:
> Moshe Zadka wrote:
> > > In the meantime, I think PEP discussions ought to taken to the
> > > sig mailing lists using python-dev as fallback solution.
> > 
> > I think most of the PEPs are not in the charter of any new sig, and
> > python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> > solve it too?
> 
> I don't think people would sign up for each and every PEP...
> and you know how hard it is to even move a discussion from
> one forum to another -- this either doesn't work out or 
> kills discussion.
> 
> I think the simplest solution would be something like the
> SF bug manager. I'm pretty sure that other projects on SF
> have similar problems, so this might be worthwhile dicussing
> with the SF folks. The technology and user interface are there,
> it would only take a few new tables in the database.


A Roundup mailing list with a category for each PEP?

Trent

-- 
Trent Mick
TrentM at ActiveState.com



From trentm at ActiveState.com  Wed Nov 22 17:40:40 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:40:40 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>; from trentm@ActiveState.com on Wed, Nov 22, 2000 at 08:38:51AM -0800
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com> <20001122083851.H16729@ActiveState.com>
Message-ID: <20001122084040.I16729@ActiveState.com>

On Wed, Nov 22, 2000 at 08:38:51AM -0800, Trent Mick wrote:
> 
> A Roundup mailing list with a category for each PEP?
> 

Or I should just read all the thread first! Y'all are way ahead of me.


Trent

-- 
Trent Mick
TrentM at ActiveState.com



From moshez at math.huji.ac.il  Wed Nov 22 17:41:50 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 18:41:50 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>
Message-ID: <Pine.GSO.4.10.10011221840420.19237-100000@sundial>

On Wed, 22 Nov 2000, Trent Mick wrote:

> A Roundup mailing list with a category for each PEP?

OK, when we get over the make-Moshe-feel-stupid-for-not-proposing-roundup
part, just let me know <wink>. Yep, Roundup is perfect for this!

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From barry at digicool.com  Wed Nov 22 20:19:54 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Wed, 22 Nov 2000 14:19:54 -0500
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
	<20001120085216.A30685@glacier.fnational.com>
	<14873.46637.76321.845888@bitdiddle.concentric.net>
	<3A19BA92.1A555299@lemburg.com>
	<20001120095445.A30774@glacier.fnational.com>
Message-ID: <14876.7258.137276.784970@anthem.concentric.net>

>>>>> "NS" == Neil Schemenauer <nas at arctrix.com> writes:

    NS> Barry or Jeremy: the formatting of 208 on sourceforge seems to
    NS> be screwy.  Part of the extended print PEP seems to be
    NS> included.

Something's really screwed up here.  The pep-0208.txt file is almost
empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
that I see on the web page.  I just did a "cvs up" and a "cvs log" and
I don't see anything new checked in for pep 208.

Did somebody forget to commit their changes to CVS?
-Barry



From fdrake at acm.org  Wed Nov 22 21:44:59 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 15:44:59 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
	<3A1B47E2.E8E57AE0@activestate.com>
	<20001121223302.L21426@lyra.org>
Message-ID: <14876.12363.340244.877950@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > By definition, isn't minidom supposed to be cheap/easy/quick? The quick
 > answer to a problem?
 > 
 > If somebody wants an exception-conforming DOM implementation with all the
 > bells and whistles, then they can go elsewhere. If minidom starts getting
 > loaded up, then it has kind of defeated its purpose, no?

  I've started a new discussion about this over in the XML SIG; if
you're interested, please join in on that list.  If you haven't been
following it, you might want to check the archives starting with
yesterday.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From cgw at fnal.gov  Wed Nov 22 22:20:05 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Wed, 22 Nov 2000 15:20:05 -0600 (CST)
Subject: [Python-Dev] more Sourceforge CVS screwiness
Message-ID: <14876.14469.534078.512149@buffalo.fnal.gov>

Haven't seen this for a while - I was unable to build the most current
version of Python from CVS and was about to squawk, when I realized
that the problem was a bogus "config.h" in my dist/src/Include
directory:

shell$ pwd
/usr/local/src/Python-CVS/python/dist/src/Include
shell$ cvs status config.h                                                                             
===================================================================
File: config.h          Status: Up-to-date

   Working revision:    2.1
   Repository revision: 2.1     /cvsroot/python/python/dist/src/Include/Attic/config.h,v
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      (none)


A fresh checkout of the whole tree does not contain this file.  I know
how to fix this (edit CVS/Entries) but it troubles me that old files
from the Attic keep re-appearing when I do updates (without any
sticky tags or anything, I always do "cvs update -dP")

Am I the only person who sees this?  Does Sourceforge just not like me
for some reason?




From moshez at zadka.site.co.il  Thu Nov 23 07:46:51 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Thu, 23 Nov 2000 08:46:51 +0200
Subject: [Python-Dev] Too much silence 
In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) 
   of "Wed, 22 Nov 2000 14:19:54 EST." <14876.7258.137276.784970@anthem.concentric.net> 
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>  <14876.7258.137276.784970@anthem.concentric.net> 
Message-ID: <E13yq9k-0007Ci-00@darjeeling.zadka.site.co.il>

> Something's really screwed up here.  The pep-0208.txt file is almost
> empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
> that I see on the web page.  I just did a "cvs up" and a "cvs log" and
> I don't see anything new checked in for pep 208.
> 
> Did somebody forget to commit their changes to CVS?

I did, when you define forget to include "didn't have the time yet
to translate the web page to PEP".
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From thomas at xs4all.net  Wed Nov 22 23:40:57 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 22 Nov 2000 23:40:57 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <14876.7258.137276.784970@anthem.concentric.net>; from barry@digicool.com on Wed, Nov 22, 2000 at 02:19:54PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com> <14876.7258.137276.784970@anthem.concentric.net>
Message-ID: <20001122234057.H540@xs4all.nl>

On Wed, Nov 22, 2000 at 02:19:54PM -0500, Barry A. Warsaw wrote:

[ The HTML version of PEP 208 is screwed ]

> Did somebody forget to commit their changes to CVS?

-rw-rw-r--   1 nowonder python      25515 Nov 10 09:31 pep-0208.html

Peter-SK wrote these versions to disk, it seems, but it could have been
edited later. Maybe he can check his .txt version to see whether that's
garbled as well ?

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From akuchlin at mems-exchange.org  Thu Nov 23 17:37:22 2000
From: akuchlin at mems-exchange.org (A.M. Kuchling)
Date: Thu, 23 Nov 2000 11:37:22 -0500
Subject: [Python-Dev] Regression testing
Message-ID: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>

This morning I was looking at test_minidom.py, which contains its own
simplistic framework for running a series of regression tests.  Maybe
another 2.1 task could be selecting a standard unit testing framework;
this should mostly be a matter of selecting one, since there are
already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
and the unittest.py from Quixote, and have a vague feeling I'm
forgetting one other module.  No prizes for guessing which one I'd
vote for. :)

--amk




From trentm at ActiveState.com  Thu Nov 23 17:48:30 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Thu, 23 Nov 2000 08:48:30 -0800
Subject: [Python-Dev] Regression testing
In-Reply-To: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>; from akuchlin@mems-exchange.org on Thu, Nov 23, 2000 at 11:37:22AM -0500
References: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>
Message-ID: <20001123084830.I31441@ActiveState.com>

On Thu, Nov 23, 2000 at 11:37:22AM -0500, A.M. Kuchling wrote:
> This morning I was looking at test_minidom.py, which contains its own
> simplistic framework for running a series of regression tests.  Maybe
> another 2.1 task could be selecting a standard unit testing framework;
> this should mostly be a matter of selecting one, since there are
> already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
> and the unittest.py from Quixote, and have a vague feeling I'm
> forgetting one other module.  No prizes for guessing which one I'd
> vote for. :)

Sure, if they provide a clear benefit. I.e. who will want to translate
all the existing tests. As well, I have a patch (which, I admit, I haven't
been maintaining or pushing much) to allow direct testing of the C API). I
have no experience with either of the unit testing frameworks you mention.
Will they be able to do that? Presumably yes, using the same hack that I
proposed.

http://sourceforge.net/patch/?func=detailpatch&patch_id=101162&group_id=5470

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From lutz at rmi.net  Sat Nov 25 16:12:56 2000
From: lutz at rmi.net (Mark Lutz)
Date: Sat, 25 Nov 2000 08:12:56 -0700
Subject: [Python-Dev] A house upon the sand
Message-ID: <004401c056f2$34a9ee00$dc73fea9@vaio>

Hi again Guido,

On a totally unrelated topic: has anyone pushed the idea of a
Python language standard lately?  Things seem to be changing too
fast for many developers to keep up, and it seems to me that a
formal standard doc might help ease a few fears I've seen out 
there.  Books used to be a sort of de facto standard, but even
they aren't reliable anymore; and the manuals aren't useful as a 
standard if they are open to arbitrary change every few months.

Frankly, some people in my classes are very concerned by the rapid 
pace of Python change, and I think their fear is justified.  I get 
burned a little almost every time a new Python release rolls out
too.  Most recently, some new book examples that worked in 1.5.2 
this summer no longer work under 2.0 this fall; I understand that
most changes are improvements (and minor), but this is not a great
story to tell.

A prime example: the string module, used in almost every Python 
program ever written by the half-million Python users out there,
has suddenly been marked as deprecated.  I expect that it won't 
really go away, but has anyone considered the impact of even the
suggestion of its deprecation on Python's acceptance?

If using Python requires that programmers invest lots of time 
tracking the whims of python-dev, then Python will become much 
less useful, imo.  Most developers just don't have the extra time
to spare.  A formal standard doc could give us at least a baseline
Python definition that developers could cling to.  Companies need 
to see a solid and reliable foundation.

Unfortunately, I don't have the time or interest in pushing this
idea through to fruition myself.  Do you have any ideas along these 
lines?  Maybe this task belongs in whatever body eventually takes 
over ownership.  I'm copying this to python-dev in the hopes that 
it might trigger some sort of discussion.

Cheers,
--Mark Lutz  (http://rmi.net/~lutz)




From mal at lemburg.com  Sat Nov 25 17:21:06 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 17:21:06 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A1FE6F2.6A612EF9@lemburg.com>

Mark Lutz wrote:
> 
> Frankly, some people in my classes are very concerned by the rapid
> pace of Python change, and I think their fear is justified.  I get
> burned a little almost every time a new Python release rolls out
> too.  Most recently, some new book examples that worked in 1.5.2
> this summer no longer work under 2.0 this fall; I understand that
> most changes are improvements (and minor), but this is not a great
> story to tell.
> 
> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

It is depreciated because string methods provide a better
model of extensibility for future versions. string.py simply
interfaces to these new methods. It won't go away, but using the
methods directly will provide better performance and an
overall better experience... this doesn't mean that Python
programmers can no longer use string.py, but it should hint
them towards using the string methods instead.

Next to come are number methods, AFAICT ;-)
 
> If using Python requires that programmers invest lots of time
> tracking the whims of python-dev, then Python will become much
> less useful, imo.  Most developers just don't have the extra time
> to spare.  A formal standard doc could give us at least a baseline
> Python definition that developers could cling to.  Companies need
> to see a solid and reliable foundation.

The only major incompatbile changes in 2.0 are the .append()
et al. changes and the str(1L) change. These have been
highlighted in the changes paper.
 
> Unfortunately, I don't have the time or interest in pushing this
> idea through to fruition myself.  Do you have any ideas along these
> lines?  Maybe this task belongs in whatever body eventually takes
> over ownership.  I'm copying this to python-dev in the hopes that
> it might trigger some sort of discussion.

I don't see how a Python standard would do any good. Standards
have version numbers as well and change at about the same rate
(e.g. take Unicode 2.0 vs. Unicode 3.0).

Besides, who forces anyone to use 2.0 instead of 1.5.x which
has been around for many years now ?

Sorry, but I sense bureaucracy kreeping into the house...
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Sat Nov 25 17:25:32 2000
From: tismer at tismer.com (Christian Tismer)
Date: Sat, 25 Nov 2000 18:25:32 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <3A1FE7FC.AF14EF85@tismer.com>

(Marc, your clock is wrong by 1 hour :)

"M.-A. Lemburg" wrote:
> 
> Mark Lutz wrote:
...
> > Unfortunately, I don't have the time or interest in pushing this
> > idea through to fruition myself.  Do you have any ideas along these
> > lines?  Maybe this task belongs in whatever body eventually takes
> > over ownership.  I'm copying this to python-dev in the hopes that
> > it might trigger some sort of discussion.
> 
> I don't see how a Python standard would do any good. Standards
> have version numbers as well and change at about the same rate
> (e.g. take Unicode 2.0 vs. Unicode 3.0).
> 
> Besides, who forces anyone to use 2.0 instead of 1.5.x which
> has been around for many years now ?
> 
> Sorry, but I sense bureaucracy kreeping into the house...

Saying things explicitly is Mark's point, IMHO.

I don't see the danger of bureaucracy so much. Instead, setting
a standard is a sign of maturity for a language. Python as it
is, and in former versions, was a per-se standard, set by
the implementation. The Python documentation explains how
things work, but this is a description of the current
implementation, not setting possible ranges, like a standard.
A standard would allow anybody to write a Python implementation
which conforms to it, without knowing all the
details of the given implementation. He would not have to
guess (or ask Guido) what design decisions are final and
considered a part of the language, or just by chance,
ease of implementation or whatever reason might have let to it.
There are many issues where we know how it works
in one implementation and the other, but it is undefined
which behavior is intended, by chance, enforced or forbidden.

Example: Python 2.0 has now a garbage collector. Not so long
ago, a garbage collector seemed out of reach, and before
Guido sketched a GC by himself, I even had the impression
that a gc was out of question. (should have borrowed the
time machine :)
JPython had garbage collection in the first place, inherited
from Java. So, with Python 1.5.2, 2.0, and JPython, we get
three different implementations, and three different
lifetimes of objects.
It would be helpful if a Python standard would define
how an implementation should do this: Do objects have
to be destructed at all, or which objects have to, or
is it completely up to the implementor to decide?
Even if so, then this would be stated, and people
who write scripts compliant to Python standard 2.0
would know that they can never rely on object destruction,
for instance.

Other Example:
There are certain design decisions in the object layout,
which are exposed to the user, like __dict__ for many
objects. It is not clear if this is a language feature
or an implementation detail. The current user practice
enforces us to support __dict__, I think, although
very different decision would be drawn if someone
was designing a true compiler for Python. Does it have
to exist, and if so, does it have to be a dictionary?
Statements like "it happens to be a dict, but anything
supported by getattr() would do, but it must be there"
would be an example statement, most helpful for alternative
implementations, which still fulfill the standard.

better-explicit-than-implicit - ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From mal at lemburg.com  Sat Nov 25 18:59:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 18:59:10 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com>
Message-ID: <3A1FFDEE.E2321B40@lemburg.com>

Christian Tismer wrote:
> 
> > Mark Lutz wrote:
> ...
> > > Unfortunately, I don't have the time or interest in pushing this
> > > idea through to fruition myself.  Do you have any ideas along these
> > > lines?  Maybe this task belongs in whatever body eventually takes
> > > over ownership.  I'm copying this to python-dev in the hopes that
> > > it might trigger some sort of discussion.
> >
> > I don't see how a Python standard would do any good. Standards
> > have version numbers as well and change at about the same rate
> > (e.g. take Unicode 2.0 vs. Unicode 3.0).
> >
> > Besides, who forces anyone to use 2.0 instead of 1.5.x which
> > has been around for many years now ?
> >
> > Sorry, but I sense bureaucracy kreeping into the house...
> 
> Saying things explicitly is Mark's point, IMHO.
>
> I don't see the danger of bureaucracy so much. Instead, setting
> a standard is a sign of maturity for a language. Python as it
> is, and in former versions, was a per-se standard, set by
> the implementation. The Python documentation explains how
> things work, but this is a description of the current
> implementation, not setting possible ranges, like a standard.
> A standard would allow anybody to write a Python implementation
> which conforms to it, without knowing all the
> details of the given implementation. He would not have to
> guess (or ask Guido) what design decisions are final and
> considered a part of the language, or just by chance,
> ease of implementation or whatever reason might have let to it.
> There are many issues where we know how it works
> in one implementation and the other, but it is undefined
> which behavior is intended, by chance, enforced or forbidden.
> 
> [GC Example, __dict__ example]

Ok. Maybe I misunderstood Mark... if what he wants is a language
definition like e.g. the one for Java and its VM then I agree.
This would be a good thing. OTOH, I don't see much of a point
in giving the user a feeling of fixed standards when we are 
constantly talking about ways to break code ;-) (e.g. take the
Py3K upgrade path PEP).

We could have a language definition for say the 2.0 version
series, but the definition would have to be extended for
the 3.0 series (possibly even within the 2.0 series). It would
gives us a feeling of what "Python" stands for and also
simplify the task of writing different Python implementations,
but nothing much else... certainly not the warm fuzzy feeling
which Mark probably had in mind.

I think the overall impression of Python 2.0 being a moving target
is a bit overrated: there aren't really all that many changes
which affect existing code, so a Python 1.5.2 introduction will
still remain mostly valid even when the students use Python 2.0.

It is true, that Python 2.0 offers a lot more features than
1.5.2 and that in some contexts (e.g. database interfacing
on Windows) these new features have made their way into third
party extensions rather fast (e.g. switching from strings to
Unicode). The visible core language hasn't changed much though...
and this is goodness.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From thomas at xs4all.net  Sat Nov 25 20:45:10 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 25 Nov 2000 20:45:10 +0100
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <3A1FE6F2.6A612EF9@lemburg.com>; from mal@lemburg.com on Sat, Nov 25, 2000 at 05:21:06PM +0100
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <20001125204509.I540@xs4all.nl>

On Sat, Nov 25, 2000 at 05:21:06PM +0100, M.-A. Lemburg wrote:
> Mark Lutz wrote:

> > A prime example: the string module, used in almost every Python
> > program ever written by the half-million Python users out there,
> > has suddenly been marked as deprecated.  I expect that it won't
> > really go away, but has anyone considered the impact of even the
> > suggestion of its deprecation on Python's acceptance?

> It is depreciated because string methods provide a better
> model of extensibility for future versions. string.py simply
> interfaces to these new methods. It won't go away, but using the
> methods directly will provide better performance and an
> overall better experience...

Unfortunately, the term 'deprecated' suggests to most people, including me,
that it will be removed in future versions. I'm with Mark here; marking it
deprecated causes a lot of confusions, IMHO.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From tismer at tismer.com  Sun Nov 26 12:35:12 2000
From: tismer at tismer.com (Christian Tismer)
Date: Sun, 26 Nov 2000 13:35:12 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com> <3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <3A20F570.6FCD09F2@tismer.com>


"M.-A. Lemburg" wrote:
> 
> Christian Tismer wrote:
> >
[favorizing a standard]
> > > Mark Lutz wrote:
[standard, to avoid confusion]
...
> Ok. Maybe I misunderstood Mark... if what he wants is a language
> definition like e.g. the one for Java and its VM then I agree.
> This would be a good thing. OTOH, I don't see much of a point
> in giving the user a feeling of fixed standards when we are
> constantly talking about ways to break code ;-) (e.g. take the
> Py3K upgrade path PEP).

Whatever other feelings come into play, a standard would
be a good thing, IMHO.

Aside, the PERL world is thinking into this direction as well.
See this rosler article (also interesting concerning Python:)
http://www.perl.com/pub/2000/06/rosler.html

Since Python is so much cleaner and smaller (still?), this
is a chance to be the winder here.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From tim.one at home.com  Sun Nov 26 23:39:42 2000
From: tim.one at home.com (Tim Peters)
Date: Sun, 26 Nov 2000 17:39:42 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>

On the deprecation of the string module:  where did this idea come from?
I've never seen anything saying that the string module is deprecated.  The
2.0 docs say that the three specific functions string.{atoi, atol, atof} are
deprecated as of 2.0, but that's it.

On the meaning of deprecation:  yes, it means they'll go away someday.

On gc semantics:  Section 3.1 ("Objects, values and types") of the Lang Ref
sez:

    Objects are never explicitly destroyed; however, when they become
    unreachable they may be garbage-collected. An implementation is
    allowed to postpone garbage collection or omit it altogether -- it is
    a matter of implementation quality how garbage collection is
    implemented, as long as no objects are collected that are still
    reachable. (Implementation note: the current implementation uses a
    reference-counting scheme which collects most objects as soon as
    they become unreachable, but never collects garbage containing
    circular references.)

If Python had a standard, the only thing that would change is that the
obsolete parenthetical comment would be removed.

On formal standardization:  not likely.  An ISO/ANSI std takes a minimum of
person-decades of work, and Guido would be unable to make the time even to
review committee documents.  IOW, he won't cooperate, because he can't, and
that's that.  Propose changes to the Lang Ref instead (for example, Guido's
intent was that binary operators of equal precedence be evaluated strictly
left to right in the absence of parens, but the Lang Ref never got around to
saying so).

On the pace of change:  by all objective measures, Python change ended the
day 2.0 was released <0.9 wink>.

the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim




From martin at loewis.home.cs.tu-berlin.de  Sun Nov 26 23:53:54 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 26 Nov 2000 23:53:54 +0100
Subject: [Python-Dev] A house upon the sand
Message-ID: <200011262253.XAA01869@loewis.home.cs.tu-berlin.de>

> the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim

Since Java has been cited as an example of a language that has a
"language definition" - it is just a language reference book, either.
Sun did a number of attempts to make Java an Internation Standard, and
they withdrew when they finally understood what it means.

For Python, it would most likely mean that the only implementations
don't conform to the standard.

Regards,
Martin



From mwh21 at cam.ac.uk  Mon Nov 27 00:22:15 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 26 Nov 2000 23:22:15 +0000
Subject: [Python-Dev] A house upon the sand
In-Reply-To: "Tim Peters"'s message of "Sun, 26 Nov 2000 17:39:42 -0500"
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>
Message-ID: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>

"Tim Peters" <tim.one at home.com> writes:

> On the deprecation of the string module:  where did this idea come from?
> I've never seen anything saying that the string module is deprecated.

I thought this, and went looking.  I found on
http://www.python.org/1.6/, about four fifths of the way down:

Changed Modules

    string - most of this module is deprecated now that strings have
    methods. This no longer uses the built-in strop module, but takes
    advantage of the new string methods to provide transparent support
    for both Unicode and ordinary strings.

I hope (and believe) this is Wrong.

http://www.python.org/2.0/new-python.html says:

    The old string module is still around for backwards compatibility,
    but it mostly acts as a front-end to the new string methods.

which is IMHO better.

Cheers,
M.

-- 
  Every day I send overnight packages filled with rabid weasels to
  people who use frames for no good reason.
                             -- The Usenet Oracle, Oracularity #1017-1




From greg at cosc.canterbury.ac.nz  Mon Nov 27 00:56:22 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 27 Nov 2000 12:56:22 +1300 (NZDT)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>

Michael Hudson <mwh21 at cam.ac.uk>:

> http://www.python.org/2.0/new-python.html says:
>
>    The old string module is still around for backwards
>    compatibility,

This still suggests that continuing to use it is frowned
upon, though.

I think there are still legitimate reasons for using some
parts of the string module. For example, if you're one of
those stubborn people who refuse to accept that ",".join(foo)
is a better way of writing string.join(foo,",").

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fdrake at acm.org  Mon Nov 27 02:00:48 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 26 Nov 2000 20:00:48 -0500 (EST)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>

Greg Ewing writes:
 > This still suggests that continuing to use it is frowned
 > upon, though.

  Backward compatible code is still being written, certainly, and not
everything is available as a method (just try ''.letters! ;).

 > I think there are still legitimate reasons for using some
 > parts of the string module. For example, if you're one of
 > those stubborn people who refuse to accept that ",".join(foo)
 > is a better way of writing string.join(foo,",").

  There will never be an excuse for that!  .join() should never have
been added as a method!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From cgw at fnal.gov  Mon Nov 27 02:08:15 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sun, 26 Nov 2000 19:08:15 -0600 (CST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
	<14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
Message-ID: <14881.46079.317598.101585@buffalo.fnal.gov>

Fred L. Drake, Jr. writes:

 >   There will never be an excuse for that!  .join() should never have
 > been added as a method!

On the other hand it is not clear to me why "capwords" and "zfill" did
not become methods of string objects. 




From fredrik at effbot.org  Mon Nov 27 02:30:10 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 02:30:10 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov>
Message-ID: <005601c05811$95f391f0$3c6340d5@hagrid>

Charles wrote:
> On the other hand it is not clear to me why "capwords" and "zfill" did
> not become methods of string objects. 

fwiw, they were both present in my original unicode
implementation:

>>> from unicode import unicode
>>> a = unicode("hello world")
>>> a.capwords()
'Hello World'
>>> a.zfill(20)
'000000000hello world'

</F>




From fdrake at acm.org  Mon Nov 27 07:00:01 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 01:00:01 -0500 (EST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.46079.317598.101585@buffalo.fnal.gov>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
	<14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
	<14881.46079.317598.101585@buffalo.fnal.gov>
	<005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <14881.63585.563272.169607@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > On the other hand it is not clear to me why "capwords" and "zfill" did
 > not become methods of string objects. 

Fredrik Lundh writes:
 > fwiw, they were both present in my original unicode
 > implementation:

  Interesting.  I'd have expected capwords() on strings.  I wonder why
/F's implementation wasn't retained?  As for zfill(), I can imagine no
one thought it was sufficiently useful to keep around.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at zadka.site.co.il  Mon Nov 27 16:06:42 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 17:06:42 +0200
Subject: [Python-Dev] PEP-0218
Message-ID: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>

Well, another proof of the bugs in the PEP process -- my remarks were
lost, so I'll send them here.

Let me note that almost everything Greg Wilson wants to do can be done
via a Python class implementing a set using a dictionary mapping to None.
Almost?

* No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
* Convertors: if we want list/tuple to have a semblance of efficiency,
  we'll need to cache the element list as a list when accessed by
  index.
* Two different constructors: set() for building from sequence, Set()
  for building from elements. Might be confusing.
* Only possible elements of a set are immutables. OTOH, I'm not sure
  how Greg intends to implement his sets if these sets are allowed 
  to contain mutable elements. 


-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From esr at thyrsus.com  Mon Nov 27 08:30:33 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 02:30:33 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>; from moshez@zadka.site.co.il on Mon, Nov 27, 2000 at 05:06:42PM +0200
References: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <20001127023033.A7616@thyrsus.com>

Moshe Zadka <moshez at zadka.site.co.il>:
> Well, another proof of the bugs in the PEP process -- my remarks were
> lost, so I'll send them here.
> 
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.

I agree.  I wrote a full-featured set library long ago.  All it's
waiting for is rich comparisons.  Wait...are those in 2.0?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

It will be of little avail to the people, that the laws are made by
men of their own choice, if the laws be so voluminous that they cannot
be read, or so incoherent that they cannot be understood; if they be
repealed or revised before they are promulgated, or undergo such
incessant changes that no man, who knows what the law is to-day, can
guess what it will be to-morrow. Law is defined to be a rule of
action; but how can that be a rule, which is little known, and less
fixed?
	-- James Madison, Federalist Papers 62



From moshez at math.huji.ac.il  Mon Nov 27 08:40:47 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 09:40:47 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127023033.A7616@thyrsus.com>
Message-ID: <Pine.GSO.4.10.10011270938120.11335-100000@sundial>

On Mon, 27 Nov 2000, Eric S. Raymond wrote:

> I agree.  I wrote a full-featured set library long ago.  All it's
> waiting for is rich comparisons.  Wait...are those in 2.0?

Nope. I'm not even sure if they're going to be in 2.1. DA?
Anyway, can you at least give the URL? I wrote something like that
too (is this another Python ritual), and I'd be happy to try and
integrate them for 2.1.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov 27 10:32:46 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 10:32:46 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <3A222A3E.D53D6D54@lemburg.com>

Fredrik Lundh wrote:
> 
> Charles wrote:
> > On the other hand it is not clear to me why "capwords" and "zfill" did
> > not become methods of string objects.
> 
> fwiw, they were both present in my original unicode
> implementation:
> 
> >>> from unicode import unicode
> >>> a = unicode("hello world")
> >>> a.capwords()
> 'Hello World'
> >>> a.zfill(20)
> '000000000hello world'

.zfill() is implemented for both strings and Unicode, .capwords()
only for Unicode. Both are disabled, though. I talked with
Guido about these methods and we decided to leave those two
methods disabled in the implementation. They just don't provide
much extra generally useful functionality.

s.capwords() can be emulated with ' '.join(s.capitalize().split()),
BTW.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gvwilson at nevex.com  Mon Nov 27 14:05:23 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:05:23 -0500 (EST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com>

(Note: PEP 0218 is very much a work in progress --- I just wanted to get
some preliminary thoughts down so that my conscience wouldn't nag me quite
so much... :-)

> Well, another proof of the bugs in the PEP process -- my remarks were lost,
> so I'll send them here.

Where did you send your comments?

Thanks,
Greg




From moshez at zadka.site.co.il  Mon Nov 27 22:24:42 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 23:24:42 +0200
Subject: [Python-Dev] PEP-0218 
In-Reply-To: Message from Greg Wilson <gvwilson@nevex.com> 
   of "Mon, 27 Nov 2000 08:05:23 EST." <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com> 
References: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com> 
Message-ID: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>

> > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > so I'll send them here.
> 
> Where did you send your comments?

To you personally?
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From gvwilson at nevex.com  Mon Nov 27 14:30:13 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:30:13 -0500 (EST)
Subject: [Python-Dev] PEP-0218 
In-Reply-To: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>

> > > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > > so I'll send them here.
> > Where did you send your comments?
> To you personally?

Yes, I got those late last week.  Today's election day here in Canada, though,
so the weekend was a write-off.  I'll get to them as soon as I can...

Thanks,

Greg

p.s. emphasis on "day" --- as in, one 24 hour period... :-)




From barry at digicool.com  Mon Nov 27 14:46:14 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 08:46:14 -0500
Subject: [Python-Dev] PEP-0218 
References: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
	<Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>
Message-ID: <14882.26022.860967.54489@anthem.concentric.net>

>>>>> "GW" == Greg Wilson <gvwilson at nevex.com> writes:

    GW> p.s. emphasis on "day" --- as in, one 24 hour period... :-)

Recount, recount!  But tell ya what.  If you have as much problems as
we had, we'll trade you Florida for a province or territory of your
choice.

-Barry



From barry at digicool.com  Mon Nov 27 15:46:40 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:46:40 -0500
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
	<3A1FE6F2.6A612EF9@lemburg.com>
	<3A1FE7FC.AF14EF85@tismer.com>
	<3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <14882.29648.538582.454004@anthem.concentric.net>

>>>>> "M" == M  <mal at lemburg.com> writes:

    M> I think the overall impression of Python 2.0 being a moving
    M> target is a bit overrated: there aren't really all that many
    M> changes which affect existing code, so a Python 1.5.2
    M> introduction will still remain mostly valid even when the
    M> students use Python 2.0.

    M> It is true, that Python 2.0 offers a lot more features than
    M> 1.5.2 and that in some contexts (e.g. database interfacing on
    M> Windows) these new features have made their way into third
    M> party extensions rather fast (e.g. switching from strings to
    M> Unicode). The visible core language hasn't changed much
    M> though...  and this is goodness.

I agree completely.  What I think freaks people out is all the new
features that cropped up in 2.0.  People think they have to learn all
that new fancy stuff like string methods, list comprehensions, and
extended print.  For those who watch python-dev from the sidelines,
pie-in-the-sky discussions on things like continuations freak people
out even more.  There's too much brain-blowing new things being added
to my nice little language!

But it doesn't have to be that way.  As others have pointed out,
almost all the code that worked for 1.5.2 works for 2.0 and 1.5.2 was
released what? 18 months ago?  That's a lot of stability these days.

I understand that adding new stuff to the language has an impact on
slow-update media like books, and that should not be ignored.  But
people don't have to learn (or be taught) the new features to be just
as productive in Python 2.0 as they were in 1.5.2.  They can learn the
new stuff once they've grasped and are comfortable with the old.

2.0 introduced a lot of new stuff, as much because of social reasons
and pressures as for technical benefits.  On the flip side, we've had
messages in this forum decrying the recent dearth of messages.  I
suspect that's partly a reaction to all the changes in 2.0, letting
the new things settle in.  I can't imagine that 2.1 will have nearly
as many new features.

-Barry



From barry at digicool.com  Mon Nov 27 15:52:19 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:52:19 -0500
Subject: [Python-Dev] A house upon the sand
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.29987.132557.43415@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg at cosc.canterbury.ac.nz> writes:

    GE> I think there are still legitimate reasons for using some
    GE> parts of the string module. For example, if you're one of
    GE> those stubborn people who refuse to accept that ",".join(foo)
    GE> is a better way of writing string.join(foo,",").

Actually, an even better way is

COMMA = ','
COMMA.join(foo)

To me, that is substantially easier to read than either of the above
two alternatives.

>>>>> "Fred" == Fred L Drake, Jr <fdrake at acm.org> writes:

    Fred>   There will never be an excuse for that!  .join() should
    Fred> never have been added as a method!

Of course, I completely disagree.  However, I have noticed that I
often define string constants like

SPACE = ' '
EMPTY = ''
NL = '\n'

just so I can write code like

NL.join(headers)
SPACE.join(names)
EMPTY.join(lines)

I doubt it's worth it, but maybe having a standard module called
stringconstants.py with some of the more obvious choices would make
things better?

toowtdi-my-foot-ly y'rs,
-Barry



From guido at python.org  Mon Nov 27 16:21:48 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:21:48 -0500
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 10:32:46 +0100."
             <3A222A3E.D53D6D54@lemburg.com> 
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>  
            <3A222A3E.D53D6D54@lemburg.com> 
Message-ID: <200011271521.KAA29437@cj20424-a.reston1.va.home.com>

> .zfill() is implemented for both strings and Unicode, .capwords()
> only for Unicode. Both are disabled, though. I talked with
> Guido about these methods and we decided to leave those two
> methods disabled in the implementation. They just don't provide
> much extra generally useful functionality.

Exactly.  There's a price to pay for adding methods, and I think these
two are below the threshold.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov 27 16:31:33 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:31:33 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: Your message of "Mon, 27 Nov 2000 09:40:47 +0200."
             <Pine.GSO.4.10.10011270938120.11335-100000@sundial> 
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial> 
Message-ID: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>

[Moshe, on rich comparisons]
> Nope. I'm not even sure if they're going to be in 2.1. DA?

Yes, they should go into 2.1.  Moshe, you are listed as co-author of
PEP 207.  What's up?  Also, I thought you were going to co-author PEP
208 (Reworking the Coercion Model).  What's up with that?  Both are
empty!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gvwilson at nevex.com  Mon Nov 27 17:29:58 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 11:29:58 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>

Hi, folks.  Is there a standard or usual way to provide version
information inside a Python module or class?  Special tokens in
the doc string, a method like "getVersion"...?

Thanks,
Greg



From fdrake at acm.org  Mon Nov 27 17:33:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 11:33:04 -0500 (EST)
Subject: [Python-Dev] versioning standards?
In-Reply-To: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
References: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
	<NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <14882.36032.794185.285527@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Hi, folks.  Is there a standard or usual way to provide version
 > information inside a Python module or class?  Special tokens in
 > the doc string, a method like "getVersion"...?

  Modules sometimes offer a "__version__" attribute, which I've also
seen called "version" or "VERSION" in strange cases (xmllib and (I
think) PIL).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Mon Nov 27 17:42:02 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 17:42:02 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <3A228EDA.56050B52@lemburg.com>

Greg Wilson wrote:
> 
> Hi, folks.  Is there a standard or usual way to provide version
> information inside a Python module or class?  Special tokens in
> the doc string, a method like "getVersion"...?

AFAIK, __version__ with a string value is in common usage both
in modules and classes.

BTW, while we're at it: with the growing number of dependencies
between modules, packages and the Python lib version... how about
creating a standard to enable versioning in module/package imports ?

It would have to meet (at least) these requirements:

* import of a specific version

* alias of the unversioned name to the most recent version
  available

Thoughts ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov 27 19:11:54 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 13:11:54 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: Your message of "Mon, 27 Nov 2000 17:42:02 +0100."
             <3A228EDA.56050B52@lemburg.com> 
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>  
            <3A228EDA.56050B52@lemburg.com> 
Message-ID: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>

> AFAIK, __version__ with a string value is in common usage both
> in modules and classes.

Correct.  This was agreed upon as a standard long ago.  It's probably
not documented anywhere as such.

> BTW, while we're at it: with the growing number of dependencies
> between modules, packages and the Python lib version... how about
> creating a standard to enable versioning in module/package imports ?
> 
> It would have to meet (at least) these requirements:
> 
> * import of a specific version
> 
> * alias of the unversioned name to the most recent version
>   available
> 
> Thoughts ?

This is a major rathole.  I don't know of any other language or system
that has solved this in a satisfactory way.  Most shared library
implementations have had to deal with this, but it's generally a very
painful manual process to ensure compatibility.  I would rather stay
out of trying to solve this for Python in a generic way -- when
specific package develop incompatible versions, it's up to the package
authors to come up with a backwards compatibility strategy.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov 27 19:23:35 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 19:23:35 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>  
	            <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <3A22A6A7.B06F603E@lemburg.com>

Guido van Rossum wrote:
> 
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

Probably should be... along with some other __special_name__
which are in use, e.g. __copyright__ or __author__.

I think it would be a good idea to add these attribute names
as virtually "reserved" attribute names.
 
> > BTW, while we're at it: with the growing number of dependencies
> > between modules, packages and the Python lib version... how about
> > creating a standard to enable versioning in module/package imports ?
> >
> > It would have to meet (at least) these requirements:
> >
> > * import of a specific version
> >
> > * alias of the unversioned name to the most recent version
> >   available
> >
> > Thoughts ?
> 
> This is a major rathole.  I don't know of any other language or system
> that has solved this in a satisfactory way.  Most shared library
> implementations have had to deal with this, but it's generally a very
> painful manual process to ensure compatibility.  I would rather stay
> out of trying to solve this for Python in a generic way -- when
> specific package develop incompatible versions, it's up to the package
> authors to come up with a backwards compatibility strategy.

Well, I think it should at least be possible to install
multiple versions of the same packages in some way which
makes it possible for other programs to choose which version
to take.

With packages and the default import mechanism this is possible
using some caching and __path__ trickery -- I think Pmw does
this. The new "from ... import ... as" also provides ways
for this:

from mx import DateTime160 as DateTime

The problem with the latter approach is that pickles will use the
versioned name and thus won't port easily to later versions. But
you're probably right: only the package itself will know if it's
ok to use the newer version or not.

Would still be nice if we had some kind of documented naming
scheme for versioned modules and packages though, so that the
user will at least recognize these as being versioned import
names.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Mon Nov 27 19:50:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 20:50:10 +0200 (IST)
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011272048300.10126-100000@sundial>

On Mon, 27 Nov 2000, Guido van Rossum wrote:

> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1.  Moshe, you are listed as co-author of
> PEP 207.  What's up?  Also, I thought you were going to co-author PEP
> 208 (Reworking the Coercion Model).  What's up with that?  Both are
> empty!

Ooops, the PEP-0207 thing is a mistake -- I meant to change 0208.
No time now, I'll change 0208 back to DA later tonight. (Sorry 
for being so israelo-centric <wink>). The 0208 PEP should get
done before the weekend. (Your weekend, not mine).

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov 27 20:31:43 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 20:31:43 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
Message-ID: <3A22B69F.D3C78D5C@lemburg.com>

> Date: 2000-Nov-27 10:12
> By: mwh
> 
> Comment:
> I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...

Hehe, that was indeed a trivial patch. What was that about trees
in a forest...

I still think that the PySequence_Fast_GETITEM() macro should at
least include a fall-back type check which causes some exception in
case the used object was not "fixed" using PySequence_Fast() (which
I only noticed in string_join() just now).

Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
used more often :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From paulp at ActiveState.com  Mon Nov 27 21:14:06 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Mon, 27 Nov 2000 12:14:06 -0800
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A22C08E.614A1A0@ActiveState.com>

Mark Lutz wrote:
> 

I read several different issues into your message:

 * standards doc
 * speed of change
 * deprecations
 * backwards incompatibilities

Standards doc: we have one but it is out of date. I think that it would
be great if someone has the bandwidth to improve it but I'm not going to
push Guido very hard to do that because there's a lot of more high
priority stuff I'd like to see him work on.

Speed of change: it wasn't really so radical if you think in terms of
how long these suggestions have been "queued". I would not like it if
Python kept up the same pace of language additions but I don't think
that's going to happen.

Deprecation: deprecations are a good thing. They are the only way that
Python will ever drop old features. If Python never drops old features
then it can only ever grow. If it only ever grows then it will only
become more and more difficult to learn. I don't think I have to justify
the importance of learnability...

Backwards Incompatibilities: I feel strongly that we need to be more
conservative about code breakage. Python 2 was not backwards compatible
enough. It should have deprecated the code and used warnings instead.
Here's my suggested solution for the future:

  http://python.sourceforge.net/peps/pep-0005.html

> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

Sometimes we make mistakes. Making mistakes is part of learning. String
functions should have been methods. We had to correct that mistake. What
other mechanism do we have for correcting mistakes other than
deprecation?

 Paul Prescod



From tim.one at home.com  Mon Nov 27 21:30:04 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 27 Nov 2000 15:30:04 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>

[Barry A. Warsaw]
> ...
> I have noticed that I often define string constants like
>
> SPACE = ' '
> EMPTY = ''
> NL = '\n'
>
> just so I can write code like
>
> NL.join(headers)
> SPACE.join(names)
> EMPTY.join(lines)
> 
> I doubt it's worth it, but maybe having a standard module called
> stringconstants.py with some of the more obvious choices would make
> things better?

-0.  Then I'd expect to see

    from stringconstants import *

at least once, which is at least once too often.  Sick trick:

SPACE, TAB, NL = " \t\n"

sequence-unpacking-is-more-general-than-you-think<wink>-ly y'rs
    - tim




From mwh21 at cam.ac.uk  Mon Nov 27 21:30:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:30:31 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 20:31:43 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com>
Message-ID: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> > Date: 2000-Nov-27 10:12
> > By: mwh
> > 
> > Comment:
> > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> 
> Hehe, that was indeed a trivial patch. What was that about trees
> in a forest...

The way I found it was perhaps instructive.  I was looking at the
function, and thought "that's a bit complicated" so I rewrote it (My
rewrite also seems to be bit quicker so I'll upload it as soon as make
test has finished[*]).  In the course of rewriting it, I saw the line
my patch touched and went "duh!".

> I still think that the PySequence_Fast_GETITEM() macro should at
> least include a fall-back type check which causes some exception in
> case the used object was not "fixed" using PySequence_Fast() (which
> I only noticed in string_join() just now).

It's hard to see how; you're not going to check each invocation of
PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
that PySequence_Fast should raise an exception on being passed a
string or Unicode object... but probably not.

> Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> used more often :-)

Yes.  But they're pretty new, aren't they?  I find them a bit
unsatisfactory that it's not possible to hoist the type check out of
the inner loop.  Still, it seems my PII's branch predictor nails that
one... (i.e. changing it so that it didn't check inside the loop made
naff-all difference to the running time).

Cheers,
M.

[*] which reminds me: test_unicodedata is failing for me at the
    moment.  Anyone else seeing this?  It looks like a python
    regrtest.py -g is all that's needed...

-- 
7. It is easier to write an incorrect program than understand a
   correct one.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From mwh21 at cam.ac.uk  Mon Nov 27 21:38:35 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:38:35 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: Michael Hudson's message of "27 Nov 2000 20:30:31 +0000"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21 at cam.ac.uk> writes:

> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > > 
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > 
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).

It's at

    https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102549&group_id=5470

I want a way of submitting patches by email.  Is roundup in a usable
state yet?

Cheers,
M.

-- 
  "The future" has arrived but they forgot to update the docs. 
                                        -- R. David Murray, 9 May 2000




From mal at lemburg.com  Mon Nov 27 22:08:41 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 22:08:41 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in 
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A22CD59.BC091CAD@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > >
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> >
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).  In the course of rewriting it, I saw the line
> my patch touched and went "duh!".

Yeah. The bug must have sneaked in there when the function was
updated to the PySequence_Fast_* implementation.

BTW, could you also add a patch for the test_string.py and
test_unicode.py tests ?

> > I still think that the PySequence_Fast_GETITEM() macro should at
> > least include a fall-back type check which causes some exception in
> > case the used object was not "fixed" using PySequence_Fast() (which
> > I only noticed in string_join() just now).
> 
> It's hard to see how; you're not going to check each invocation of
> PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> that PySequence_Fast should raise an exception on being passed a
> string or Unicode object... but probably not.

Since not using PySequence_Fast() to initialize the protocol,
I'd suggest doing a Py_FatalError() with some explanatory
text which gets printed to stderr -- still better than a
segfault at some later point due to some dangling pointers...

> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they? 

Yep. Fredrik added them quite late in the 2.0 release process.

> I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.  Still, it seems my PII's branch predictor nails that
> one... (i.e. changing it so that it didn't check inside the loop made
> naff-all difference to the running time).

I think Fredrik speculated on having the compiler optimizers
taking care of the check... hmm, it would probably also help
to somehow declare PyTypeObject slots "const" -- is this possible
in a struct definition ?
 
> Cheers,
> M.
> 
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

Is that for the CVS version or the release version ?
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at effbot.org  Mon Nov 27 22:29:25 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:29:25 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <000f01c058b9$1fc03c60$3c6340d5@hagrid>

Michael Hudson wrote:
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).

your patch changes the behaviour of join on non-standard sequences.

if you use join on a long synthetic sequence, your new version may run
out of memory where the old one worked just fine.

not sure that's a good idea.

-0

</F>




From tim.one at home.com  Mon Nov 27 22:18:35 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 27 Nov 2000 16:18:35 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEIDIBAA.tim.one@home.com>

[Michael Hudson]
> ...
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

All tests pass for me under current CVS, Windows.  More info?




From fredrik at effbot.org  Mon Nov 27 22:35:08 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:35:08 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
Message-ID: <000901c058b9$ec78aee0$3c6340d5@hagrid>

> your patch changes the behaviour of join on non-standard sequences.
> 
> if you use join on a long synthetic sequence, your new version may run
> out of memory where the old one worked just fine.

looking again, this was obviously only true for the unicode version
(which uses PySequence_GetItem), not for the 8-bit string version
(which uses the PySequence_Fast API).

maybe we should change the 8-bit implementation instead?

if not, +1.

</F>




From barry at digicool.com  Mon Nov 27 22:30:06 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:30:06 -0500
Subject: [Python-Dev] A house upon the sand
References: <14882.29987.132557.43415@anthem.concentric.net>
	<LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>
Message-ID: <14882.53854.350972.738517@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim.one at home.com> writes:

    TP> at least once, which is at least once too often.  Sick trick:

    TP> SPACE, TAB, NL = " \t\n"

Oh, that is perversely delicious!  I love it.

super-cali-fragi-listic-expi-ali-chomp-chomp-ly y'rs,
-Barry



From fredrik at effbot.org  Mon Nov 27 22:41:17 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:41:17 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <002701c058ba$c99ced40$3c6340d5@hagrid>

Michael Hudson wrote:
> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they?  I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.

if you can figure out a way to do that with C macros,
I'd love to see it...

on the other hand, PySequence_Fast guarantees that the
returned object is either a list or a tuple, so you can easily
move the test outside the loop yourself:

    if (PyTuple_Check(seq))
        for (...)
            item = PyTuple_GET_ITEM(seq, i)
            ...
    else
        for (...)
            item = PyList_GET_ITEM(seq, i)
            ...

> Still, it seems my PII's branch predictor nails that one... (i.e. changing
> it so that it didn't check inside the loop made naff-all difference to the
> running time).

yeah, the type isn't likely to change inside the loop...

</F>




From barry at digicool.com  Mon Nov 27 22:32:05 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:32:05 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
	<3A22B69F.D3C78D5C@lemburg.com>
	<m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
	<m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <14882.53973.591033.946156@anthem.concentric.net>

>>>>> "MH" == Michael Hudson <mwh21 at cam.ac.uk> writes:

    MH> I want a way of submitting patches by email.

I want a way of submitting bugs by email too.
-Barry



From greg at cosc.canterbury.ac.nz  Tue Nov 28 00:32:55 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 12:32:55 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>

Anyone with an interest in the string functions vs.
string methods debate should read this article, which
was referenced in comp.lang.python recently:

    How Non-Member Functions Improve Encapsulation
    by Scott Meyers
    http://www.cuj.com/archive/1802/feature.html

Mr. Meyers presents some very well-reasoned arguments
against the everything-should-be-a-method mentality.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From mal at lemburg.com  Tue Nov 28 00:45:44 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 00:45:44 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <3A22F228.810F7B29@lemburg.com>

Greg Ewing wrote:
> 
> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Note that the motivation for turning to string methods was
that of migrating from strings to Unicode. Adding Unicode
support to the strop C module would have caused very complicated
code -- methods helped by enabling polymorphic code which is
one of the great advantages of writing software for an interface
rather than an implementation.

Note that functions can make very good use of methods and
thus implement polymorphic functionality -- this is not
about methods vs. functions it's about methods to enable
polymorphic functions.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Tue Nov 28 00:58:52 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 18:58:52 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 12:32:55 +1300."
             <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>

> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Just skimmed it -- seems to be a good argument.  (Also for why
capwords and zfill aren't worth adding as methods. :-)

It would be easy to propose join() as a built-in, and this looks
attractive, if it weren't for the existence of os.path.join().  Some
people probably write ``from os.path import join'' and once join() is
a built-in function, this may be confusing for readers who miss that a
different join is imported.  (I'm not saying that it breaks existing
code -- just that it's confusing to have two joins.)  But maybe this
is a moot argument since string.join() is already a function.

I wonder what Java enthusiasts would say after reading Meyers'
article: there *are* no non-member non-friend functions in Java.  The
best approximation is static methods, but they still live in a
class...

--Guido van Rossum (home page: http://www.python.org/~guido/)




From barry at digicool.com  Tue Nov 28 01:10:56 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:10:56 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
	<200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.63504.970688.304915@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg at cosc.canterbury.ac.nz> writes:

    GE> Anyone with an interest in the string functions vs.
    GE> string methods debate should read this article, which
    GE> was referenced in comp.lang.python recently:

    |     How Non-Member Functions Improve Encapsulation
    |     by Scott Meyers
    |     http://www.cuj.com/archive/1802/feature.html

    GE> Mr. Meyers presents some very well-reasoned arguments
    GE> against the everything-should-be-a-method mentality.

Remember that Meyers is talking about C++ here, where there's a
distinction in the `friendliness' of a function.  He himself says that
the choice is between non-member non-friend functions and member
functions (i.e. methods).  Friend functions have the same dependency
on implementation changes as methods.

In Python, there is no such friend distinction, although you can come
close to faking it with private names.  So the argument based on
degrees of encapsulation doesn't hold water in Python.

in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,
-Barry



From guido at python.org  Tue Nov 28 01:07:38 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:07:38 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 00:45:44 +0100."
             <3A22F228.810F7B29@lemburg.com> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
            <3A22F228.810F7B29@lemburg.com> 
Message-ID: <200011280007.TAA30005@cj20424-a.reston1.va.home.com>

> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

Of course.  Meyers starts by saying that if it needs implementation
details it must be a method.  This is true (if only for efficiency
reasons) for most string methods.

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

Meyers also says that if it needs to be virtual it needs to be a
method.  Polymorphism is roughly equivalent to virtual in this
context, and this alone justifies the move to methods.

But join() is special: it is polymorphic in two arguments, and making
it a method of the separator argument doesn't help.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at effbot.org  Tue Nov 28 01:24:42 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 01:24:42 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com>
Message-ID: <042401c058d1$9b61f030$3c6340d5@hagrid>

mal wrote:
> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

as Alex pointed out on comp.lang.python, the first if-clause
in Meyers algorithm is:

    if (f needs to be virtual)
       make f a member function of C;

which probably covers the join case pretty well...

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

a bit further down, the algorithm goes:

    else if (f can be implemented via C's
             public interface)
       make f a non-member function;

which covers all remaining non-trivial functions in the string
module...

(okay, you could get around this by renaming the method to
something a bit more annoying; let's say the "public interface"
contains a virtual method called "use_myself_to_join_sequence",
in which "join" could be a non-member convenience function
using nothing but the public interface.  but that's just silly...)

:::

fwiw, I still think that Meyers article cannot be fully applied
to Python, since he's ignoring the case:

    else if (f can be implemented via C's
             public interface, but may have
             to be overridden by a subclass
             to C)
       make f a member function;

this isn't a problem in a language that supports function over-
loading (see the interfaces/packages section in meyers article),
but as we all know, Python doesn't...

</F>




From barry at digicool.com  Tue Nov 28 01:17:32 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:17:32 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
	<3A22F228.810F7B29@lemburg.com>
	<200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <14882.63900.127012.804027@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> But join() is special: it is polymorphic in two arguments,
    GvR> and making it a method of the separator argument doesn't
    GvR> help.

What about calling the built-in strjoin().  That avoids the conflict
with os.path.join() and strengthens the connection with the intended
behavior -- that the return value is a string and the elements are
str()-ified.

or-use-sjoin()-ly y'rs,
-Barry



From guido at python.org  Tue Nov 28 01:12:01 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:12:01 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 19:10:56 EST."
             <14882.63504.970688.304915@anthem.concentric.net> 
References: <14882.29987.132557.43415@anthem.concentric.net> <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
            <14882.63504.970688.304915@anthem.concentric.net> 
Message-ID: <200011280012.TAA30062@cj20424-a.reston1.va.home.com>

> Remember that Meyers is talking about C++ here, where there's a
> distinction in the `friendliness' of a function.  He himself says that
> the choice is between non-member non-friend functions and member
> functions (i.e. methods).  Friend functions have the same dependency
> on implementation changes as methods.
> 
> In Python, there is no such friend distinction, although you can come
> close to faking it with private names.  So the argument based on
> degrees of encapsulation doesn't hold water in Python.
> 
> in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,

I disagree.  While non-methods *can* look inside an implementation,
it's still implied that they break encapsulation if they do.  (This
may explain why you like to use private names -- they make the
encapsulation explicit.  For me, it exists even when it is implicit.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Tue Nov 28 01:26:25 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:26:25 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
	<200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
	<14882.63504.970688.304915@anthem.concentric.net>
	<200011280012.TAA30062@cj20424-a.reston1.va.home.com>
Message-ID: <14882.64433.691041.119209@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> I disagree.  While non-methods *can* look inside an
    GvR> implementation, it's still implied that they break
    GvR> encapsulation if they do.  (This may explain why you like to
    GvR> use private names -- they make the encapsulation explicit.
    GvR> For me, it exists even when it is implicit.)

True, but it still means that changing the implementation has an
unknown effect on existing code, because there /could/ be a lot of
breakage.  That can be (and has been! :) written off as the fault of
the 3rd party author, but still, it isn't enforcible like it is in
C++, where, if it ain't a friend, it's not going to break.  In C++ you
can count the number of functions that will break to determine the
degree of encapsulation, but you can't in Python.

Witness the recent change to cgi.FieldStorage.  We removed the
self.lines attribute because it was a memory bloat, and we /thought/
that it wouldn't affect existing code, but there was no way to really
know for sure because the intent of the cgi module author wasn't at
all clear.  Was self.lines intended to be public but just accidentally
undocumented?  Was it a debugging tool?

As an aside, this argument's not really true with built-in types such
as strings because the internal implementation isn't publically
available to the Python programmer.

-Barry



From greg at cosc.canterbury.ac.nz  Tue Nov 28 01:24:07 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:24:07 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>
Message-ID: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz>

Guido said:

> It would be easy to propose join() as a built-in, and this looks
> attractive, if it weren't for the existence of os.path.join().

and

> join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

How about an operator, then?

We already have % rather than a string.format or some such.
Presumably this is because Guido thought it was such a handy
thing that it should be instantly available at our fingertips.
I think a similar argument could be made for string.join, and
also its inverse string.split.

So, how about:

   x & s   == string.join(x, s)

   s1 | s2 == string.split(s1, s2)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Tue Nov 28 01:31:35 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:31:35 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <042401c058d1$9b61f030$3c6340d5@hagrid>
Message-ID: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>

Fredrik Lundh <fredrik at effbot.org>:

> he's ignoring the case:
>
>    else if (f can be implemented via C's
>             public interface, but may have
>             to be overridden by a subclass
>             to C)

Isn't that the same as "f needs to be virtual"?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fredrik at effbot.org  Tue Nov 28 02:11:48 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 02:11:48 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>
Message-ID: <044a01c058d8$3144b320$3c6340d5@hagrid>

Greg Ewing wrote:
> > he's ignoring the case:
> >
> >    else if (f can be implemented via C's
> >             public interface, but may have
> >             to be overridden by a subclass
> >             to C)
> 
> Isn't that the same as "f needs to be virtual"?

in Python, sure.

in C++, it depends.  thanks to function overloading, you can have
polymorphism without having to make everything virtual...

</F>




From guido at python.org  Tue Nov 28 02:22:07 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 20:22:07 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 13:24:07 +1300."
             <200011280024.NAA03278@s454.cosc.canterbury.ac.nz> 
References: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011280122.UAA30268@cj20424-a.reston1.va.home.com>

> How about an operator, then?
> 
> We already have % rather than a string.format or some such.
> Presumably this is because Guido thought it was such a handy
> thing that it should be instantly available at our fingertips.
> I think a similar argument could be made for string.join, and
> also its inverse string.split.
> 
> So, how about:
> 
>    x & s   == string.join(x, s)
> 
>    s1 | s2 == string.split(s1, s2)

I don't see the mnemonic.  (For %, the mnemonic is clear: %s, %d etc.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From esr at thyrsus.com  Tue Nov 28 02:25:03 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 20:25:03 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 10:31:33AM -0500
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial> <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <20001127202503.A1013@thyrsus.com>

Guido van Rossum <guido at python.org>:
> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1. 

Good.  Within a few hours after I see that feature in the beta I'll
have a very rich Set class for the standard library.  It includes all
the standard boolean operations, symmetric difference, powerset, and
useful overloads for most of the standard operators.  As the header
comment says:

# A set-algebra module for Python
#
# The functions work on any sequence type and return lists.
# The set methods can take a set or any sequence type as an argument.

Most of the code is tested already.  All it's waiting on is rich comparisons
so I can handle partial-ordering of sets properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

When only cops have guns, it's called a "police state".
        -- Claire Wolfe, "101 Things To Do Until The Revolution" 



From jeremy at alum.mit.edu  Tue Nov 28 03:10:17 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 27 Nov 2000 21:10:17 -0500 (EST)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <000f01c058b9$1fc03c60$3c6340d5@hagrid>
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
	<3A22B69F.D3C78D5C@lemburg.com>
	<m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
	<m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
	<000f01c058b9$1fc03c60$3c6340d5@hagrid>
Message-ID: <14883.5129.304362.407763@bitdiddle.concentric.net>

test_unicodedata is failing for me on Linux.  Fredrik, you made a
change on Nov. 3 that changed test/output/test_unicodedata but not
test/test_unicodedata.pu.  Did you verify that the test suite worked
then?

Jeremy



From mwh21 at cam.ac.uk  Tue Nov 28 09:52:00 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 28 Nov 2000 08:52:00 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in   <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 22:08:41 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> Michael Hudson wrote:
> > 
> > "M.-A. Lemburg" <mal at lemburg.com> writes:
> > 
> > > > Date: 2000-Nov-27 10:12
> > > > By: mwh
> > > >
> > > > Comment:
> > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > >
> > > Hehe, that was indeed a trivial patch. What was that about trees
> > > in a forest...
> > 
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).  In the course of rewriting it, I saw the line
> > my patch touched and went "duh!".
> 
> Yeah. The bug must have sneaked in there when the function was
> updated to the PySequence_Fast_* implementation.
> 
> BTW, could you also add a patch for the test_string.py and
> test_unicode.py tests ?

Here's an effort, but it's a memory scribbling bug.  If I change
test_unicode.py thus:

Index: test_unicode.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
retrieving revision 1.22
diff -c -r1.22 test_unicode.py
*** test_unicode.py     2000/10/23 17:22:08     1.22
--- test_unicode.py     2000/11/28 08:49:11
***************
*** 70,76 ****
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = 'wxyz'
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
--- 70,76 ----
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
***************
*** 78,83 ****
--- 78,87 ----
  test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
  test('join', u' ', u'w x y z', Sequence())
  test('join', u' ', TypeError, 7)
+ test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
+ test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
+ test('join', ' ', u'w x y z', Sequence())
+ test('join', ' ', TypeError, 7)
  
  class BadSeq(Sequence):
      def __init__(self): self.seq = [7, u'hello', 123L]

and back out the fix for the join bug, this happens:

...
...
Testing Unicode formatting strings... done.
Testing builtin codecs...
Traceback (most recent call last):
  File "test_unicode.py", line 378, in ?
    assert unicode('hello','utf8') == u'hello'
  File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
 ?
    import codecs,aliases
SystemError: compile.c:185: bad argument to internal function
Segmentation fault

i.e. it crashes miles away from the problem.

I'll reply to the other stuff later - no time now.

Cheers,
M.

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                         and quoted by Aahz Maruch on comp.lang.python




From mal at lemburg.com  Tue Nov 28 10:17:29 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:17:29 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in   
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com> <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A237829.EA13903D@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > Michael Hudson wrote:
> > >
> > > "M.-A. Lemburg" <mal at lemburg.com> writes:
> > >
> > > > > Date: 2000-Nov-27 10:12
> > > > > By: mwh
> > > > >
> > > > > Comment:
> > > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > > >
> > > > Hehe, that was indeed a trivial patch. What was that about trees
> > > > in a forest...
> > >
> > > The way I found it was perhaps instructive.  I was looking at the
> > > function, and thought "that's a bit complicated" so I rewrote it (My
> > > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > > test has finished[*]).  In the course of rewriting it, I saw the line
> > > my patch touched and went "duh!".
> >
> > Yeah. The bug must have sneaked in there when the function was
> > updated to the PySequence_Fast_* implementation.
> >
> > BTW, could you also add a patch for the test_string.py and
> > test_unicode.py tests ?
> 
> Here's an effort, but it's a memory scribbling bug.  If I change
> test_unicode.py thus:
> 
> Index: test_unicode.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
> retrieving revision 1.22
> diff -c -r1.22 test_unicode.py
> *** test_unicode.py     2000/10/23 17:22:08     1.22
> --- test_unicode.py     2000/11/28 08:49:11
> ***************
> *** 70,76 ****
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = 'wxyz'
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> --- 70,76 ----
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> ***************
> *** 78,83 ****
> --- 78,87 ----
>   test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
>   test('join', u' ', u'w x y z', Sequence())
>   test('join', u' ', TypeError, 7)
> + test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
> + test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
> + test('join', ' ', u'w x y z', Sequence())
> + test('join', ' ', TypeError, 7)
> 
>   class BadSeq(Sequence):
>       def __init__(self): self.seq = [7, u'hello', 123L]
> 
> and back out the fix for the join bug, this happens:
> 
> ...
> ...
> Testing Unicode formatting strings... done.
> Testing builtin codecs...
> Traceback (most recent call last):
>   File "test_unicode.py", line 378, in ?
>     assert unicode('hello','utf8') == u'hello'
>   File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
>  ?
>     import codecs,aliases
> SystemError: compile.c:185: bad argument to internal function
> Segmentation fault
> 
> i.e. it crashes miles away from the problem.

The test is only supposed to assure that we don't trip again. It's
not intended to work in some way *before* applying your patch.

I always try to integrate tests for bugs into the test suites
for my mx stuff and AFAICTL this also seems to be the Python
dev style.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Tue Nov 28 10:21:30 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:21:30 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
	            <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <3A23791A.9C5ACD60@lemburg.com>

Guido van Rossum wrote:
> 
> But join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

join() is special indeed, but what about the semantics we talked
about last year (?)...

join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]

This should fit all uses of join() (accept maybe os.path.join).

How about naming the beast concat() with sep defaulting to '' to
avoid the problems with os.path.join() ?!


-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at effbot.org  Tue Nov 28 10:54:31 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 10:54:31 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com><3A22B69F.D3C78D5C@lemburg.com><m3bsv16tug.fsf@atrus.jesus.cam.ac.uk><m38zq56th0.fsf@atrus.jesus.cam.ac.uk><000f01c058b9$1fc03c60$3c6340d5@hagrid> <14883.5129.304362.407763@bitdiddle.concentric.net>
Message-ID: <00a401c05923$73ca2950$3c6340d5@hagrid>

Jeremy wrote:

> test_unicodedata is failing for me on Linux.  Fredrik, you made a
> change on Nov. 3 that changed test/output/test_unicodedata but not
> test/test_unicodedata.pu.  Did you verify that the test suite worked
> then?

I added 38,000 missing characters to the unicode database.
that didn't change the test, only the test output...

the november 3 checkin included the following files:

    Lib/test/output/test_unicodedata
    Modules/unicodedata_db.h
    Objects/unicodetype_db.h
    Tools/unicode/makeunicodedata.py

:::

hmm.  looks like the makefile dependencies haven't been
updated (is this done by hand on Unix???)

in Modules/Makefile.pre.in

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h

should be

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h unicodedata_db.h

and in Objects/Makefile.in

    unicodectype.o: unicodectype.c

should be:

    unicodectype.o: unicodectype.c unicodectype_db.h
    
</F>




From mwh21 at cam.ac.uk  Tue Nov 28 11:19:14 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:19:14 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
In-Reply-To: <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>

On Mon, 27 Nov 2000, M.-A. Lemburg wrote:

> Michael Hudson wrote:
> 
> > > I still think that the PySequence_Fast_GETITEM() macro should at
> > > least include a fall-back type check which causes some exception in
> > > case the used object was not "fixed" using PySequence_Fast() (which
> > > I only noticed in string_join() just now).
> > 
> > It's hard to see how; you're not going to check each invocation of
> > PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> > that PySequence_Fast should raise an exception on being passed a
> > string or Unicode object... but probably not.
> 
> Since not using PySequence_Fast() to initialize the protocol,
> I'd suggest doing a Py_FatalError() with some explanatory
> text which gets printed to stderr -- still better than a
> segfault at some later point due to some dangling pointers...

So you'd want PySequence_Fast_GETITEM to look like

#define PySequence_Fast_GETITEM(s,i) \
    (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
          (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
                  Py_FatalError("muffin!") ))

?  That'd probably be fair - you'd want to check any performance hit, but
I wouldn't be surprised if it was neglible.

> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they? 
> 
> Yep. Fredrik added them quite late in the 2.0 release process.
> 
> > I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.  Still, it seems my PII's branch predictor nails that
> > one... (i.e. changing it so that it didn't check inside the loop made
> > naff-all difference to the running time).
> 
> I think Fredrik speculated on having the compiler optimizers
> taking care of the check... 

gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like

.stabn 68,0,825,.LM312-string_join
.LM312:
        movl PyList_Type at GOT(%ebx),%eax
        cmpl %eax,4(%edi)
        jne .L875
        movl 12(%edi),%eax
        jmp .L898
        .p2align 4,,7
.L875:
        leal 12(%edi),%eax
.L898:
        movl -32(%ebp),%edx
        movl (%eax,%edx,4),%esi

... but I'm no expert in assembly reading.

> hmm, it would probably also help
> to somehow declare PyTypeObject slots "const" -- is this possible
> in a struct definition ?

I have no idea!

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:35:57 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:35:57 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <002701c058ba$c99ced40$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281025490.11335-100000@localhost.localdomain>

On Mon, 27 Nov 2000, Fredrik Lundh wrote:

> Michael Hudson wrote:
> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they?  I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.
> 
> if you can figure out a way to do that with C macros,
> I'd love to see it...

No.  You can consider my above comment as a round-about way of saying "I
hate C" :-)

> on the other hand, PySequence_Fast guarantees that the
> returned object is either a list or a tuple, so you can easily
> move the test outside the loop yourself:
> 
>     if (PyTuple_Check(seq))
>         for (...)
>             item = PyTuple_GET_ITEM(seq, i)
>             ...
>     else
>         for (...)
>             item = PyList_GET_ITEM(seq, i)
>             ...

But doing that would slightly eliminate the point of the PySequenceFast*
API wouldn't it?  Rhetorical question; it doesn't really seem to matter.

Slightly less rhetorical question: string_join currently just assumes a
list of strings.  But it is conceivable that one day it might be changed
to call __str__ on instances it finds, and then you'd have the issue of
__str__ methods changing the length of the list you're iterating over
(this might more plausibly apply in other situations where you might want
to use PySequence_Fast but I haven't gone looking for them). So would it
be feasible to change PySequence_Fast to smash the type of a list it's
given with the "immutable_list_type" from listobject.c and then have a
PySequence_Fast_End function that would decref the sequence and un-smash
the type as appropriate?

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:42:10 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:42:10 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <00a401c05923$73ca2950$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281036320.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, Fredrik Lundh wrote:

> Jeremy wrote:
> 
> > test_unicodedata is failing for me on Linux.  Fredrik, you made a
> > change on Nov. 3 that changed test/output/test_unicodedata but not
> > test/test_unicodedata.pu.  Did you verify that the test suite worked
> > then?
> 
> I added 38,000 missing characters to the unicode database.
> that didn't change the test, only the test output...
> 
> the november 3 checkin included the following files:
> 
>     Lib/test/output/test_unicodedata
>     Modules/unicodedata_db.h
>     Objects/unicodetype_db.h
>     Tools/unicode/makeunicodedata.py
> 
> :::
> 
> hmm.  looks like the makefile dependencies haven't been
> updated (is this done by hand on Unix???)

I think so.  I've done a complete rebuild and the test now passes (I was
sure I'd done one of those more recently than Nov 3...).  So, false alarm,
sorry.

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:43:49 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:43:49 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in    <stringobject>.join(<unicodestring>)
In-Reply-To: <3A237829.EA13903D@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, M.-A. Lemburg wrote:

> The test is only supposed to assure that we don't trip again. It's
> not intended to work in some way *before* applying your patch.

Yes.  I don't know what I was thinking earlier.  Too many beers last
night, perhaps.

> I always try to integrate tests for bugs into the test suites
> for my mx stuff and AFAICTL this also seems to be the Python
> dev style.

Do you want to check it in then?  I can't.

Cheers,
M.




From mal at lemburg.com  Tue Nov 28 12:07:22 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:07:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in    
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>
Message-ID: <3A2391EA.133A2CEF@lemburg.com>

Michael Hudson wrote:
> 
> On Tue, 28 Nov 2000, M.-A. Lemburg wrote:
> 
> > The test is only supposed to assure that we don't trip again. It's
> > not intended to work in some way *before* applying your patch.
> 
> Yes.  I don't know what I was thinking earlier.  Too many beers last
> night, perhaps.
> 
> > I always try to integrate tests for bugs into the test suites
> > for my mx stuff and AFAICTL this also seems to be the Python
> > dev style.
> 
> Do you want to check it in then?  I can't.

Best is to put it up on SF as patch. Someone will then pick it up and
check it in sooner or later. In any case that'll prevent it
getting lost.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Tue Nov 28 12:12:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:12:56 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in  
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <3A239338.800E84E8@lemburg.com>

Michael Hudson wrote:
> 
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair - you'd want to check any performance hit, but
> I wouldn't be surprised if it was neglible.

Yes.
 
> [Fredrik's PySequence_Fast_* APIs]
> >
> > I think Fredrik speculated on having the compiler optimizers
> > taking care of the check...
> 
> gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like
> 
> .stabn 68,0,825,.LM312-string_join
> .LM312:
>         movl PyList_Type at GOT(%ebx),%eax
>         cmpl %eax,4(%edi)
>         jne .L875
>         movl 12(%edi),%eax
>         jmp .L898
>         .p2align 4,,7
> .L875:
>         leal 12(%edi),%eax
> .L898:
>         movl -32(%ebp),%edx
>         movl (%eax,%edx,4),%esi
> 
> ... but I'm no expert in assembly reading.
> 
> > hmm, it would probably also help
> > to somehow declare PyTypeObject slots "const" -- is this possible
> > in a struct definition ?
> 
> I have no idea!

Me neither -- it could help the compiler when moving type
checks out of inner loops: the type slot of PyObject normally
doesn't change (except on Windows where you have to patch it
in order make VC++ happy).

Anyone got a working idea in this direction ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Tue Nov 28 12:32:22 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2000 12:32:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <012c01c0592e$e0b463e0$0900a8c0@SPIFF>

michael wrote:
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair

sorry, but that's just silly -- the function is called GET_ITEM,
not GetItem.

it's no less safe than PyList_GET_ITEM or PyTuple_GET_ITEM or
any other "trade speed for safety" macro.

</F>




From guido at python.org  Tue Nov 28 13:13:57 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 07:13:57 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 10:21:30 +0100."
             <3A23791A.9C5ACD60@lemburg.com> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>  
            <3A23791A.9C5ACD60@lemburg.com> 
Message-ID: <200011281213.HAA02449@cj20424-a.reston1.va.home.com>

> join() is special indeed, but what about the semantics we talked
> about last year (?)...
> 
> join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> 
> This should fit all uses of join() (accept maybe os.path.join).

This is much more general than the current definition --
e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
the definition this much.

> How about naming the beast concat() with sep defaulting to '' to
> avoid the problems with os.path.join() ?!

Hm... if we can stick to the string semantics this would be okay.  But
we'd lose the symmetry of split/join.  Note that string.join has a
default separator of ' ' to roughly match the default behavoir of
split.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Tue Nov 28 16:10:51 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 10:10:51 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: Your message of "26 Nov 2000 23:22:15 GMT."
             <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> 
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>  
            <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>

----------
My feelings about this topic have been expressed adequately by others.

I'm not sure what to say about the code that worked in 1.5.2 but broke
in 2.0 -- if these were book examples, I suspect that the book was
probably using undocumented (if commonly seen) features, line
multi-argument append() or connect().

As someone else said:  Change happens.  Get over it. :-)

> "Tim Peters" <tim.one at home.com> writes:
> 
> > On the deprecation of the string module:  where did this idea come from?
> > I've never seen anything saying that the string module is deprecated.

Michael Hudson:

> I thought this, and went looking.  I found on
> http://www.python.org/1.6/, about four fifths of the way down:
> 
> Changed Modules
> 
>     string - most of this module is deprecated now that strings have
>     methods. This no longer uses the built-in strop module, but takes
>     advantage of the new string methods to provide transparent support
>     for both Unicode and ordinary strings.
> 
> I hope (and believe) this is Wrong.

Because of its importance, the deprecation time of the string module
will be longer than that of most deprecated modules.  I expect it
won't be removed until Python 3000.

> http://www.python.org/2.0/new-python.html says:
> 
>     The old string module is still around for backwards compatibility,
>     but it mostly acts as a front-end to the new string methods.
> 
> which is IMHO better.

Yes.

And Greg Ewing:

> I think there are still legitimate reasons for using some
> parts of the string module. For example, if you're one of
> those stubborn people who refuse to accept that ",".join(foo)
> is a better way of writing string.join(foo,",").

This has been discussed -- jut note that continuing to use the string
module *is* frowned upon, and such stubborn code will get its just
desserts when Py3K arrives.

I suggest adding the following to the string module's documentation
(rather than marking it as explicitly deprecated):

    This module exists for backwards compatibility only.  It will
    eventually be deprecated and its use should be avoided in new code.

I also suggest that someone go through the standard library and get
rid of all uses of the string module.  (What to do with string.letters
c.s.?)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Tue Nov 28 16:22:18 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 16:22:18 +0100
Subject: [Python-Dev] A house upon the sand
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>  
	            <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <3A23CDAA.F14E0011@lemburg.com>

Guido van Rossum wrote:
> 
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
> 
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new code.
> 
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

We will need some place to put constants and other non-method
based string related functions (much like the Unicode 
database module which serves a similar purpose)... 
the ideal complement would be a stringdata module which provides
the same interfaces as the unicodedata module if reasonably 
possible (we'd get into encoding problems again...) !

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Tue Nov 28 15:33:38 2000
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 28 Nov 2000 16:33:38 +0200
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>  
	            <3A23791A.9C5ACD60@lemburg.com> <200011281213.HAA02449@cj20424-a.reston1.va.home.com>
Message-ID: <3A23C242.C24BE5D9@tismer.com>


Guido van Rossum wrote:
> 
> > join() is special indeed, but what about the semantics we talked
> > about last year (?)...
> >
> > join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> >
> > This should fit all uses of join() (accept maybe os.path.join).
> 
> This is much more general than the current definition --
> e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
> the definition this much.

No, if n is the length of the sequence, the above definition
would calculate 10 and then raise IndexError :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From akuchlin at mems-exchange.org  Tue Nov 28 23:17:35 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 17:17:35 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282213.OAA31146@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Tue, Nov 28, 2000 at 02:13:47PM -0800
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
Message-ID: <20001128171735.A21996@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
>229
>is not sufficiently worked out and doesn't seem a priority.  (This can
>still make it into 2.1 if the author does the work.)

I was waiting for further comments on the idea before actually doing
the work.  Anyone see gaping holes in the basic idea?  (I *can* just
charge ahead and work on it anyway...)

--amk



From guido at python.org  Wed Nov 29 00:01:38 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 18:01:38 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: Your message of "Tue, 28 Nov 2000 17:17:35 EST."
             <20001128171735.A21996@kronos.cnri.reston.va.us> 
References: <200011282213.OAA31146@slayer.i.sourceforge.net>  
            <20001128171735.A21996@kronos.cnri.reston.va.us> 
Message-ID: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>

> On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
> >229
> >is not sufficiently worked out and doesn't seem a priority.  (This can
> >still make it into 2.1 if the author does the work.)

Andrew:
> I was waiting for further comments on the idea before actually doing
> the work.  Anyone see gaping holes in the basic idea?  (I *can* just
> charge ahead and work on it anyway...)

I'd say charge ahead, but check where you are every once in a while.
Some things I'm not sure about include:

- Always shared libs.  What about Unixish systems that don't have
  shared libs?  What if you just want something to be hardcoded as
  statically linked, e.g. for security reasons?  (On the other hand
  there are some reasons why defaulting to shared libs is a good
  idea -- e.g. it makes it easier to upgrade a single extension module
  in an installation, e.g. for a critical fix.)

- Doesn't distutils currently always look for an existing installed
  Python installation to get its Makefile info?  How to force it to
  use the info from the build tree?

BTW off-topic: I didn't demote your PEP 222 (Web Library Enhancements)
but I'm pretty neutral on this -- except that it's probably a good
idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
than making the existing cgi.py even more complicated.  (Also, it's
very fragile -- you never know who uses what mis-feature.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Wed Nov 29 00:21:44 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 28 Nov 2000 18:21:44 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
	<20001128171735.A21996@kronos.cnri.reston.va.us>
	<200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <14884.15880.264559.125483@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > idea to start from scratch with a new cgi-ish module (cgi2.py?) rather

  Or it could be called "cgilib"...


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From jeff at ollie.clive.ia.us  Wed Nov 29 00:42:27 2000
From: jeff at ollie.clive.ia.us (Jeffrey C. Ollie)
Date: Tue, 28 Nov 2000 17:42:27 -0600
Subject: [Python-Dev] Python embedded in Exim
Message-ID: <20001128174227.A24208@ollie.clive.ia.us>

Well, I had an epiphany over the US Thanksgiving holiday weekend (or
maybe it was a brain aneurism) so I've spent some late nights and some
time stolen from work and come up with a first pass at embedding
Python in Exim.

I imagine that embedding Python in Exim will be interesting to those
folks writing virus scanners or for VERY tight integration of Mailman
with Exim.  You can also do all of those things that you can do with
the Perl string expansions.

Note: this code is VERY lightly tested. It compiles on my home
hacked-up RedHat Linux system and works on a couple of simple tests
that I've done, but no real e-mails have passed through this code yet.
I'm tossing this out for public comment on the design and
implementation...  It's been a while since I've done a lot of C
coding.

Unfortunately, you must embed Python 2.0.  This is because previous
versions of Python used a hacked copy of PCRE as it's regular
expression engine that conflicts with the copy of PCRE in Exim.

Starting in Python 2.0 the default regular expression is a completely
new one called SRE that supports Unicode.  The PCRE-based regular
expression engine is still included in Python 2.0 but I get around
that by creating a private copy of the Python library and delete the
offending object modules.  You could do that in versions of Python
prior to 2.0 but a lot of the standard library depends on regular
expressions I didn't think that there was much point in trying to make
the embedding work with 1.5.2 or 1.6.

Well, on to the patch itself.  The patch is made against Exim v3.20.
After you've patched the source code, you need to set four variables
in the Local/Makefile:

EXIM_PYTHON=python.o
PYTHON_INC=-I/usr/local/include/python2.0
PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a
PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil

Then build Exim as usual.

There are three runtime directives that control the embedding, all of which
are optional:

python_at_start		boolean - force startup of Python interpreter
python_module_paths	colon separated list of paths to append to sys.path
python_initial_import   colon separated list of modules to import at
                        interpreter initialization time

There are also two new command line switches -ys and -yd that will
force an immediate startup of Python or delay startup, overriding
python_at_start.

Then you use it:

${python{<module>.<function>}{<arg1>}...}

You must specify the module name and the function name.  The named
module will be imported automatically.  And currently you must specify
a function that returns a string (I'll look into adding accessing
attributes, methods of classes, and converting non-strings into
strings).  There can be up to eight arguments.  Each argument will be
expanded by Exim before it's passed to the Python interpreter.

Your python code has access to a module called "exim", which currently
defines two things:

exim.expand_string(s) which calls back to Exim to expand a string and
exim.error            which is an exception object used by expand_string

One simple example:

${python{string.upper}{$local_part}}

Error reporting isn't the best right now, I'm going to look into
formatting exceptions and tracebacks for more meaningful error
messages.

Well, I think that's the gist of it.  I'll work on patches to the
documentation when this thing becomes more stable.  As I said before
I've only lightly tested this... beware!

Comments, criticisms, suggestions, flames welcome...

Jeff

-------------- next part --------------
Index: exim/OS/Makefile-Base
diff -u exim/OS/Makefile-Base:1.1.1.1 exim/OS/Makefile-Base:1.1.1.1.2.3
--- exim/OS/Makefile-Base:1.1.1.1	Mon Nov 27 08:18:15 2000
+++ exim/OS/Makefile-Base	Tue Nov 28 16:49:25 2000
@@ -192,6 +192,16 @@
 	@chmod a+x ../util/convert4r3
 	@echo ">>> convert4r3 script built in util directory"; echo ""
 
+libpython.a: $(PYTHON_LIB)
+	if [ -n $(PYTHON_LIB) ] ;\
+	then \
+		cp $(PYTHON_LIB) libpython.a ;\
+		ar d libpython.a pcremodule.o pypcre.o ;\
+		ranlib libpython.a ;\
+	else \
+		ar cq libpython.a ;\
+		ranlib libpython.a ;\
+	fi
 
 # Targets for final binaries; the main one has a build number which is
 # updated each time. We don't bother with that for the auxiliaries.
@@ -201,11 +211,12 @@
         header.o host.o log.o match.o moan.o os.o parse.o queue.o \
         readconf.o retry.o rewrite.o \
         route.o search.o smtp_in.o smtp_out.o spool_in.o spool_out.o \
-        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PERL)
+        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PERL) \
+	$(EXIM_PYTHON)
 
 exim:   libident/libident.a pcre/libpcre.a lookups/lookups.a auths/auths.a \
         directors/directors.a routers/routers.a transports/transports.a \
-        $(OBJ_EXIM) version.c
+        $(OBJ_EXIM) version.c libpython.a
 	awk '{ print ($$1+1) }' cnumber.h > cnumber.temp
 	/bin/rm -f cnumber.h; mv cnumber.temp cnumber.h
 	$(CC) -c $(CFLAGS) $(INCLUDE) $(IPV6_INCLUDE) $(TLS_INCLUDE) version.c
@@ -215,7 +226,8 @@
 	  routers/routers.a transports/transports.a lookups/lookups.a \
 	  auths/auths.a \
 	  $(LIBS) $(LIBS_EXIM) $(IPV6_LIBS) $(EXTRALIBS) $(EXTRALIBS_EXIM) \
-	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) $(TLS_LIBS)
+	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) libpython.a $(PYTHON_EXTRA_LIBS) \
+	  $(TLS_LIBS)
 	$(EXIM_CHMOD)
 	@echo " "
 	@echo ">>> exim binary built"
@@ -316,6 +328,11 @@
 
 perl.o:          $(HDRS) perl.c
 	$(PERL_CC) $(PERL_CCOPTS) $(CFLAGS) $(INCLUDE) -c perl.c
+
+# Build instructions for python.o for when EXIM_PYTHON is set
+
+python.o:          $(HDRS) python.c
+	$(CC) -c $(CFLAGS) $(INCLUDE) $(PYTHON_INC) -I. python.c
 
 # Dependencies for the "ordinary" exim modules
 
Index: exim/scripts/MakeLinks
diff -u exim/scripts/MakeLinks:1.1.1.1 exim/scripts/MakeLinks:1.1.1.1.2.1
--- exim/scripts/MakeLinks:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/scripts/MakeLinks	Mon Nov 27 08:27:46 2000
@@ -189,6 +189,7 @@
 ln -s ../src/moan.c            moan.c
 ln -s ../src/parse.c           parse.c
 ln -s ../src/perl.c            perl.c
+ln -s ../src/python.c          python.c
 ln -s ../src/queue.c           queue.c
 ln -s ../src/readconf.c        readconf.c
 ln -s ../src/retry.c           retry.c
Index: exim/src/EDITME
diff -u exim/src/EDITME:1.1.1.2 exim/src/EDITME:1.1.1.2.2.4
--- exim/src/EDITME:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/EDITME	Tue Nov 28 16:20:19 2000
@@ -260,6 +260,10 @@
 # PERL_CCOPTS=
 # PERL_LIBS=
 
+# EXIM_PYTHON=python.o
+# PYTHON_INC=-I/usr/local/include/python2.0
+# PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a
+# PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil
 
 # This parameter sets the maximum length of the header portion of a message
 # that Exim is prepared to process. The default setting is one megabyte. There
Index: exim/src/config.h.defaults
diff -u exim/src/config.h.defaults:1.1.1.1 exim/src/config.h.defaults:1.1.1.1.2.1
--- exim/src/config.h.defaults:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/config.h.defaults	Mon Nov 27 08:43:04 2000
@@ -36,6 +36,8 @@
 
 #define EXIM_PERL
 
+#define EXIM_PYTHON
+
 #define HAVE_SA_LEN
 #define HEADER_MAXSIZE  (1024*1024)
 
Index: exim/src/exim.c
diff -u exim/src/exim.c:1.1.1.2 exim/src/exim.c:1.1.1.2.2.4
--- exim/src/exim.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/exim.c	Tue Nov 28 00:46:52 2000
@@ -351,6 +351,9 @@
 #ifdef EXIM_PERL
 int  perl_start_option = 0;
 #endif
+#ifdef EXIM_PYTHON
+int  python_start_option = 0;
+#endif
 int  recipients_arg = argc;
 int  sender_address_domain = 0;
 int  test_retry_arg = -1;
@@ -583,6 +586,17 @@
 simply recorded for checking and handling afterwards. Do a high-level switch
 on the second character (the one after '-'), to save some effort. */
 
+#ifdef EXIM_PYTHON
+opt_python_original_argc = argc;
+opt_python_original_argv = store_get(sizeof(char *) * (argc + 1));
+opt_python_original_argv[argc] = NULL;
+
+for (i = 0; i < argc; i++)
+  {
+  opt_python_original_argv[i] = string_copy(argv[i]);
+  }
+#endif
+
 for (i = 1; i < argc; i++)
   {
   BOOL badarg = FALSE;
@@ -1521,6 +1535,17 @@
     break;
     #endif
 
+    /* -ys: force Python startup; -yd force delayed Python startup */
+
+    #ifdef EXIM_PYTHON
+    case 'y':
+    if (*argrest == 's' && argrest[1] == 0) python_start_option = 1;
+    else
+      if (*argrest == 'd' && argrest[1] == 0) python_start_option = -1;
+    else badarg = TRUE;
+    break;
+    #endif
+
 
     case 'q':
 
@@ -1967,6 +1992,27 @@
   opt_perl_started = TRUE;
   }
 #endif /* EXIM_PERL */
+
+/* Start up Python interpreter if Python support is configured and
+there is a and the configuration or the command line specifies
+initializing starting. Note that the global variables are actually
+called opt_python_xxx. */
+
+#ifdef EXIM_PYTHON
+if (python_start_option != 0)
+  opt_python_at_start = (python_start_option > 0);
+if (opt_python_at_start)
+  {
+  char *errstr;
+  DEBUG(9) debug_printf("Starting Python interpreter\n");
+  errstr = init_python();
+  if (errstr != NULL)
+    {
+    fprintf(stderr, "exim: error during Python startup: %s\n", errstr);
+    return EXIT_FAILURE;
+    }
+  }
+#endif /* EXIM_PYTHON */
 
 /* Log the arguments of the call if the configuration file said so. This is
 a debugging feature for finding out what arguments certain MUAs actually use.
Index: exim/src/expand.c
diff -u exim/src/expand.c:1.1.1.2 exim/src/expand.c:1.1.1.2.2.3
--- exim/src/expand.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/expand.c	Tue Nov 28 00:23:10 2000
@@ -1318,7 +1318,7 @@
 
 Operators:
    domain                  extracts domain from an address
-   escape                  escapes non-printing characters
+   escape                  escapes non-printing characters 
    expand                  expands the string one more time
    hash_<n>_<m>            hash the string, making one that is of length n,
                              using m (default 26) characters from hashcodes
@@ -1865,6 +1865,100 @@
     continue;
     }
   #endif /* EXIM_PERL */
+
+  /* If Perl support is configured, handle calling embedded perl subroutines,
+  unless locked out at this time. Syntax is ${perl{sub}} or ${perl{sub}{arg}}
+  or ${perl{sub}{arg1}{arg2}} or up to a maximum of EXIM_PERL_MAX_ARGS
+  arguments (defined below). */
+
+  #ifdef EXIM_PYTHON
+  #define EXIM_PYTHON_MAX_ARGS 8
+
+  if (strcmp(name, "python") == 0)
+    {
+    int i = 0;
+    char *sub_name;
+    char *sub_arg[EXIM_PYTHON_MAX_ARGS + 1];
+    char *new_yield;
+
+    if (expand_forbid_python)
+      {
+      expand_string_message = "Python calls are not permitted";
+      goto EXPAND_FAILED;
+      }
+
+    while (isspace((uschar)*s)) s++;
+    if (*s != '{') goto EXPAND_FAILED_CURLY;
+    sub_name = expand_string_internal(s+1, TRUE, &s, FALSE);
+    if (sub_name == NULL) goto EXPAND_FAILED;
+    while (isspace((uschar)*s)) s++;
+    if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+
+    while (isspace((uschar)*s)) s++;
+
+    while (*s == '{')
+      {
+      if (i == EXIM_PYTHON_MAX_ARGS)
+        {
+        expand_string_message =
+          string_sprintf("Too many arguments passed to Python subroutine \"%s\" "
+            "(max is %d)", sub_name, EXIM_PYTHON_MAX_ARGS);
+        goto EXPAND_FAILED;
+        }
+      sub_arg[i] = expand_string_internal(s+1, TRUE, &s, FALSE);
+      if (sub_arg[i++] == NULL) goto EXPAND_FAILED;
+      while (isspace((uschar)*s)) s++;
+      if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+      while (isspace((uschar)*s)) s++;
+      }
+
+    if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+    sub_arg[i] = 0;
+
+    /* Start the interpreter if necessary */
+
+    if (!opt_python_started)
+      {
+      char *initerror;
+      DEBUG(9) debug_printf("Starting Python interpreter\n");
+      initerror = init_python();
+      if (initerror != NULL)
+        {
+        expand_string_message =
+          string_sprintf("error during Python startup: %s\n", initerror);
+        goto EXPAND_FAILED;
+        }
+      }
+
+    /* Call the function */
+
+    new_yield = call_python_cat(yield, &size, &ptr, &expand_string_message,
+      sub_name, sub_arg);
+
+    /* NULL yield indicates failure; if the message pointer has been set to
+    NULL, the yield was undef, indicating a forced failure. Otherwise the
+    message will indicate some kind of Perl error. */
+
+    if (new_yield == NULL)
+      {
+      if (expand_string_message == NULL)
+        {
+        expand_string_message =
+          string_sprintf("Python subroutine \"%s\" returned undef to force "
+            "failure", sub_name);
+        expand_string_forcedfail = TRUE;
+        }
+      goto EXPAND_FAILED;
+      }
+
+    /* Yield succeeded. Ensure forcedfail is unset, just in case it got
+    set during a callback from Python. */
+
+    expand_string_forcedfail = FALSE;
+    yield = new_yield;
+    continue;
+    }
+  #endif /* EXIM_PYTHON */
 
   /* Handle character translation for "tr" */
 
Index: exim/src/functions.h
diff -u exim/src/functions.h:1.1.1.2 exim/src/functions.h:1.1.1.2.2.1
--- exim/src/functions.h:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/functions.h	Mon Nov 27 10:13:25 2000
@@ -16,6 +16,12 @@
 extern char *init_perl(char *);
 #endif
 
+#ifdef EXIM_PYTHON
+extern char *call_python_cat(char *, int *, int *, char **, char *, char **);
+extern void  cleanup_python(void);
+extern char *init_python(void);
+#endif
+
 #ifdef SUPPORT_TLS
 extern BOOL  tls_client_start(int, host_item *, address_item *, char *,
                char *, char *, char *, char *, int);
Index: exim/src/globals.c
diff -u exim/src/globals.c:1.1.1.1 exim/src/globals.c:1.1.1.1.2.2
--- exim/src/globals.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.c	Mon Nov 27 11:10:15 2000
@@ -35,6 +35,15 @@
 BOOL   opt_perl_started       = FALSE;
 #endif
 
+#ifdef EXIM_PYTHON
+BOOL   opt_python_at_start       = FALSE;
+char  *opt_python_module_paths   = NULL;
+char  *opt_python_initial_import = NULL;
+BOOL   opt_python_started        = FALSE;
+int    opt_python_original_argc  = 0;
+char **opt_python_original_argv  = NULL;
+#endif
+
 #ifdef HAVE_AUTH
 BOOL   auth_always_advertise  = TRUE;
 char  *auth_hosts             = NULL;
@@ -310,6 +319,7 @@
 BOOL   expand_forbid_exists   = FALSE;
 BOOL   expand_forbid_lookup   = FALSE;
 BOOL   expand_forbid_perl     = FALSE;
+BOOL   expand_forbid_python   = FALSE;
 int    expand_nlength[EXPAND_MAXN+1];
 int    expand_nmax            = -1;
 char  *expand_nstring[EXPAND_MAXN+1];
Index: exim/src/globals.h
diff -u exim/src/globals.h:1.1.1.1 exim/src/globals.h:1.1.1.1.2.2
--- exim/src/globals.h:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.h	Mon Nov 27 11:10:15 2000
@@ -21,6 +21,15 @@
 extern BOOL   opt_perl_started;       /* Set once interpreter started */
 #endif
 
+#ifdef EXIM_PYTHON
+extern BOOL   opt_python_at_start;       /* start Python at startup */
+extern char  *opt_python_module_paths;   /* list of paths to append to sys.path */
+extern char  *opt_python_initial_import; /* list of modules to import at interpreter initialization time */
+extern BOOL   opt_python_started;        /* set once Python interpreter started */
+extern int    opt_python_original_argc;
+extern char **opt_python_original_argv;
+#endif
+
 #ifdef HAVE_AUTH
 extern BOOL   auth_always_advertise;  /* If FALSE, advertise only when needed */
 extern char  *auth_hosts;             /* These must authenticate */
@@ -208,6 +217,7 @@
 extern BOOL   expand_forbid_exists;   /* Lock out exists checking */
 extern BOOL   expand_forbid_lookup;   /* Lock out lookups */
 extern BOOL   expand_forbid_perl;     /* Lock out Perl calls */
+extern BOOL   expand_forbid_python;   /* Lock out Python calls */
 extern int    expand_nlength[];       /* Lengths of numbered strings */
 extern int    expand_nmax;            /* Max numerical value */
 extern char  *expand_nstring[];       /* Numbered strings */
Index: exim/src/python.c
diff -u /dev/null exim/src/python.c:1.1.2.11
--- /dev/null	Tue Nov 28 16:50:15 2000
+++ exim/src/python.c	Tue Nov 28 16:22:03 2000
@@ -0,0 +1,291 @@
+#include "exim.h"
+
+#include "Python.h"
+
+static void init_exim_module(void);
+
+char *init_python(void)
+{
+   if (opt_python_started)
+   {
+      return NULL;
+   }
+
+   Py_SetProgramName(opt_python_original_argv[0]);
+   
+   Py_Initialize();
+   
+   PySys_SetArgv(opt_python_original_argc, opt_python_original_argv);
+
+   init_exim_module();
+
+   if (opt_python_module_paths)
+   {
+      char *listptr = opt_python_module_paths;
+      int separator = 0;
+      char buffer[256];
+      PyObject *path;
+      PyObject *sys_module;
+      PyObject *sys_dict;
+      PyObject *sys_path;
+      PyObject *name;
+
+      name = PyString_FromString("sys");
+      if (name == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem creating string \"sys\"";
+      }
+
+      sys_module = PyImport_Import(name);
+      Py_DECREF(name);
+
+      if (sys_module == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to import sys";
+      }
+
+      sys_dict = PyModule_GetDict(sys_module);
+      if (sys_dict == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying get dictionary from module sys";
+      }
+
+      sys_path = PyDict_GetItemString(sys_dict, "path");
+      if (sys_path == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to get sys.path";
+      }
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)))
+      {
+	 path = PyString_FromString(buffer);
+	 if (path == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while allocating Python string for \"%s\"", buffer);
+	 }
+
+	 if(!PyList_Append(sys_path, path))
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while insering string \"%s\"into sys.path", buffer);
+	 }
+      }
+   }
+
+   if (opt_python_initial_import)
+   {
+      char *listptr = opt_python_initial_import;
+      int separator = 0;
+      char buffer[256];
+      PyObject *module;
+      PyObject *name;
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)))
+      {
+	 name = PyString_FromString(buffer);
+	 if (name == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem creating string \"%s\"", buffer);
+	 }
+
+	 module = PyImport_Import(name);
+	 Py_DECREF(name);
+
+	 if (module == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem importing module %s", buffer);
+	 }
+      }
+   }
+
+      
+   opt_python_started = TRUE;
+   return NULL;
+}
+
+void cleanup_python(void)
+{
+  Py_Finalize();
+}
+
+static PyObject *find_function(char *name, char **errstrp)
+{
+   PyObject *module_name;
+   PyObject *function_name;
+   PyObject *module;
+   PyObject *dictionary;
+   PyObject *result;
+
+   char *ptr;
+
+   ptr = strrchr(name, '.');
+   
+   if (ptr == NULL)
+   {
+      *errstrp = string_sprintf("function name must include module: <module>.<name>");
+      return NULL;
+   }
+
+   function_name = PyString_FromString(ptr + 1);
+
+   if (function_name == NULL)
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("error trying to allocate function name");
+      return NULL;
+   }
+
+   module_name = PyString_FromStringAndSize(name, ptr - name);
+
+   if (function_name == NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp = string_sprintf("error trying to allocate module name");
+      return NULL;
+   }
+
+   module = PyImport_Import(module_name);
+
+   Py_DECREF(module_name);
+
+   if (module == NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp = string_sprintf("error trying to import module while trying to find %s", name);
+      return NULL;
+   }
+
+   dictionary = PyModule_GetDict(module);
+
+   result = PyDict_GetItem(dictionary, function_name);
+
+   Py_DECREF(function_name);
+
+   if (!PyCallable_Check(result))
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("%s is not a callable object", name);
+      return NULL;
+   }
+
+   return result;
+}
+
+char *call_python_cat(char *yield, int *sizep, int *ptrp, char **errstrp,
+		      char *name, char **arg)
+{
+   PyObject *function;
+   PyObject *arguments;
+   PyObject *result;
+   int index;
+   int count=0;
+   char **p;
+   char *buffer;
+   int length;
+
+   function = find_function(name, errstrp);
+
+   if (function == NULL)
+      return NULL;
+
+   p = arg;
+
+   while(*p)
+   {
+      count++;
+      p++;
+   }
+
+   arguments = PyTuple_New(count);
+
+   index = 0;
+   while(index < count)
+   {
+
+      PyTuple_SetItem(arguments, index, PyString_FromString(arg[index]));
+      index++;
+   }
+
+   result = PyObject_CallObject(function, arguments);
+
+   Py_DECREF(arguments);
+
+   if (result == NULL)
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("error in Python function %s", name);
+      return NULL;
+   }
+
+   if(!PyString_Check(result))
+   {
+      Py_DECREF(result);
+      *errstrp = string_sprintf("Python function %s returned non-string", name);
+      return NULL;
+   }
+
+   if(PyString_AsStringAndSize(result, &buffer, &length))
+   {
+      PyErr_Clear();
+      Py_DECREF(result);
+      *errstrp = string_sprintf("could not extract the results of Python function %s", name);
+      return NULL;
+   }
+
+   yield = string_cat(yield, sizep, ptrp, buffer, length);
+
+   Py_DECREF(result);
+
+   return yield;
+}
+
+static PyObject *exim_error = NULL;
+
+static PyObject *exim_expand_string(PyObject *self, PyObject *args)
+{
+   char *i;
+   char *o;
+
+   if(!PyArg_ParseTuple(args, "s", i))
+      return NULL;
+   
+   o = expand_string(i);
+   
+   if (o == NULL)
+   {
+      PyErr_SetString(exim_error, expand_string_message);
+      return NULL;
+   }
+  
+   return PyString_FromString(o);
+}
+
+static PyMethodDef exim_methods[] = {
+   {"expand_string", exim_expand_string, 1},
+   {NULL,            NULL} /* sentinel */
+};
+
+static void init_exim_module(void)
+{
+   PyObject *m;
+   PyObject *d;
+   PyObject *o;
+
+   m = PyImport_AddModule("exim");
+   d = PyModule_GetDict(m);
+
+   Py_InitModule("exim", exim_methods);
+
+   exim_error = PyErr_NewException("exim.error", NULL, NULL);
+   PyDict_SetItemString(d, "error", exim_error);
+}
+
Index: exim/src/readconf.c
diff -u exim/src/readconf.c:1.1.1.1 exim/src/readconf.c:1.1.1.1.2.2
--- exim/src/readconf.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/readconf.c	Mon Nov 27 10:14:17 2000
@@ -160,6 +160,11 @@
   { "perl_at_start",            opt_bool,        &opt_perl_at_start },
   { "perl_startup",             opt_stringptr,   &opt_perl_startup },
 #endif
+#ifdef EXIM_PYTHON
+  { "python_at_start",          opt_bool,        &opt_python_at_start },
+  { "python_module_paths",      opt_stringptr,   &opt_python_module_paths },
+  { "python_initial_import",    opt_stringptr,   &opt_python_initial_import },
+#endif
 #ifdef LOOKUP_PGSQL
   { "pgsql_servers",            opt_stringptr,   &pgsql_servers },
 #endif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001128/ab04e5bc/attachment.pgp>

From gward at mems-exchange.org  Wed Nov 29 00:51:47 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Tue, 28 Nov 2000 18:51:47 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 01:11:54PM -0500
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com> <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <20001128185146.A19116@ludwig.cnri.reston.va.us>

On 27 November 2000, Guido van Rossum said:
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

I think that the "Distributing Python Modules" manual should have a
"Recommended Practices" section to cover things like this.  My personal
convention is that every single source file has a __revision__ variable,
and the "root" module [1] of a module distribution has __version__.

[1] the "root module" is the highest-level __init__.py in a package-ized
    distribution, or the module itself in a single-file distribution.
    Most non-packageized multi-module distributions have a fairly
    obvious place to put __version__.

        Greg



From akuchlin at mems-exchange.org  Wed Nov 29 03:57:48 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 21:57:48 -0500
Subject: [Python-Dev] PEP 229 and 222
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 28, 2000 at 06:01:38PM -0500
References: <200011282213.OAA31146@slayer.i.sourceforge.net> <20001128171735.A21996@kronos.cnri.reston.va.us> <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <20001128215748.A22105@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 06:01:38PM -0500, Guido van Rossum wrote:
>- Always shared libs.  What about Unixish systems that don't have
>  shared libs?  What if you just want something to be hardcoded as
>  statically linked, e.g. for security reasons?  (On the other hand

Beats me.  I'm not even sure if the Distutils offers a way to compile
a static Python binary.  (GPW: well, does it?)

>idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
>than making the existing cgi.py even more complicated.  (Also, it's
>very fragile -- you never know who uses what mis-feature.)

Indeed.  My inclination is to design two new Request and Response
classes; haven't done that yet, though...

--amk




From fdrake at users.sourceforge.net  Wed Nov 29 07:20:31 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Tue, 28 Nov 2000 22:20:31 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011290620.WAA09888@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From tim.one at home.com  Wed Nov 29 07:23:10 2000
From: tim.one at home.com (Tim Peters)
Date: Wed, 29 Nov 2000 01:23:10 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMDIBAA.tim.one@home.com>

[Guido]
> ...
> Because of its importance, the deprecation time of the string module
> will be longer than that of most deprecated modules.  I expect it
> won't be removed until Python 3000.

I see nothing in the 2.0 docs, code, or "what's new" web pages saying that
it's deprecated.  So I don't think you can even start the clock on this one
before 2.1 (a fuzzy stmt on the web page for the unused 1.6 release doesn't
count ...).

> ...
> This has been discussed -- jut note that continuing to use the string
> module *is* frowned upon, and such stubborn code will get its just
> desserts when Py3K arrives.
>
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
>
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new
>     code.
>
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

They have to be in a module.  They've always been in the string module.
Common sense thus dictates "leave them in the string module" <0.3 wink>.

change-for-the-sake-of-irritation-is-arguably-irritating-ly y'rs  - tim




From bckfnn at worldonline.dk  Wed Nov 29 22:37:38 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Wed, 29 Nov 2000 21:37:38 GMT
Subject: [Python-Dev] long formatting
Message-ID: <3a2576da.50607279@smtp.worldonline.dk>

Hi,

While adding support to Jython for formatting of longs in "%[duxXo]", I
inadvertently introduced another difference between Python and Jython:

Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
000000000000000000037777777766
>>>

Jython 2.0alpha1 on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
-00000000000000000000000000012
>>>

The CPython behaviour looks as if the number have been through a C long
before it is formatted. It that intended? Would jython be wrong if it
used the result above?

regards,
finn



From fdrake at acm.org  Wed Nov 29 22:49:53 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 29 Nov 2000 16:49:53 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
References: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012
...
 > The CPython behaviour looks as if the number have been through a C long
 > before it is formatted. It that intended? Would jython be wrong if it
 > used the result above?

  It doesn't look like either of them is doing the right thing!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From tim_one at email.msn.com  Wed Nov 29 23:10:34 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:10:34 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com>

[Moshe Zadka]
> ...
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.
> Almost?
>
> * No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
> * Convertors: if we want list/tuple to have a semblance of efficiency,
>   we'll need to cache the element list as a list when accessed by
>   index.

Like everyone else, I have a Set class too.  Its __getitem__ is

    # so "for x in set:" works
    # see also set.tolist()
    def __getitem__(self, i):
        if i == 0:
            self.keys = self.d.keys()
        return self.keys[i]

Of course I don't *want* set[i] to mean anything at all; this is just a hack
to work with the current iteration protocol (i.e., Set.__getitem__ makes no
sense except in that it must be implemented so that "for x in set" works
today).  The one thing I could never get used to in Aaron's kjbuckets
implementation of sets is that for/in did not work as expected.

> * Two different constructors: set() for building from sequence, Set()
>   for building from elements. Might be confusing.

My Set.__init__:

    def __init__(self, seq=[]):
        self.d = d = {}
        for x in seq:
            d[x] = 1

That is, the constructor takes a sequence.  Same thing in Icon, which added
a set type late in the game.  No problem in practice.

> * Only possible elements of a set are immutables. OTOH, I'm not sure
>   how Greg intends to implement his sets if these sets are allowed
>   to contain mutable elements.

This needs to be addressed, but several approaches are possible.  For
example, my Set class allows for Sets of Sets (& so on), because I needed
them, and Sets are certainly mutable.

Now immutable objects are required because dicts require immutable objects
as keys.  However, a class (like Set!) can fool dicts, by supplying a
__hash__ and a __cmp__ "that work".  Dicts can be compared directly, so
__cmp__ is no problem.  The difficulty comes with the __hash__.  My solution
was to "freeze" a Set the instant __hash__ was invoked:  this allows
mutation up until the point a hash is captured, and disallows it thereafter.
Concretely,

    def __hash__(self):
        if self.frozen:
            hashcode = self.hashcode
        else:
            # The hash code must not depend on the order of the
            # keys.
            self.frozen = 1
            hashcode = 0
            _hash = hash
            for x in self.d.keys():
                hashcode = hashcode ^ _hash(x)
            self.hashcode = hashcode
        return hashcode

and all the mutating methods check self.frozen, raising ValueError if the
target set is currently frozen.  For example,

    # updating union
    def unionu(self, other):
        if self.frozen:
            raise ValueError(_frozenmsg)
        self.d.update(other.d)

Oddly enough, I almost never got a "frozen" error in practice; it seems that
by the time I was ready to build a set of sets, the constituent sets were at
their final values; and the few times I did get a frozen error, it was
actually a logic bug (I was mutating the set in error).  It's hard to guess
whether that's unique to me <0.5 wink>.

Icon has a different approach entirely:  an Icon set acts like a Python dict
*would* act if you inserted the id()s of the keys rather than the keys
themselves; that is, Icon's sets (and dicts) are based on object identity,
not object value.  Since object identity is invariant regardless of whether
the object is mutable, a hash table implementation has no difficulties.

I find identity semantics for sets less useful than value semantics, though.

[Eric Raymond]
> ...
> Within a few hours after I see [rich comparisons] in the beta I'll
> have a very rich Set class for the standard library.  It includes
> all the standard boolean operations, symmetric difference, powerset,
> and useful overloads for most of the standard operators.  As the
> header comment says:
>
> # A set-algebra module for Python
> #
> # The functions work on any sequence type and return lists.
> # The set methods can take a set or any sequence type as an argument.
> ...

I'm afraid this is a good example of why a set type is unlikely to make it
into the std distribution:  whenever a data structure is added to Python,
the agitation for variations is endless.  For example, returning a list
makes your flavor of sets unsuitable (because inefficient) for many
applications (e.g., there's no efficient way to test a list for element
membership).  "Almost all" set implementations I've seen for Python use
dicts for that reason.

Another set of complaints will be due to some people wanting functional
versions of the set operations, while others want mutating versions.  My own
Set class supplies both, because both are really needed in practice; e.g.,
the updating union was shown above; the functional union builds on that:

    # functional union
    def union(self, other):
        answer = self.__copy__()
        answer.unionu(other)
        return answer

(subtlety:  the functional union has no problem with "frozen" sets;
Set.__copy__ always returns a melted set).

Since it's Greg's PEP, it's up to him to make everyone happy <wink> -- but
since there's so much variation possible, I'm not sure Guido will ever bless
any Set class as "the" Set class.

BTW, one lesson to take from SETL:  a vital set operation in practice is a
mutating "pick a 'random' element E from the set, remove it from the set,
and return it".  An enormous number of set algorithms are of the form

    while not S.empty():
        pick some element from S
        deal with it, possibly mutating S

This operation can't be done efficiently in Python code if the set is
represented by a dict (the best you can do is materialize the full list of
keys first, and pick one of those).  That means my Set class often takes
quadratic time for what *should* be linear-time algorithms.

if-the-set-type-is-a-toy-there's-no-need-to-put-it-in-the-
    distribution-but-if-it's-not-a-toy-it-can't-be-written-
    in-python-today-ly y'rs  - tim





From tim_one at email.msn.com  Wed Nov 29 23:32:29 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:32:29 -0500
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENHIBAA.tim_one@email.msn.com>

[Finn Bock]
> While adding support to Jython for formatting of longs in "%[duxXo]", I
> inadvertently introduced another difference between Python and Jython:
>
> Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> 000000000000000000037777777766
> >>>
>
> Jython 2.0alpha1 on java1.3.0 (JIT: null)
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> -00000000000000000000000000012
> >>>
>
> The CPython behaviour looks as if the number have been through a C long
> before it is formatted. It that intended?

Hard to say, but it is what the code does <wink>.  In 1.5.2, "true longs"
raised an error in this context, so the code first looked to see whether the
putative Python long actually fit in a C long first; if so, it did not raise
an error, and ran the value thru C long formatting instead (in
stringobject.c):

				if (PyLong_Check(v) && PyLong_AsLong(v) == -1
				    && PyErr_Occurred()) {
					/* Too big for a C long. */

That test survived in 2.0; the difference is that it no longer raises an
error if the value doesn't fit in a C long.

> Would jython be wrong if it used the result above?

Hard to say!  Since the size of a Python int varies across platforms, not
even CPython will display the same thing for all Python longs across all
platforms.  I'm in favor of changing the test above to

				if (PyLong_Check(v)) {

Then CPython will match JPython here, and CPython will produce the same
result across all platforms when given a Python long.  The downside is that
it will not produce the same result as 2.0 (or earlier).

Guido, you can Pronounce now <wink>.





From guido at python.org  Thu Nov 30 04:14:47 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:14:47 -0500
Subject: [Python-Dev] Mail to guido@python.org lost or delayed
Message-ID: <200011300314.WAA18820@cj20424-a.reston1.va.home.com>

Folks, mail to guido at python.org may have been bouncing or dropped in a
black hole, since approximately 12:40 pm EST today -- due to a mail
server problem at my ISP (@home.com -- fast reliable cable bandwidth
but lousy mail and DNS service).  I found out about this and *thought*
I had fixed it by switching to guido at digicool.com, but in fact there
was still a forward to @home.com in effect.  I just finally fixed that
(around 10:10 pm EST).

You don't need to change the email address you have for me:
guido at python.org is still the preferred address.  However if you sent
me mail between the approximate times above, I probably haven't seen
it yet.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 04:17:44 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:17:44 -0500
Subject: [Python-Dev] Re: long formatting
Message-ID: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>

I just skimmed the pipermail python-dev archives and found one item
to pronounce on: what "%o" should do with negative longs.  I agree
with Tim.  Also for "%x".  Reason: this matches oct() and hex().

>>> oct(-10L)
'-012L'
>>> oct(-10)
'037777777766'
>>> hex(-10L)
'-0xAL'
>>> hex(-10)
'0xfffffff6'
>>> 

Compatibility be damned (in this case).

(One could argue that for plain ints, the same result are desirable;
however when these are used as bitmasks, I as human reader prefer to
see '0xfffffff6' rather than '-0xA'.  Ah the insanity of consistency!)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov 30 06:41:03 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 30 Nov 2000 00:41:03 -0500
Subject: [Python-Dev] Re: long formatting
In-Reply-To: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>

Alas, Guido and I have the same ISP (@home), and its mail server has been
dead all day.  Conveniently <snort>, my other ISP (MSN) coincidentally
stopped letting me send email over my cable connection too.

[Guido]
> I just skimmed the pipermail python-dev archives and found one item
> to pronounce on: what "%o" should do with negative longs.  I agree
> with Tim.  Also for "%x".  Reason: this matches oct() and hex().

Good point!

> >>> oct(-10L)
> '-012L'
> >>> oct(-10)
> '037777777766'

Note that this string varies across platforms.

> >>> hex(-10L)
> '-0xAL'
> >>> hex(-10)
> '0xfffffff6'

Ditto.

> >>>
>
> Compatibility be damned (in this case).

This became SF bug 123859, and has been closed.  Current CVS does:

>>> "%#o" % -10L
'-012'
>>> "%#X" % -10L
'-0XA'
>>>

> (One could argue that for plain ints, the same result are
> desirable; however when these are used as bitmasks, I as human
> reader prefer to see '0xfffffff6' rather than '-0xA'.  Ah the
> insanity of consistency!)

I doubt that we're done with this forever:  we're moving in the direction of
erasing user-visible distinctions between ints and longs.  For bitmask
output, we should probably introduce a new format code.

Fred, I see that you complained about Finn's output examples, but didn't say
why (just "they both look incorrect").  Were you hallucinating, or do you
have a legit complaint?





From fdrake at acm.org  Thu Nov 30 07:22:02 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 01:22:02 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
References: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
	<LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>
	<3a2576da.50607279@smtp.worldonline.dk>
	<14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
Message-ID: <14885.61962.901996.857623@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012

Fred L. Drake, Jr. writes:
 >   It doesn't look like either of them is doing the right thing!

Tim Peters writes:
 > Fred, I see that you complained about Finn's output examples, but didn't say
 > why (just "they both look incorrect").  Were you hallucinating, or do you
 > have a legit complaint?

[Fred scrutinizes everything very carefully again... blushes when he
 notices that this was the 'o' format code, not 'd', and quietly
 slinks off to walk the dog...]


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at zadka.site.co.il  Thu Nov 30 15:52:59 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 16:52:59 +0200
Subject: [Python-Dev] PEP-0218 
In-Reply-To: Message from "Tim Peters" <tim_one@email.msn.com> 
   of "Wed, 29 Nov 2000 17:10:34 EST." <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> 
Message-ID: <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>

> BTW, one lesson to take from SETL:  a vital set operation in practice is a
> mutating "pick a 'random' element E from the set, remove it from the set,
> and return it".  An enormous number of set algorithms are of the form
> 
>     while not S.empty():
>         pick some element from S
>         deal with it, possibly mutating S
> 
> This operation can't be done efficiently in Python code if the set is
> represented by a dict (the best you can do is materialize the full list of
> keys first, and pick one of those).  That means my Set class often takes
> quadratic time for what *should* be linear-time algorithms.

Hmmm...actually, I've been wanting a method .key() for dictionaries 
a long time. So if we give dictionaries this one small method, then
we *can* do this in Python.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From fdrake at users.sourceforge.net  Thu Nov 30 08:33:55 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Wed, 29 Nov 2000 23:33:55 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011300733.XAA12728@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/

Today's changes are primarily in the xml.dom API documentation:

	http://python.sourceforge.net/devel-docs/lib/markup.html



From mal at lemburg.com  Thu Nov 30 09:30:22 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 09:30:22 +0100
Subject: [Python-Dev] PEP-0218
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>
Message-ID: <3A26101E.7FF2467B@lemburg.com>

Moshe Zadka wrote:
> 
> > BTW, one lesson to take from SETL:  a vital set operation in practice is a
> > mutating "pick a 'random' element E from the set, remove it from the set,
> > and return it".  An enormous number of set algorithms are of the form
> >
> >     while not S.empty():
> >         pick some element from S
> >         deal with it, possibly mutating S
> >
> > This operation can't be done efficiently in Python code if the set is
> > represented by a dict (the best you can do is materialize the full list of
> > keys first, and pick one of those).  That means my Set class often takes
> > quadratic time for what *should* be linear-time algorithms.
> 
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

Shouldn't be hard to do... the C API for this is already in place:
PyDict_Next(). I'd prefer a method .getitem(), though, which then
returns a tuple (key, value).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Thu Nov 30 09:44:38 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 10:44:38 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <3A26101E.7FF2467B@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011301043050.24575-100000@sundial>

[Me]
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

[MAL]
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

Well, I'm sure the C API allows it. And if we do it, we might as well
do it right: add .key(), .value() and .item().

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From pf at artcom-gmbh.de  Thu Nov 30 10:25:37 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 10:25:37 +0100 (MET)
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: <3A26101E.7FF2467B@lemburg.com> from "M.-A. Lemburg" at "Nov 30, 2000  9:30:22 am"
Message-ID: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>

M.-A. Lemburg:
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

IMO '.pickitem()' would be a better name, since many people would
think, that 'getitem()' would take some kind of index as parameter.
Nevertheless I think this is a nice idea, though.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Thu Nov 30 11:55:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 11:55:33 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <3A263225.E2FFDADB@lemburg.com>

Peter Funk wrote:
> 
> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Point taken. How about .fetchitem() -- this doesn't imply any
ordering and can't get confused with __getitem__() et al.

BTW, Moshe's .key() and .value() would break the strong binding
between dictionary keys and values -- I don't see much use in them.
Besides, .fetchitem() provides the same implied functionality.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Thu Nov 30 12:10:33 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:10:33 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:25:37 +0100."
             <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>

> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Pronouncement:

It is only efficient to get the *first* item, so let's make that
explicit.  The method names will be:

 .firstkey()
 .firstvalue()
 .firstitem()

Moshe will check in a patch.

Thinking aloud:

Would it be useful to also implement popkey(), popvalue(), popitem(),
which would remove the first item and then return the relevant part of
it?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Thu Nov 30 12:30:09 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 12:30:09 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <3A263A41.AFEFFB53@lemburg.com>

Guido van Rossum wrote:
> 
> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> >
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()

-1

>  .firstvalue()

-1

>  .firstitem()

+1

I really think that .firstitem() is both more intuitive and 
efficient -- .firstvalue() and .firstkey() are simply derivatives
of this method or how would you define the return value of
those two methods ?
 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

+1

Same comment as above... .popitem() suffices.
 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From pf at artcom-gmbh.de  Thu Nov 30 12:33:30 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 12:33:30 +0100 (MET)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 30, 2000  6:10:33 am"
Message-ID: <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>

Guido van Rossum:
[...]
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

Since [].pop() takes in optional index as parameter which defaults
to the *LAST* element, it would be better to make the name explicit:
.popfirstitem()

This is however some what long.  DEJAVU: the {}.setdefault() discussion.
May it is better to leave these methods out for the sake of simplicity, 
although they look useful.

Regards, Peter



From guido at python.org  Thu Nov 30 12:52:10 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:52:10 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:30:09 +0100."
             <3A263A41.AFEFFB53@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>  
            <3A263A41.AFEFFB53@lemburg.com> 
Message-ID: <200011301152.GAA26000@cj20424-a.reston1.va.home.com>

> > Pronouncement:
> > 
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> > 
> >  .firstkey()
> 
> -1
> 
> >  .firstvalue()
> 
> -1
> 
> >  .firstitem()
> 
> +1
> 
> I really think that .firstitem() is both more intuitive and 
> efficient -- .firstvalue() and .firstkey() are simply derivatives
> of this method or how would you define the return value of
> those two methods ?

Yes, firstvalue() and firstkey() return just the value or just the
key.  I can see that the first value is not very useful (since you
don't have the key, you can't mutate the dict).  firstkey() might
be just the ticket e.g. for set implementations.  OTOH you may be
right that it's best to add the minimal number of new methods.

> > Moshe will check in a patch.
> > 
> > Thinking aloud:
> > 
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> +1
> 
> Same comment as above... .popitem() suffices.

And for minimality reasons I'm at most -0 on it.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 13:09:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:09:03 -0500
Subject: [Python-Dev] Python track--O'Reilly Open Source Convention
Message-ID: <200011301209.HAA26079@cj20424-a.reston1.va.home.com>

Laura Lewin is looking for chairs for two Python tracks (or a Python
track and a Zope track) at the O'Reilly conference.  Anybody
interested in volunteering?  Some background info:

> We actually have two tracks, and there's room for strong differentiation 
> between the two.  Each track has a room for two days. We'll also want 
> tutorials on the Monday and Tuesday.
> 
> Like O'Reilly's books, we're not really aimed at the novice. We want to 
> give people who already use the technology more information. There should 
> be at least one introductory tutorial, but we don't want to focus 
> exclusively on that. However, if you, as Chair, have a vision for what you 
> want O'Reilly's conference to be, we'd be very happy to work with you to 
> realize that vision.
> 
> This year we're trying to raise the bar on our Python and Zope offerings. 
> We'd like to have a lot of editorial involvement from a program committee, 
> so that potential speakers have a chance to get their talks reviewed and 
> commented-upon by others.
> 
> The program chair would really be the point person for the conference, to 
> drum up papers and help publicize the conference to the Python community. 
> The chair would appoint a committee, receive submissions, select 
> presentations, seek out talks from people who
> didn't submit, and generally put together the material.
> 
> We can handle any combination of program committees and chairs: one person 
> overseeing two committees, two chairs two committees, whatever gets it all 
> done.
> 
> Does this sound like a job you're interested in? It doesn't pay, but it 
> would be great for the Python community, as I'm sure you know.

For more info, please write or call Laura Lewin LLewin at oreilly.com
(631) 262-9278.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 13:40:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:40:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 04:31:07 PST."
             <200011301231.EAA22228@slayer.i.sourceforge.net> 
References: <200011301231.EAA22228@slayer.i.sourceforge.net> 
Message-ID: <200011301240.HAA26261@cj20424-a.reston1.va.home.com>

Aargh!  I thought I told you to submit a patch (to SF), not to check
it in.

The code looks okay but we'll have to discuss whether we really want
all three (as opposed to only firstitem()).  I'm also not sure whether
KeyError might be a better exception.

Don't touch it for now, but let's think this over a bit.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mwh21 at cam.ac.uk  Thu Nov 30 14:16:44 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 13:16:44 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Guido van Rossum's message of "Thu, 30 Nov 2000 06:10:33 -0500"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> > 
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()
>  .firstvalue()
>  .firstitem()
> 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

I must admit I can't think of a situation where .firstkey would be
more useful than .popkey; I thought that this discussion was
discussion was inspired by the wish to write code like:

while not set.empty():
    x = set.popfirstkey()
    ...

Maybe I'm jsut being dense, though.

FWIW, I think 

    dict.firstkey()

reads better than

    dict.firstitem()[0]

if the former's what you're actually trying to say.  Why not write

    for k,_ in dict.items():
        ...

? (Overstating the case slightly for effect, but I hope the point is
clear).

Cheers,
M.

-- 
  Arrrrgh, the braindamage!  It's not unlike the massively
  non-brilliant decision to use the period in abbreviations 
  as well as a sentence terminator.  Had these people no 
  imagination at _all_?                 -- Erik Naggum, comp.lang.lisp




From fdrake at acm.org  Thu Nov 30 14:34:07 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:34:07 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43
In-Reply-To: <200011301231.EAA22229@slayer.i.sourceforge.net>
References: <200011301231.EAA22229@slayer.i.sourceforge.net>
Message-ID: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > +   \lineiii{\var{a}.firstitem()}
 > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
 > +           {(2)}

  So what happens if the mapping is empty?  That's needed for all
three.
  You should also update UserDict.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Thu Nov 30 14:40:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 14:40:12 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A2658BC.7922B8B1@lemburg.com>

Michael Hudson wrote:
> 
> Guido van Rossum <guido at python.org> writes:
> 
> > > M.-A. Lemburg:
> > > > Shouldn't be hard to do... the C API for this is already in place:
> > > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > > returns a tuple (key, value).
> > >
> > > IMO '.pickitem()' would be a better name, since many people would
> > > think, that 'getitem()' would take some kind of index as parameter.
> > > Nevertheless I think this is a nice idea, though.
> >
> > Pronouncement:
> >
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> >
> >  .firstkey()
> >  .firstvalue()
> >  .firstitem()
> >
> > Moshe will check in a patch.
> >
> > Thinking aloud:
> >
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> I must admit I can't think of a situation where .firstkey would be
> more useful than .popkey; I thought that this discussion was
> discussion was inspired by the wish to write code like:
> 
> while not set.empty():
>     x = set.popfirstkey()
>     ...
> 
> Maybe I'm jsut being dense, though.

Just curious: how would you get at the deleted value when
using .popfirstkey() ?

But you have a point: dict.pop() returning the first filled slot
in the dictionary as tuple (key,value) would probably be most
effective. It's also nice and short.

BTW, I don't get the inspiration for the "first" part in those
names... it makes you think that there's an order to dictionary
items when in reality there isn't.

> FWIW, I think
> 
>     dict.firstkey()
> 
> reads better than
> 
>     dict.firstitem()[0]
> 
> if the former's what you're actually trying to say.  Why not write
> 
>     for k,_ in dict.items():
>         ...
> 
> ? (Overstating the case slightly for effect, but I hope the point is
> clear).

No, the point is that you can modify the dictionary within
the loop. You wouldn't see any changes using the second technique.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Thu Nov 30 14:43:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 15:43:10 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>

On Thu, 30 Nov 2000, Fred L. Drake, Jr. wrote:

> 
> Moshe Zadka writes:
>  > +   \lineiii{\var{a}.firstitem()}
>  > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
>  > +           {(2)}
> 
>   So what happens if the mapping is empty?  That's needed for all
> three.
>   You should also update UserDict.
>

Right. Will be done soon.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Thu Nov 30 14:43:48 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:43:48 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011301542250.4645-100000@sundial>
Message-ID: <14886.22932.50399.879012@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Right. Will be done soon.

  Given Guido's surprise at the checkin, you might want to wait until
the discussions are over and just make one set of checkins, with all
the results.  ;)
  Hmm.  I need to think about adding version annotations to those
tables of operations on built-in types.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov 30 14:52:07 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 08:52:07 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 14:40:12 +0100."
             <3A2658BC.7922B8B1@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>  
            <3A2658BC.7922B8B1@lemburg.com> 
Message-ID: <200011301352.IAA26550@cj20424-a.reston1.va.home.com>

> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?
> 
> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Nice!

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

Actually there is -- the (arbitrary) order revealed by .keys().

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at zadka.site.co.il  Thu Nov 30 23:01:14 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Fri, 01 Dec 2000 00:01:14 +0200
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43 
In-Reply-To: Your message of "Thu, 30 Nov 2000 08:43:48 EST."
             <14886.22932.50399.879012@cj42289-a.reston1.va.home.com> 
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com> <Pine.GSO.4.10.10011301542250.4645-100000@sundial>  <14886.22932.50399.879012@cj42289-a.reston1.va.home.com> 
Message-ID: <20001130220114.56E85A9BB@darjeeling.zadka.site.co.il>

>   Given Guido's surprise at the checkin, you might want to wait until
> the discussions are over and just make one set of checkins, with all
> the results.  ;)

Hence my not doing it right now.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Thu Nov 30 15:00:36 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 15:00:36 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>  
	            <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>
Message-ID: <3A265D84.9A80267A@lemburg.com>

Guido van Rossum wrote:
> 
> > Just curious: how would you get at the deleted value when
> > using .popfirstkey() ?
> >
> > But you have a point: dict.pop() returning the first filled slot
> > in the dictionary as tuple (key,value) would probably be most
> > effective. It's also nice and short.
> 
> Nice!
>
> > BTW, I don't get the inspiration for the "first" part in those
> > names... it makes you think that there's an order to dictionary
> > items when in reality there isn't.
> 
> Actually there is -- the (arbitrary) order revealed by .keys().

Hmm, but this can change if you modify the dictionary inside the
loop or if you modify the dictionary from some other thread.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Thu Nov 30 15:02:17 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 30 Nov 2000 15:02:17 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <02d301c05ad6$26851150$0900a8c0@SPIFF>

thanks, mal !

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

+1 on "pop" (short for dist.items().pop())

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

-0 on "first" (on the other hand, are we really
allowed to vote on pronouncements? ;-)

</F>




From mwh21 at cam.ac.uk  Thu Nov 30 15:06:58 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 14:06:58 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 14:40:12 +0100"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <m3pujd5zb1.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:
> Michael Hudson wrote:
> > I must admit I can't think of a situation where .firstkey would be
> > more useful than .popkey; I thought that this discussion was
> > discussion was inspired by the wish to write code like:
> > 
> > while not set.empty():
> >     x = set.popfirstkey()
> >     ...
> > 
> > Maybe I'm jsut being dense, though.
> 
> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?

You wouldn't.  I was thinking of the dict-as-sets code that was being
flung around earlier.

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Yes.

> > FWIW, I think
> > 
> >     dict.firstkey()
> > 
> > reads better than
> > 
> >     dict.firstitem()[0]
> > 
> > if the former's what you're actually trying to say.  Why not write
> > 
> >     for k,_ in dict.items():
> >         ...
> > 
> > ? (Overstating the case slightly for effect, but I hope the point is
> > clear).
> 
> No, the point is that you can modify the dictionary within
> the loop. You wouldn't see any changes using the second technique.

I was trying to make an analogy between .firstkey/.firstitem and
.keys/.items - the 

    for k,_ in dict.items():
        ...

was meant to be compared to 

    for k in dict.keys():
        ...

Sorry for being obscure.

Cheers,
M.

-- 
  "declare"?  my bogometer indicates that you're really programming
  in some other language and trying  to force Common Lisp into your
  mindset.  this won't work.            -- Erik Naggum, comp.lang.lisp




From guido at python.org  Thu Nov 30 15:10:34 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 09:10:34 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 15:00:36 +0100."
             <3A265D84.9A80267A@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>  
            <3A265D84.9A80267A@lemburg.com> 
Message-ID: <200011301410.JAA26719@cj20424-a.reston1.va.home.com>

> > > BTW, I don't get the inspiration for the "first" part in those
> > > names... it makes you think that there's an order to dictionary
> > > items when in reality there isn't.
> > 
> > Actually there is -- the (arbitrary) order revealed by .keys().
> 
> Hmm, but this can change if you modify the dictionary inside the
> loop or if you modify the dictionary from some other thread.

Let me explain.  When Moshe first proposed a .key() method, I was a
bit confused.  I thought he wanted a way to get to the i-th key (which
can't be done in O(1) time).  Then I thought he wanted a way to get a
random key.  Finally I realized that all he wanted was *a* key, and
the first key in the hash table would work fine.  Since that's also
the first key of .keys(), I figured "first" was an appropriate
qualification.  Sure, it's not guaranteed to remain the first key, but
neither is the first item in a list.

That said, getting rid of the first*() suite and adding popitem()
instead sounds attractive...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Thu Nov 30 16:14:53 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:14:53 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
	<m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14886.28397.991041.190430@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf at artcom-gmbh.de> writes:

    PF> Since [].pop() takes in optional index as parameter which
    PF> defaults to the *LAST* element, it would be better to make the
    PF> name explicit: .popfirstitem()

{}.pop() sounds like the right thing given its symmetry with [].pop().

It would have the semantics that it would remove and return a random
/item/ from the dictionary, which the implementation can make the
"first" item for convenience purposes.

It shouldn't dictate though, that {}.pop() == dict.items().pop() or
make any other guarantees about which item gets popped, because other
implementations (or indeed, other mapping-conformant objects) may use
different underlying representations that could make this inefficient.

For added symmetry, {}.pop() should take an optional argument which is
the key of the item to remove and return.

-Barry



From barry at digicool.com  Thu Nov 30 16:23:21 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:23:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
	<200011301240.HAA26261@cj20424-a.reston1.va.home.com>
Message-ID: <14886.28905.691007.74511@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> Aargh!  I thought I told you to submit a patch (to SF), not
    GvR> to check it in.

    GvR> The code looks okay but we'll have to discuss whether we
    GvR> really want all three (as opposed to only firstitem()).  I'm
    GvR> also not sure whether KeyError might be a better exception.

    GvR> Don't touch it for now, but let's think this over a bit.

And don't forget that when we /do/ decide on the right thing, that
UserDict needs to map <ha ha> the change too.

-Barry



From guido at python.org  Thu Nov 30 16:22:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 10:22:06 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:14:53 EST."
             <14886.28397.991041.190430@anthem.concentric.net> 
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>  
            <14886.28397.991041.190430@anthem.concentric.net> 
Message-ID: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>

> {}.pop() sounds like the right thing given its symmetry with [].pop().
> 
> It would have the semantics that it would remove and return a random
> /item/ from the dictionary, which the implementation can make the
> "first" item for convenience purposes.
> 
> It shouldn't dictate though, that {}.pop() == dict.items().pop() or
> make any other guarantees about which item gets popped, because other
> implementations (or indeed, other mapping-conformant objects) may use
> different underlying representations that could make this inefficient.
> 
> For added symmetry, {}.pop() should take an optional argument which is
> the key of the item to remove and return.

That seems a nice touch, until you realize that [].pop() returns the
value, while {}.pop() returns a (key, value) pair.  Also: if you
already know the key, the argument that you can't do it in O(1) time
is no longer valid.  So, a -0.5.

One more concern: if you repeatedly remove the *first* item, the hash
table will start looking lobsided.  Since we don't resize the hash
table on deletes, maybe picking an item at random (but not using an
expensive random generator!) would be better.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gvwilson at nevex.com  Thu Nov 30 16:21:24 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Thu, 30 Nov 2000 10:21:24 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <14886.28397.991041.190430@anthem.concentric.net>
Message-ID: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>

> Barry Warsaw wrote:
> {}.pop() sounds like the right thing given its symmetry with [].pop().
> [details]

Should we then add dict.push(), which would add a key/value pair to the
dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
but I think anyone who's gone through a CS course in data structures
and algorithms will expect push() where there's pop(), and it will allow
functions to be polymorphic over lists and dicts.

Greg (who's not lobbying for it, just wondering if it's a loose end)



From mwh21 at cam.ac.uk  Thu Nov 30 16:35:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Thu, 30 Nov 2000 15:35:31 +0000 (GMT)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re:
 {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <Pine.LNX.4.10.10011301534070.27976-100000@localhost.localdomain>

On Thu, 30 Nov 2000, Greg Wilson wrote:

> > Barry Warsaw wrote:
> > {}.pop() sounds like the right thing given its symmetry with [].pop().
> > [details]
> 
> Should we then add dict.push(), which would add a key/value pair to the
> dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
> but I think anyone who's gone through a CS course in data structures
> and algorithms will expect push() where there's pop(), and it will allow
> functions to be polymorphic over lists and dicts.

'cept there's no list.push - and I don't think you're proposing spelling
"dict.push" "dict.append", are you?

Cheers,
M.




From fdrake at acm.org  Thu Nov 30 16:34:50 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 10:34:50 -0500 (EST)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
References: <14886.28397.991041.190430@anthem.concentric.net>
	<NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <14886.29594.947340.480556@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Should we then add dict.push(), which would add a key/value pair to the
 > dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),

  We don't have [].push(), and I don't think the metaphor really works
well with dictionaries.  There's also an implied ordering relationship
between push() and pop(), and this doesn't hold for dictionaries
either.

 > but I think anyone who's gone through a CS course in data structures
 > and algorithms will expect push() where there's pop(), and it will allow
 > functions to be polymorphic over lists and dicts.

  There's only a limited amount on conceptual polymorphism between the
two; artificial extending that may lead to a lot of programmers using
the wrong structure for their application.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From gstein at lyra.org  Thu Nov 30 18:08:10 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 09:08:10 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <200011301231.EAA22228@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Thu, Nov 30, 2000 at 04:31:07AM -0800
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
Message-ID: <20001130090810.A25840@lyra.org>

On Thu, Nov 30, 2000 at 04:31:07AM -0800, Moshe Zadka wrote:
>...
> *** dictobject.c	2000/09/01 23:29:27	2.65
> --- dictobject.c	2000/11/30 12:31:00	2.66
> ***************
> *** 710,713 ****
> --- 710,782 ----
>   
>   static PyObject *
> + dict_firstkey(register dictobject *mp, PyObject *args)
> + {
> + 	register int i;
> + 
> + 	if (!PyArg_NoArgs(args))
> + 		return NULL;

PyArg_NoArgs() is deprecated because you cannot specify the method name. The
code should be:

    if (!PyArg_ParseTuple(args, ":firstkey"))
        return NULL;

(same in the next two funcs)

>...
> + 		if (mp->ma_table[i].me_value != NULL) {
> + 			PyObject *key = mp->ma_table[i].me_key;
> + 			PyObject *value = mp->ma_table[i].me_value;
> + 			PyObject *item = PyTuple_New(2);
> + 			if (item == NULL) {
> + 				return NULL;
> + 			}
> + 			Py_INCREF(key);
> + 			PyTuple_SetItem(item, 0, key);
> + 			Py_INCREF(value);
> + 			PyTuple_SetItem(item, 1, value);

These can be PyTuple_SET_ITEM() since you know you have a tuple.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From jeremy at alum.mit.edu  Thu Nov 30 18:23:04 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 30 Nov 2000 12:23:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130090810.A25840@lyra.org>
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
	<20001130090810.A25840@lyra.org>
Message-ID: <14886.36088.34790.149929@bitdiddle.concentric.net>

Let's back out these changes and put them in a patch.  There is still
a lot of discussion going on.  The patch itself has some small
problems.  And it is incomplete -- no test cases, no documentation,
etc.

Jeremy



From guido at python.org  Thu Nov 30 18:39:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 12:39:16 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:23:04 EST."
             <14886.36088.34790.149929@bitdiddle.concentric.net> 
References: <200011301231.EAA22228@slayer.i.sourceforge.net> <20001130090810.A25840@lyra.org>  
            <14886.36088.34790.149929@bitdiddle.concentric.net> 
Message-ID: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>

> Let's back out these changes and put them in a patch.  There is still
> a lot of discussion going on.  The patch itself has some small
> problems.  And it is incomplete -- no test cases, no documentation,
> etc.

Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
clever with cvs admin, just use -j.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Thu Nov 30 19:13:24 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 13:13:24 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
Message-ID: <14886.39108.175356.483917@anthem.concentric.net>

I was just surprised by something I've never observed before from
string.capitalize().  There is clearly either a bug (really two) in
the code or in the documentation.

>>> '__Myfoo'.capitalize()
'__myfoo'

Huh?  The documentation says:

    capitalize () 
       Return a copy of the string with only its first character capitalized.

So it's only doing one of the three things it's promising!  For one
thing, it's modifying something other than the first character, and
for another, it's actually swapping the case of the character it /is/
changing.  At least it returns a copy. :)

Given the documentation, I would have expected the return value to be
the same as the original string, i.e. unchanged.

So which is it?  Does the description of the method need to be made
more complicated, or does the code need to be simplified <wink>?

-Barry



From mal at lemburg.com  Thu Nov 30 19:19:29 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:19:29 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A269A31.35FE0AEF@lemburg.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize ()
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)
> 
> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

Huh ? The "documentation" says that you will get a copy in which
only the first character is capitalized (and all others are
converted to lower-case).

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

I'd say this is a documentation bug (and I'm authorized since
I wrote that snippet called "documentation" ;-).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mwh21 at cam.ac.uk  Thu Nov 30 19:21:42 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:21:42 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: barry@digicool.com's message of "Thu, 30 Nov 2000 13:13:24 -0500"
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>

barry at digicool.com (Barry A. Warsaw) writes:

> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize () 
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)

>>> 'AaAaAa'.capitalize()
'Aaaaaa'

That's not a ridiculous reading of the above docs.  It all depends
whether you think "being capitalized" is a property or an action, I
guess.

> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

I would probably have expected that, too.  But I'm not really sure
why.

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

Clarifying the docs won't break any code.  Not sure that changing the
code will much, either.

Oooh, here's something a bit more serious though:

>>> u'aAaAaA'.capitalize()
u'AAaAaA'

Something obviously Needs To Be Done.  My hunch is to change
string_capitalize, but that may be just me (and probably Barry).

Cheers,
M.

-- 
  Famous remarks are very seldom quoted correctly.
                                                    -- Simeon Strunsky




From mal at lemburg.com  Thu Nov 30 19:38:04 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:38:04 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A269E8C.6E1E0B05@lemburg.com>

> Oooh, here's something a bit more serious though:
> 
> >>> u'aAaAaA'.capitalize()
> u'AAaAaA'
> 
> Something obviously Needs To Be Done.  My hunch is to change
> string_capitalize, but that may be just me (and probably Barry).

Ouch. That's a bug.

Here's what the string.py module has to say about this BTW:

# Capitalize a string, e.g. "aBc  dEf" -> "Abc  def".
def capitalize(s):
    """capitalize(s) -> string

    Return a copy of the string s with only its first character
    capitalized.

    """
    return s.capitalize()


Note that .title() is very similar to the way string.capitalize()
works. unicode.title() also uses the title case information available
for Unicode characters.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mwh21 at cam.ac.uk  Thu Nov 30 19:47:37 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:47:37 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 19:38:04 +0100"
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>
Message-ID: <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> > Oooh, here's something a bit more serious though:
> > 
> > >>> u'aAaAaA'.capitalize()
> > u'AAaAaA'
> > 
> > Something obviously Needs To Be Done.  My hunch is to change
> > string_capitalize, but that may be just me (and probably Barry).
> 
> Ouch. That's a bug.

Yes.
 
> Here's what the string.py module has to say about this BTW:

I said "string_capitalize", i.e. "stringobject.c:string_capitalize",
i.e I think 

>>> 'aAaA'.capitalize()

should result in

'AAaA'

We're not doing too well at understanding each other today, are we?

> Note that .title() is very similar to the way string.capitalize()
> works. unicode.title() also uses the title case information available
> for Unicode characters.

Ah yes.  So in the interests of reducing redunancy, .capitalize should
probably do something usefully different...

Cheers,
M.

-- 
59. In English every word can be verbed. Would that it were so in
    our programming languages.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From jim at interet.com  Thu Nov 30 19:49:28 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Thu, 30 Nov 2000 13:49:28 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A26A138.350235C@interet.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:

I guess I am a total geek, but it seems right to me.  The
capital of "_" is "_", and the rest was converted to lower
case.

JimA



From moshez at math.huji.ac.il  Thu Nov 30 19:55:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 20:55:56 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects
 dictobject.c,2.65,2.66
In-Reply-To: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>

On Thu, 30 Nov 2000, Guido van Rossum wrote:

> > Let's back out these changes and put them in a patch.  There is still
> > a lot of discussion going on.  The patch itself has some small
> > problems.  And it is incomplete -- no test cases, no documentation,
> > etc.
> 
> Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> clever with cvs admin, just use -j.

OK, I'll try. If CVS gets the best of me, I'll ask for your help...
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Thu Nov 30 19:56:45 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:56:45 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com> <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A26A2ED.EFF487C3@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Oooh, here's something a bit more serious though:
> > >
> > > >>> u'aAaAaA'.capitalize()
> > > u'AAaAaA'
> > >
> > > Something obviously Needs To Be Done.  My hunch is to change
> > > string_capitalize, but that may be just me (and probably Barry).
> >
> > Ouch. That's a bug.
> 
> Yes.
> 
> > Here's what the string.py module has to say about this BTW:
> 
> I said "string_capitalize", i.e. "stringobject.c:string_capitalize",

The string methods were built closely after the string module
functions, so that's what the reference is...

> i.e I think
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'
> 
> We're not doing too well at understanding each other today, are we?

:)
 
> > Note that .title() is very similar to the way string.capitalize()
> > works. unicode.title() also uses the title case information available
> > for Unicode characters.
> 
> Ah yes.  So in the interests of reducing redunancy, .capitalize should
> probably do something usefully different...

Depends... we'll still need some sort of compatibility with
string.py.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gstein at lyra.org  Thu Nov 30 20:12:54 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 11:12:54 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 30, 2000 at 08:55:56PM +0200
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial>
Message-ID: <20001130111254.N25840@lyra.org>

On Thu, Nov 30, 2000 at 08:55:56PM +0200, Moshe Zadka wrote:
> On Thu, 30 Nov 2000, Guido van Rossum wrote:
> 
> > > Let's back out these changes and put them in a patch.  There is still
> > > a lot of discussion going on.  The patch itself has some small
> > > problems.  And it is incomplete -- no test cases, no documentation,
> > > etc.
> > 
> > Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> > clever with cvs admin, just use -j.
> 
> OK, I'll try. If CVS gets the best of me, I'll ask for your help...

Not sure what -j does, but you can just do a reverse diff. For example:

$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

Apply the patch to your working copy, then check it in.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From akuchlin at mems-exchange.org  Thu Nov 30 20:18:29 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 30 Nov 2000 14:18:29 -0500
Subject: [Python-Dev] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130111254.N25840@lyra.org>; from gstein@lyra.org on Thu, Nov 30, 2000 at 11:12:54AM -0800
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial> <20001130111254.N25840@lyra.org>
Message-ID: <20001130141829.B20259@kronos.cnri.reston.va.us>

On Thu, Nov 30, 2000 at 11:12:54AM -0800, Greg Stein wrote:
>Not sure what -j does, but you can just do a reverse diff. For example:
>
>$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

-j basically does the work for you; you could do:
$ cvs update -j2.66 -j2.65 dictobject.c 

And then do a commit.  See the CVS book: 
http://cvsbook.red-bean.com/cvsbook.html#Examining%20And%20Reverting%20Changes

--amk



From barry at digicool.com  Thu Nov 30 20:31:00 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 14:31:00 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
	<3A269A31.35FE0AEF@lemburg.com>
Message-ID: <14886.43764.472482.888678@anthem.concentric.net>

    MAL> Huh ? The "documentation" says that you will get a copy in
    MAL> which only the first character is capitalized (and all others
    MAL> are converted to lower-case).

I think if the documentation actually said exactly that, I wouldn't
have been confused. :)

>>>>> "MH" == Michael Hudson <mwh21 at cam.ac.uk> writes:

    MH> That's not a ridiculous reading of the above docs.  It all
    MH> depends whether you think "being capitalized" is a property or
    MH> an action, I guess.

Good way to look at it.

I vote to clarify the documentation.

-Barry



From guido at python.org  Thu Nov 30 20:53:23 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 14:53:23 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: Your message of "30 Nov 2000 18:47:37 GMT."
             <m3itp55mba.fsf@atrus.jesus.cam.ac.uk> 
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>  
            <m3itp55mba.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011301953.OAA27996@cj20424-a.reston1.va.home.com>

> i.e I think 
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'

No.  The capitalize() method, which descends from the
string.capitalize(), and which definitely should return 'Aaaa' here.

This means that (a) the documentation needs some clarification, and
(b) the Unicode capitalize() method is broken and needs to be fixed.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From tim.one at home.com  Thu Nov 30 23:46:52 2000
From: tim.one at home.com (Tim Peters)
Date: Thu, 30 Nov 2000 17:46:52 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEADICAA.tim.one@home.com>

[Guido]
> ...
> One more concern: if you repeatedly remove the *first* item, the hash
> table will start looking lobsided.  Since we don't resize the hash
> table on deletes, maybe picking an item at random (but not using an
> expensive random generator!) would be better.

Which is the reason SETL doesn't specify *which* set item is removed:  if
you always start looking at "the front" of a dict that's being consumed, the
dict fills with turds without shrinking, you skip over them again and again,
and consuming the entire dict is still quadratic time.

Unfortunately, while using a random start point is almost always quicker
than that, the expected time for consuming the whole dict remains quadratic.

The clearest way around that is to save a per-dict search finger, recording
where the last search left off.  Start from its current value.  Failure if
it wraps around.  This is linear time in non-pathological cases (a
pathological case is one in which it isn't linear time <wink>).




From gregor at hoffleit.de  Sat Nov  4 00:14:15 2000
From: gregor at hoffleit.de (Gregor Hoffleit)
Date: Sat, 4 Nov 2000 00:14:15 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Include pyport.h,2.20,2.21
In-Reply-To: <200010050142.SAA08326@slayer.i.sourceforge.net>; from tim_one@users.sourceforge.net on Wed, Oct 04, 2000 at 06:42:32PM -0700
References: <200010050142.SAA08326@slayer.i.sourceforge.net>
Message-ID: <20001104001415.A2093@53b.hoffleit.de>

FYI: This misdefinition with LONG_BIT was due to a bug in glibc's limits.h. It
has been fixed in glibc 2.96.

    Gregor
    


On Wed, Oct 04, 2000 at 06:42:32PM -0700, Tim Peters wrote:
> Update of /cvsroot/python/python/dist/src/Include
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv5758/python/dist/src/Include
> 
> Modified Files:
> 	pyport.h 
> Log Message:
> Move LONG_BIT from intobject.c to pyport.h.  #error if it's already been
> #define'd to an unreasonable value (several recent gcc systems have
> misdefined it, causing bogus overflows in integer multiplication).  Nuke
> CHAR_BIT entirely.



From moshez at math.huji.ac.il  Wed Nov  1 15:38:38 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 1 Nov 2000 16:38:38 +0200 (IST)
Subject: [Python-Dev] Bug Changes
Message-ID: <Pine.GSO.4.10.10011011635030.21805-200000@sundial>

I've noticed the SF-FAQ still has the old "Use Jitterbug" thing about
bugs, even though we've moved to SourceForge bug manager. Attached is
a patch to correct everything. 

I haven't checked it in, because I'm not sure I my explanations are
clear at all. I'd be happy if someone went over it and saw if it's all
right.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez
-------------- next part --------------
Index: sf-faq.html
===================================================================
RCS file: /cvsroot/python/python/nondist/sf-html/sf-faq.html,v
retrieving revision 1.15
diff -c -r1.15 sf-faq.html
*** sf-faq.html	2000/08/25 07:55:48	1.15
--- sf-faq.html	2000/11/01 14:33:50
***************
*** 57,63 ****
  <h2><a href="#bugs">6. Bug Reporting</a></h2>
  <ol>
    <li><a href="#b1">Where can I submit/view bugs for Python?</a></li>
!   <li><a href="#b2">How do I use jitterbug?</a></li>
  </ol>
  
  <h2><a href="#appendix">A. Appendix</a></h2>
--- 57,63 ----
  <h2><a href="#bugs">6. Bug Reporting</a></h2>
  <ol>
    <li><a href="#b1">Where can I submit/view bugs for Python?</a></li>
!   <li><a href="#b2">How do I use the sourceforge bug manager?</a></li>
  </ol>
  
  <h2><a href="#appendix">A. Appendix</a></h2>
***************
*** 446,472 ****
  <h4>Q: Where can I submit/view bugs for Python?</h4>
  
  <h4>A:</h4>
! As of now [25-Jul-200] the Python project does not use SourceForge's bug
! tracking facility. This may change when there is a way to import the existing
! jitterbug database. The jitterbug database can be accessed through the
! following interface:
  
- <blockquote>
-   <tt><a
-   href="http://www.python.org/python-bugs">http://www.python.org/python-bugs</a></tt></blockquote>
- 
  <h3><a name="b1" id="b1"></a>6.1.:</h3>
  
! <h4>Q: How do I use jitterbug?</h4>
  
  <h4>A:</h4>
! To get the list of open bugs click on <tt>open</tt> (hidden between the second
! last and the last horizontal ruler).
  
! <p>To get a list of the bugs which are related to some area, enter an
! appropriate regular expression and press "Select Messages". Then select
! <tt>open</tt> (or whatever category you would like to view) as described
! above.</p>
  
  <h1><a name="appendix" id="appendix"></a>A. Appendix</h1>
  
--- 446,466 ----
  <h4>Q: Where can I submit/view bugs for Python?</h4>
  
  <h4>A:</h4>
! The Python project uses SourceForge's bug
! tracking facility. Go to 
! <a href="http://sourceforge.net/bugs/?group_id=5470">http://sourceforge.net/bugs/?group_id=5470</a> for all bug management needs.
  
  <h3><a name="b1" id="b1"></a>6.1.:</h3>
  
! <h4>Q: How do I use the sourceforge bug manager?</h4>
  
  <h4>A:</h4>
! By default, you will see the list of all Open bugs. You can change
! which bugs you're viewing by selecting the assigned_to/status/area/type
! select boxs.
  
! <p>To submit a bug, use the "Submit a Bug" link, near the top of the page.
! </p>
  
  <h1><a name="appendix" id="appendix"></a>A. Appendix</h1>
  
***************
*** 600,616 ****
      mailing list address; this address should no longer be used for patch
      submission. The patch manager is for <b>patches</b> only; if you have a
      problem or suggestion but don't know how to write the code for it, use the
!     <a href="http://www.python.org/search/search_bugs.html">Python Bugs
!     List</a> instead. The bugs list is searchable; if you have a problem and
      you're not sure if it has been reported or fixed already, this is the
      first place to look. (There used to be a separate TODO list; we now prefer
      that you use the bugs list for suggestions and requests too.)</li>
    <li style="list-style: none"><b>Submit documentation patches the same
      way.</b> When adding the patch, be sure to set the "<b>Category</b>" field
      to "<b>documentation</b>". For documentation errors without patches,
!     please use the <a
!     href="http://www.python.org/search/search_bugs.html">Python Bugs List</a>
!     instead.</li>
    <li>We like context diffs. We grudgingly accept unified diffs. <b>Straight
      ("ed-style") diffs are right out!</b> If you don't know how to generate
      context diffs, you're probably not qualified to produce high-quality
--- 594,608 ----
      mailing list address; this address should no longer be used for patch
      submission. The patch manager is for <b>patches</b> only; if you have a
      problem or suggestion but don't know how to write the code for it, use the
!     <a href="#b1">bug reporting mechanism</a> instead. 
!     The bugs list is searchable; if you have a problem and
      you're not sure if it has been reported or fixed already, this is the
      first place to look. (There used to be a separate TODO list; we now prefer
      that you use the bugs list for suggestions and requests too.)</li>
    <li style="list-style: none"><b>Submit documentation patches the same
      way.</b> When adding the patch, be sure to set the "<b>Category</b>" field
      to "<b>documentation</b>". For documentation errors without patches,
!     please use the <a href="b1">bug reporting mechanism</a>.</li>
    <li>We like context diffs. We grudgingly accept unified diffs. <b>Straight
      ("ed-style") diffs are right out!</b> If you don't know how to generate
      context diffs, you're probably not qualified to produce high-quality

From mwh21 at cam.ac.uk  Wed Nov  1 19:13:07 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 01 Nov 2000 18:13:07 +0000
Subject: [Python-Dev] Python Call Mechanism 
In-Reply-To: Jeremy Hylton's message of "Mon, 30 Oct 2000 09:59:00 -0500 (EST)"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>
Message-ID: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>

Jeremy Hylton <jeremy at alum.mit.edu> writes:

> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> Jeremy Hylton wrote:
>   >>
>   >> Update of /cvsroot/python/python/dist/src/Python In directory
>   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
>   >>
>   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
>   >> implementation is getting really hairy; should review it to see
>   >> if it can be simplified.
> 
>   MAL> How about a complete redesign of the whole call mechanism ?!
>
[chomp]
> 
> I'd be interested in looking at it.

Random idea that occurred while answering a post on comp.lang.python:

How about dumping the CALL_FUNCTION* opcodes, and replacing them with
two non-argumented opcodes, called for the sake of argument NCALL_FUNC
and NCALL_FUNC_KW.

NCALL_FUNC would pop a function object and a tuple off the stack and
apply the function to the tuple.

NCALL_FUNC_KW would do the same, then pop a dictionary and then do
the moral equivalent of f(*args,**kw).

As a preliminary it would be sensible to rework BUILD_MAP so that it
built dictionaries off the stack (a bit like BUILD_LIST, and like
CALL_FUNCTION now does with keyword arguments...) (and extend the
compiler to use this for literal dictionaries).

This would add an opcode or so per function call, but would probably
make life much simpler.

No time for implementation tonight, but could probably knock something
up tomorrow (depending how hard it turns out to be).

Thoughts?  Is that like what you did, Marc?

M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp




From jeremy at alum.mit.edu  Wed Nov  1 20:06:46 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:06:46 -0500 (EST)
Subject: [Python-Dev] Python Call Mechanism 
In-Reply-To: <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
References: <200010301715.JAA32564@slayer.i.sourceforge.net>
	<39FDB5EA.B60EA39A@lemburg.com>
	<14845.36020.17063.951147@bitdiddle.concentric.net>
	<m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <14848.27078.923932.758419@bitdiddle.concentric.net>

My first impression is that this sounds like a nice simplifcation.

One question is how expensive this is for the common case.  Right now
arguments are pushed on the interpreter stack before CALL_FUNCTION is
executed, which is just a pointer assignment.  The pointers on the
stack are then assigned into the fast locals of the function after the
call.  Your scheme sounds like it would increase all function calls by
the cost of a tuple allocation.

It certainly wouldn't hurt to implement this, as it would provide some
practical implementation experience that would inform a PEP on the
subject.

On a related note, I have proposed a pep to add nested lexical scopes
for Python 2.1.  Barry's away for the moment, so it hasn't been
assigned a number yet.  It's just a proposal, not sure what Guido will
say in the end, but it also involves revising the function call
architecture.  I'll send a copy of the current draft (just notes)
under a separate subject.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 20:07:10 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 14:07:10 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
Message-ID: <14848.27102.223001.369662@bitdiddle.concentric.net>

Title: Statically Nested Scopes
Author: Jeremy Hylton <jeremy at digicool.com>
Status: Draft
Type: Standards Track
Created: 01-Nov-2000

Abstract

    This PEP proposes the additional of statically nested scoping
    (lexical scoping) for Python 2.1.  The current language definition
    defines exactly three namespaces that are used to resolve names --
    the local, global, and built-in namespaces.  The addition of
    nested scopes would allow resolution of unbound local names in
    enclosing functions' namespaces.

    One consequence of this change that will be most visible to Python
    programs is that lambda statements could reference variables in
    the namespaces where the lambda is defined.  Currently, a lambda
    statement uses default arguments to explicitly creating bindings
    in the lambda's namespace.

Notes

    This section describes several issues that will be fleshed out and
    addressed in the final draft of the PEP.  Until that draft is
    ready, please direct comments to the author.

    This change has been proposed many times in the past.  It has
    always been stymied by the possibility of creating cycles that
    could not be collected by Python's reference counting garbage
    collector.  The additional of the cycle collector in Python 2.0
    eliminates this concern.

    Guido once explained that his original reservation about nested
    scopes was a reaction to their overuse in Pascal.  In large Pascal
    programs he was familiar with, block structure was overused as an
    organizing principle for the program, leading to hard-to-read
    code. 

    Greg Ewing developed a proposal "Python Nested Lexical Scoping
    Enhancement" in Aug. 1999.  It is available from
    http://www.cosc.canterbury.ac.nz/~greg/python/lexscope.html

    Michael Hudson's bytecodehacks projects at
    http://sourceforge.net/projects/bytecodehacks/
    provides facilities to support nested scopes using the closure
    module. 

    Examples:

    def make_adder(n):
        def adder(x):
            return x + n
        return adder
    add2 = make_adder(2)
    add2(5) == 7

    
    from Tkinter import *
    root = Tk()
    Button(root, text="Click here",
           command = lambda : root.test.configure(text="..."))

    
    One controversial issue is whether it should be possible to modify
    the value of variables defined in an enclosing scope.  

    One part of the issue is how to specify that an assignment in the
    local scope should reference to the definition of the variable in
    an enclosing scope.  Assignment to a variable in the current scope
    creates a local variable in the scope.  If the assignment is
    supposed to refer to a global variable, the global statement must
    be used to prevent a local name from being created.  Presumably,
    another keyword would be required to specify "nearest enclosing
    scope."

    Guido is opposed to allow modifications (need to clarify exactly
    why).  If you are modifying variables bound in enclosing scopes,
    you should be using a class, he says.

    The problem occurs only when a program attempts to rebind the name
    in the enclosing scope.  A mutable object, e.g. a list or
    dictionary, can be modified by a reference in a nested scope; this
    is an obvious consequence of Python's reference semantics.  The
    ability to change mutable objects leads to an inelegant
    workaround: If a program needs to rebind an immutable object,
    e.g. a number or tuple, store the object in a list and have all
    references to the object use this list:

    def bank_account(initial_balance):
        balance = [initial_balance]
        def deposit(amount):
            balance[0] = balance[0] + amount
        def withdraw(amount):
            balance[0] = balance[0] - amount
        return deposit, withdraw

    I would prefer for the language to support this style of
    programming directly rather than encouraging programs to use this
    somewhat obfuscated style.  Of course, an instance would probably
    be clearer in this case.
 
    One implementation issue is how to represent the environment that
    stores variables that are referenced by nested scopes.  One
    possibility is to add a pointer to each frame's statically
    enclosing frame and walk the chain of links each time a non-local
    variable is accessed.  This implementation has some problems,
    because access to nonlocal variables is slow and causes garbage to
    accumulate unncessarily.  Another possibility is to construct an
    environment for each function that provides access to only the
    non-local variables.  This environment would be explicitly passed
    to nested functions.



From mal at lemburg.com  Wed Nov  1 21:31:08 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:31:08 +0100
Subject: [Python-Dev] Python Call Mechanism
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>
Message-ID: <3A007D8C.A9943D90@lemburg.com>

Michael Hudson wrote:
> 
> Jeremy Hylton <jeremy at alum.mit.edu> writes:
> 
> > >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> >
> >   MAL> Jeremy Hylton wrote:
> >   >>
> >   >> Update of /cvsroot/python/python/dist/src/Python In directory
> >   >> slayer.i.sourceforge.net:/tmp/cvs-serv32349/Python
> >   >>
> >   >> Modified Files: ceval.c ...  N.B. The CALL_FUNCTION
> >   >> implementation is getting really hairy; should review it to see
> >   >> if it can be simplified.
> >
> >   MAL> How about a complete redesign of the whole call mechanism ?!
> >
> [chomp]
> >
> > I'd be interested in looking at it.
> 
> Random idea that occurred while answering a post on comp.lang.python:
> 
> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).
> 
> As a preliminary it would be sensible to rework BUILD_MAP so that it
> built dictionaries off the stack (a bit like BUILD_LIST, and like
> CALL_FUNCTION now does with keyword arguments...) (and extend the
> compiler to use this for literal dictionaries).
> 
> This would add an opcode or so per function call, but would probably
> make life much simpler.
> 
> No time for implementation tonight, but could probably knock something
> up tomorrow (depending how hard it turns out to be).
> 
> Thoughts?  Is that like what you did, Marc?

No, I just cleaned up the intertwine calling scheme currently
implemented in ceval.c. This allows a few improvments, one of
them being the possibility to inline C function calls in the
main loop (anyone ever trace the path Python takes when calling
a builtin function or method... you'd be surprised).

About your idea with the new opcodes: you could be touching
a performance relevant section there -- a ceval round-trip
may cost more than the added if()s in the CALL_FUNCION opcode.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Wed Nov  1 21:37:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:37:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <3A007EF8.2D9BDCF5@lemburg.com>

[pre-PEP]

This will break code... I'm not sure whether it's worth going
down this path just for the sake of being able to define
functions within functions.

Wouldn't it be a better idea to somehow add native acqusition
to Python's objects ?

We already have a slot which implements the "contains"
relationship. All we'd need is a way for a contained
object to register itself with the container in a 
way that doesn't produce cycles.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jeremy at alum.mit.edu  Wed Nov  1 21:48:53 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 15:48:53 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A007EF8.2D9BDCF5@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
Message-ID: <14848.33205.361821.679508@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.

How will this break code?  Any code written to use the scoping rules
will not work today.

Python already allows programs to define functions within functions.
That's not at issue.  The issue is how hard it is to use nested
functions, including lambdas.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?

No.

Seriously, I don't see how acquistion addresses the same issues at
all.  Feel free to explain what you mean.

  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.

The contains relationship has to do with container objects and their
contents.  A function's environment is not a container in the same
sense, so I don't see how this is related.

As I noted in the PEP, I don't see a compelling reason to avoid
cycles.

Jeremy



From mal at lemburg.com  Wed Nov  1 21:51:16 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Nov 2000 21:51:16 +0100
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
		<3A007EF8.2D9BDCF5@lemburg.com> <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <3A008244.8F558C8B@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> [pre-PEP] This will break code... I'm not sure whether it's
>   MAL> worth going down this path just for the sake of being able to
>   MAL> define functions within functions.
> 
> How will this break code?  Any code written to use the scoping rules
> will not work today.
> 
> Python already allows programs to define functions within functions.
> That's not at issue.  The issue is how hard it is to use nested
> functions, including lambdas.

The problem is that with nested scoping, a function defined
within another function will suddenly reference the variables
of the enclosing function as globals and not the module
globals... this could break code.

Another problem is that you can't reach out for the defining
module globals anymore (at least not in an easy way like today).

>   MAL> Wouldn't it be a better idea to somehow add native acqusition
>   MAL> to Python's objects ?
> 
> No.
> 
> Seriously, I don't see how acquistion addresses the same issues at
> all.  Feel free to explain what you mean.

It's not related to *statically* nested scopes, but is to
dynamically nested ones. Acquisition is basically about the
same thing: you acquire attributes from containers. The only
difference here is that the containment relationships are
defined at run-time.
 
>   MAL> We already have a slot which implements the "contains"
>   MAL> relationship. All we'd need is a way for a contained object to
>   MAL> register itself with the container in a way that doesn't
>   MAL> produce cycles.
> 
> The contains relationship has to do with container objects and their
> contents.  A function's environment is not a container in the same
> sense, so I don't see how this is related.
> 
> As I noted in the PEP, I don't see a compelling reason to avoid
> cycles.

Ok, we have cycle GC, but why create cycles when you don't
have to (Python can also be run without GC and then you'd run
into problems...) ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From skip at mojam.com  Wed Nov  1 22:57:17 2000
From: skip at mojam.com (Skip Montanaro)
Date: Wed, 1 Nov 2000 15:57:17 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <14848.37309.3616.710295@beluga.mojam.com>

    MAL> [pre-PEP] This will break code...

    Jeremy> How will this break code?

Suppose you have

    x = 1
    def f1():
        x = 2
        def inner():
            print x
        inner()

Today, calling f1() prints "1".  After your proposed changes I suspect it
would print "2".

Skip



From jeremy at alum.mit.edu  Wed Nov  1 22:18:49 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:18:49 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
	<3A008244.8F558C8B@lemburg.com>
Message-ID: <14848.35001.109767.606384@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> [pre-PEP] This will break code... I'm not sure whether it's
  MAL> worth going down this path just for the sake of being able to
  MAL> define functions within functions.
  >>
  >> How will this break code?  Any code written to use the scoping
  >> rules will not work today.
  >>
  >> Python already allows programs to define functions within
  >> functions.  That's not at issue.  The issue is how hard it is to
  >> use nested functions, including lambdas.

  MAL> The problem is that with nested scoping, a function defined
  MAL> within another function will suddenly reference the variables
  MAL> of the enclosing function as globals and not the module
  MAL> globals... this could break code.

That's right it could, in the unlikely case that someone has existing
code today using nested functions where an intermediate function
defines a local variable that shadows a variable defined in an
enclosing scope.  

It should be straightfoward to build a tool that would detect this
case.

It would be pretty poor programming style, so I think it would be fine
to break backwards compatibility here.

  MAL> Another problem is that you can't reach out for the defining
  MAL> module globals anymore (at least not in an easy way like
  MAL> today).

I think we would want to keep globals implemented just the way they
are.  The compiler would need to determine exactly which variables are
access from enclosing scopes and which are globals.

  MAL> Wouldn't it be a better idea to somehow add native acqusition
  MAL> to Python's objects ?
  >>
  >> Seriously, I don't see how acquistion addresses the same issues
  >> at all.  Feel free to explain what you mean.

  MAL> It's not related to *statically* nested scopes, but is to
  MAL> dynamically nested ones. Acquisition is basically about the
  MAL> same thing: you acquire attributes from containers. The only
  MAL> difference here is that the containment relationships are
  MAL> defined at run-time.

Static scoping and dynamic scoping are entirely different beasts,
useful for different things.  I want to fix, among other things,
lambdas.  That's a static issue.
 
  MAL> We already have a slot which implements the "contains"
  MAL> relationship. All we'd need is a way for a contained object to
  MAL> register itself with the container in a way that doesn't
  MAL> produce cycles.
  >>
  >> The contains relationship has to do with container objects and
  >> their contents.  A function's environment is not a container in
  >> the same sense, so I don't see how this is related.
  >>
  >> As I noted in the PEP, I don't see a compelling reason to avoid
  >> cycles.

  MAL> Ok, we have cycle GC, but why create cycles when you don't have
  MAL> to (Python can also be run without GC and then you'd run into
  MAL> problems...) ?

If we can avoid cycles, sure.  I would prefer a simple design that
allowed cycles to a complicated design that avoided them.  Exactly
where to draw the line between simple and complex is a matter of
taste, of course.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 22:20:08 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 16:20:08 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.37309.3616.710295@beluga.mojam.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<3A007EF8.2D9BDCF5@lemburg.com>
	<14848.33205.361821.679508@bitdiddle.concentric.net>
	<14848.37309.3616.710295@beluga.mojam.com>
Message-ID: <14848.35080.73182.888834@bitdiddle.concentric.net>

Thanks.  I expect there is very little code that depends on this sort
of behavior, since it is confusing to read.  Many readers,
particularly novices, could reasonably expect Python to print 2 now.

As I explained to MAL, I think we would need to provide a code
analysis tool that identified these problems.  It's probably helpful
to generate warning about this right now, since it's rather
obfuscated.

Jeremy



From jeremy at alum.mit.edu  Wed Nov  1 23:35:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:35:14 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
Message-ID: <14848.39586.832800.139182@bitdiddle.concentric.net>

I just did a clean configure and make from the latest CVS tree.  It
seems to get stuck in a loop calling makesetup over and over again.
(Output below.)

Any idea what's going wrong?

Jeremy

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
making Makefile in subdirectory .
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3'
making Makefile in subdirectory Parser
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Parser'
making Makefile in subdirectory Grammar
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
make[1]: Nothing to be done for `Makefile'.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Grammar'
making Makefile in subdirectory Objects
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Objects'
making Makefile in subdirectory Python
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Python'
make[1]: `Makefile' is up to date.
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Python'
making Makefile in subdirectory Modules
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
cat: Setup.local: No such file or directory
cat: Setup: No such file or directory
make[1]: Leaving directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
make[1]: Entering directory `/home/jeremy/src/python/dist/src/build-O3/Modules'
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
-----------------------------------------------

[etc.]



From thomas at xs4all.net  Wed Nov  1 23:41:34 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 1 Nov 2000 23:41:34 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <200011012237.OAA06642@slayer.i.sourceforge.net>; from twouters@users.sourceforge.net on Wed, Nov 01, 2000 at 02:37:39PM -0800
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
Message-ID: <20001101234134.O12812@xs4all.nl>

On Wed, Nov 01, 2000 at 02:37:39PM -0800, Thomas Wouters wrote:

> Modified Files:
> 	pep-0204.txt 
> Log Message:

> Update this PEP to current, harsh, reality. It's been rejected :) If at all
> possible, the reasoning should be extended to include the real reasons it
> was rejected -- this is just guesswork from my side. (This means you, Guido,
> or anyone who can channel Guido enough to call himself Guido.)

In addition to that, PEP 0 also needs to be updated. Shall I do that myself,
now that Barry is apparently away ? While I was at it, I also noticed PEP
0200 still says 'Incomplete', though that might be by design.

Yay-another-first-for-me---first-rejected-PEP-ly y'rs, ;)
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jeremy at alum.mit.edu  Wed Nov  1 23:54:07 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 1 Nov 2000 17:54:07 -0500 (EST)
Subject: [Python-Dev] Move idle PEPs to status deferred
Message-ID: <14848.40719.914227.606929@bitdiddle.concentric.net>

Barry,

We should start working on any PEPs that are going to be considered
for Python 2.1.  There are a bunch of old PEPs that have been
assigned and then ignored.  You have marked many of them as deferred.
We should assign new authors for the PEPs we care about and move all
the other deferred PEPs somewhere else in the index.

The following PEPs have been inactive for at least two months:
 I     2  pep-0002.txt  Procedure for Adding New Modules       Raymond
 S   202  pep-0202.txt  List Comprehensions                    Peters
 SD  205  pep-0205.txt  Weak References                        Drake
 I   206  pep-0206.txt  2.0 Batteries Included                 Zadka
 SD  207  pep-0207.txt  Rich Comparisons                       Ascher
 SD  208  pep-0208.txt  Reworking the Coercion Model           Ascher
 SD  209  pep-0209.txt  Adding Multidimensional Arrays         Ascher
 SD  210  pep-0210.txt  Decoupling the Interpreter Loop        Ascher
 SD  211  pep-0211.txt  Adding New Linear Algebra Operators    Wilson
 SD  212  pep-0212.txt  Loop Counter Iteration                 Schneider-Kamp
 SD  213  pep-0213.txt  Attribute Access Handlers              Prescod
 SD  215  pep-0215.txt  String Interpolation                   Yee
 I   216  pep-0216.txt  Docstring Format                       Zadka
 SD  217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 SD  218  pep-0218.txt  Adding a Built-In Set Object Type      Wilson
 SD  219  pep-0219.txt  Stackless Python                       McMillan
 I   220  pep-0220.txt  Coroutines, Generators, Continuations  McMillan
 S   224  pep-0224.txt  Attribute Docstrings                   Lemburg

I think we should find new authors for PEPs 207 and 208 and work on
them for 2.1.  I imagine David would be happy to pass the torch on
these issues.

I assume Gordon will be working on the stackless PEPs, but we should
confirm this.

For all of the other PEPs, authors who want to have them considered
for 2.1 should provide some updates to their PEPs.

We should also simplfy the PEP index so that deferred PEPs are
collected at the bottom or something like that.

Jeremy



From greg at cosc.canterbury.ac.nz  Thu Nov  2 01:34:04 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:34:04 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.33205.361821.679508@bitdiddle.concentric.net>
Message-ID: <200011020034.NAA29444@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy at alum.mit.edu>:

> Seriously, I don't see how acquistion addresses the same issues at
> all.

My proposal for nested scopes was actually an acquisition-like
mechanism. The idea was to avoid unbreakable cycles by deferring
the creation of a closure from when the function is defined
to when it is used.

Guido rejected my implementation for various good reasons.
It could be modified to overcome most of those objections,
however.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Thu Nov  2 01:37:00 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2000 13:37:00 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A008244.8F558C8B@lemburg.com>
Message-ID: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> The problem is that with nested scoping, a function defined
> within another function will suddenly reference the variables
> of the enclosing function

This could be avoided by requiring that variables which are
to be visible in an inner scope be marked somehow in the
scope where they are defined.

I don't think it's a serious enough problem to be worth
fixing that way, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From gmcm at hypernet.com  Thu Nov  2 02:37:09 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Wed, 1 Nov 2000 20:37:09 -0500
Subject: [Python-Dev] Move idle PEPs to status deferred
In-Reply-To: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <3A007EF5.21540.6BA02C9@localhost>

Jeremy wrote:

> I assume Gordon will be working on the stackless PEPs, but we should
> confirm this.

Yes, I will.

- Gordon



From tim_one at email.msn.com  Thu Nov  2 08:02:04 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 02:02:04 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

Note that this problem will be much worse in Python:  in Pascal, you could
always "look up" for the closest-containing func/proc that explicitly
declares a referenced vrbl.  In Python, you have to indirectly *deduce*
which vrbls are local to a def, by searching the entire body for an
appearance as a binding target.  So you have to "look up" and "look down"
from the reference point, and it's easy to miss a binding target.

    i = 6
    def f(x):
        def g():
            print i
        # ...
        # skip to the next page
        # ...
        for i in x:  # ah, i *is* local to f, so this is what g sees
            pass
        g()

>    def bank_account(initial_balance):
>        balance = [initial_balance]
>        def deposit(amount):
>            balance[0] = balance[0] + amount
>        def withdraw(amount):
>            balance[0] = balance[0] - amount
>        return deposit, withdraw

Unfortunately for proponents, this is exactly the kind of SICP example that
is much better done via a class.  Not only is the closure version strained
by comparison, but as is usual it manages to create a bank account with a
write-only balance <0.9 wink>.

    def deposit(amount):
        global bank_account.balance
        balance += amount

is one old suggested way to explicitly declare non-local names and the
enclosing block to which they are local (and in analogy with current
"global", mandatory if you want to rebind the non-local name, optional if
you only want to reference it).  There are subtleties here, but explicit is
better than implicit, and the subtleties are only subtler if you refuse
(like Scheme) to make the intent explicit.

for-real-fun-think-about-"exec"-abuses-ly y'rs  - tim





From moshez at math.huji.ac.il  Thu Nov  2 09:32:11 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 10:32:11 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>

[Jeremy Hylton]
> ...
>     Guido once explained that his original reservation about nested
>     scopes was a reaction to their overuse in Pascal.  In large Pascal
>     programs he was familiar with, block structure was overused as an
>     organizing principle for the program, leading to hard-to-read
>     code.

[Tim Peters]
> Note that this problem will be much worse in Python:  in Pascal, you could
> always "look up" for the closest-containing func/proc that explicitly
> declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> which vrbls are local to a def, by searching the entire body for an
> appearance as a binding target.  So you have to "look up" and "look down"
> from the reference point, and it's easy to miss a binding target.

This is a tool problem, and should be solved with good tools.
Of course, installing the corret tools in people's minds will require
some technological discoveries.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From gstein at lyra.org  Thu Nov  2 09:46:04 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 00:46:04 -0800 (PST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
Message-ID: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>

On Thu, 2 Nov 2000, Moshe Zadka wrote:
> [Tim Peters]
> > Note that this problem will be much worse in Python:  in Pascal, you could
> > always "look up" for the closest-containing func/proc that explicitly
> > declares a referenced vrbl.  In Python, you have to indirectly *deduce*
> > which vrbls are local to a def, by searching the entire body for an
> > appearance as a binding target.  So you have to "look up" and "look down"
> > from the reference point, and it's easy to miss a binding target.
> 
> This is a tool problem, and should be solved with good tools.
> Of course, installing the corret tools in people's minds will require
> some technological discoveries.

Bleck. Those tools are a crutch to deal with a poor language design /
feature. And are those tools portable? Are they part of everybody's
standard tool set? Will vi, emacs, and MS DevStudio all have those
capabilities?

Not a chance. Personally, I'll take Guido's point of view and say they are
inherently hard to deal with; therefore, punt them.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/





From mal at lemburg.com  Thu Nov  2 13:12:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 13:12:42 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
Message-ID: <3A015A3A.4EC7DBC6@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com>:
> 
> > The problem is that with nested scoping, a function defined
> > within another function will suddenly reference the variables
> > of the enclosing function
> 
> This could be avoided by requiring that variables which are
> to be visible in an inner scope be marked somehow in the
> scope where they are defined.
> 
> I don't think it's a serious enough problem to be worth
> fixing that way, though.

It may not look serious, but changing the Python lookup scheme
is, since many inspection tools rely and reimplement exactly
that scheme. With nested scopes, there would be next to no
way to emulate the lookups using these tools.

To be honest, I don't think static nested scopes buy us all that
much. You can do the same now, by using keyword arguments which
isn't all that nice, but works great and makes the scope clearly
visible.

Dynamic nested scopes is another topic... those are *very*
useful; especially when it comes to avoiding global variables
and implementing programs which work using control objects
instead of global function calls.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Thu Nov  2 15:30:26 2000
From: barry at wooz.org (barry at wooz.org)
Date: Thu, 2 Nov 2000 09:30:26 -0500 (EST)
Subject: [Python-Dev] Re: Move idle PEPs to status deferred
References: <14848.40719.914227.606929@bitdiddle.concentric.net>
Message-ID: <14849.31362.805142.553781@anthem.concentric.net>

I was away from email for most of the day yesterday.  I'll do a swing
through all the outstanding PEP updates later today -- after I finish
catching up on email. :(

-Barry




From moshez at math.huji.ac.il  Thu Nov  2 15:49:37 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 16:49:37 +0200 (IST)
Subject: [Python-Dev] PEP-0217
Message-ID: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>

I need some help:

1) BDFL pronouncement
2) someone to see about the Jython issue.

Thank you for your co-operation.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From jeremy at alum.mit.edu  Thu Nov  2 16:18:47 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 10:18:47 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
References: <Pine.GSO.4.10.10011021029590.1290-100000@sundial>
	<Pine.LNX.4.10.10011020044130.25590-100000@nebula.lyra.org>
Message-ID: <14849.34263.310260.404940@bitdiddle.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

  GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
  >> This is a tool problem, and should be solved with good tools.  Of
  >> course, installing the corret tools in people's minds will
  >> require some technological discoveries.

  GS> Bleck. Those tools are a crutch to deal with a poor language
  GS> design / feature. And are those tools portable? Are they part of
  GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
  GS> all have those capabilities?

Are you saying that compilers are a crutch and we should get rid of
them?  I don't think you intend that, but this is a completely
straightforward tool to build.  It is needed only for backwards
compatibility -- to identify scripts that depend on the changed
behavior.  There is no need for vi, emacs, or devstudio to understand
what's going on.

Jeremy



From moshez at math.huji.ac.il  Thu Nov  2 16:16:41 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:16:41 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.34263.310260.404940@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> >>>>> "GS" == Greg Stein <gstein at lyra.org> writes:
> 
>   GS> On Thu, 2 Nov 2000, Moshe Zadka wrote:
>   >> This is a tool problem, and should be solved with good tools.  Of
>   >> course, installing the corret tools in people's minds will
>   >> require some technological discoveries.
> 
>   GS> Bleck. Those tools are a crutch to deal with a poor language
>   GS> design / feature. And are those tools portable? Are they part of
>   GS> everybody's standard tool set? Will vi, emacs, and MS DevStudio
>   GS> all have those capabilities?
> 
> Are you saying that compilers are a crutch and we should get rid of
> them?  I don't think you intend that, but this is a completely
> straightforward tool to build.  It is needed only for backwards
> compatibility -- to identify scripts that depend on the changed
> behavior.  There is no need for vi, emacs, or devstudio to understand
> what's going on.

you guys are talking about different things.
Jeremy is talking about a tool to warn against incompatible changes
Greg is talking about a tool to identify, for each variable, what scope
it belongs to.

as-usual-the-answer-is-"you're-both-right"-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  2 04:26:14 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:26:14 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
             <3A015A3A.4EC7DBC6@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
            <3A015A3A.4EC7DBC6@lemburg.com> 
Message-ID: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>

[MAL]
> Dynamic nested scopes is another topic... those are *very*
> useful; especially when it comes to avoiding global variables
> and implementing programs which work using control objects
> instead of global function calls.

Marc-Andre, what are Dynamic nested scopes?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at math.huji.ac.il  Thu Nov  2 16:26:20 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:26:20 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> [MAL]
> > Dynamic nested scopes is another topic... those are *very*
> > useful; especially when it comes to avoiding global variables
> > and implementing programs which work using control objects
> > instead of global function calls.
> 
> Marc-Andre, what are Dynamic nested scopes?

If MAL means dynamic scoping (which I understood he does), then this 
simply means:

when looking for a variable "foo", you first search for it in the local
namespace. If not there, the *caller's* namespace, and so on. In the
end, the caller is the __main__ module, and if not found there, it is
a NameError.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  2 04:29:03 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 22:29:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:12:42 +0100."
             <3A015A3A.4EC7DBC6@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
            <3A015A3A.4EC7DBC6@lemburg.com> 
Message-ID: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>

> It may not look serious, but changing the Python lookup scheme
> is, since many inspection tools rely and reimplement exactly
> that scheme. With nested scopes, there would be next to no
> way to emulate the lookups using these tools.

So fix the tools.

> To be honest, I don't think static nested scopes buy us all that
> much. You can do the same now, by using keyword arguments which
> isn't all that nice, but works great and makes the scope clearly
> visible.

Yes.  It's a hack that gets employed over and over.  And it has
certain problems.  We added 'import as' to get rid of a common
practice that was perceived unclean.  Maybe we should support nested
scopes to get rid of another unclean common practice?

I'm not saying that we definitely should add this to 2.1 (there's
enough on our plate already) but we should at least consider it, and
now that we have cycle GC, the major argument against it (that it
causes cycles) is gone...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at math.huji.ac.il  Thu Nov  2 16:29:55 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 17:29:55 +0200 (IST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>

On Wed, 1 Nov 2000, Guido van Rossum wrote:

> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

This is the perfect moment to ask: what do we have on our plates for 2.1?
Shouldn't we have a list of goals for it or something? As a first-order
approximation, what PEPs are expected to be included? And, most the
conspiracy-theory question, what are Digital Creations' goals for Python?

We now return you to our regularily scheduled bug fixing.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Thu Nov  2 16:34:42 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 10:34:42 -0500
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <Pine.GSO.4.10.10011021727341.1290-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 02, 2000 at 05:29:55PM +0200
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>
Message-ID: <20001102103442.B5027@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
>Shouldn't we have a list of goals for [Python 2.1] or something? As a 
>first-order approximation, what PEPs are expected to be included? And, 

Stuff I personally want to get done:
  * Finish PEP 222, "Web Programming Improvements" and implement whatever
    emerges from it.

  * Write a PEP on using Distutils to build the modules that come with
    Python, and implement it if accepted.

  * Work on something CPAN-like.  This may or may not have repercussions for
    the core; I don't know.

--amk



From thomas at xs4all.net  Thu Nov  2 16:42:39 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 16:42:39 +0100
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102103442.B5027@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Thu, Nov 02, 2000 at 10:34:42AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <20001102164239.R12812@xs4all.nl>

On Thu, Nov 02, 2000 at 10:34:42AM -0500, Andrew Kuchling wrote:

>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Probably not, though perhaps a new module would be nice. As for the
CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
Apachecon, and I think it would be suited extremely well as the transmission
protocol for SPAM (or however you want to call the Python CPAN ;). You can
do the uploading, downloading and searching for modules using WebDAV without
too much pain, and there's excellent WebDAV support for Apache ;)

Is anyone working on something like this, or even thinking about it ? I'm
not deep enough into distutils to join that SIG, but I definately would join
a CPyAN SIG ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From bckfnn at worldonline.dk  Thu Nov  2 16:37:43 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Thu, 02 Nov 2000 15:37:43 GMT
Subject: [Python-Dev] PEP-0217
In-Reply-To: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
Message-ID: <3a0186d5.25979536@smtp.worldonline.dk>

[Moshe Zadka]

>I need some help:
>
>1) BDFL pronouncement
>2) someone to see about the Jython issue.

I don't see any problems with this. This is already handled by a method
in the jython runtime (Py.printResult).

However, I think your example implementation should be:

    def displayhook(o):
        if o is None:
            return
        __builtin__._ = None
        print `o`
        __builtin__._ = o

I don't why, but that is the implementation currently used by Jython
(and I think CPython too).

regards,
finn



From moshez at math.huji.ac.il  Thu Nov  2 17:05:20 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:05:20 +0200 (IST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <Pine.GSO.4.10.10011021804210.1290-100000@sundial>

On Thu, 2 Nov 2000, Finn Bock wrote:

> However, I think your example implementation should be:
> 
>     def displayhook(o):
>         if o is None:
>             return
>         __builtin__._ = None
>         print `o`
>         __builtin__._ = o
> 

You're right.
I'll just add the necessary Jython changes to PEP-0217.
Thanks a lot.

I don't like this either, but the good news is that in Py2.1, you'll
be able to change this in site.py <wink>

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Thu Nov  2 17:05:14 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:05:14 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <3A0190BA.940FFADA@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 1 Nov 2000, Guido van Rossum wrote:
> 
> > [MAL]
> > > Dynamic nested scopes is another topic... those are *very*
> > > useful; especially when it comes to avoiding global variables
> > > and implementing programs which work using control objects
> > > instead of global function calls.
> >
> > Marc-Andre, what are Dynamic nested scopes?
> 
> If MAL means dynamic scoping (which I understood he does), then this
> simply means:
> 
> when looking for a variable "foo", you first search for it in the local
> namespace. If not there, the *caller's* namespace, and so on. In the
> end, the caller is the __main__ module, and if not found there, it is
> a NameError.

That would be one application, yes.

With dynamic scoping I meant that the context of a lookup is
defined at run-time and by explicitely or implicitely
hooking together objects which then define the nesting.

Environment acquisition is an example of such dynamic scoping:
attribute lookups are passed on to the outer scope in case they
don't resolve on the inner scope, e.g. say you have
object a with a.x = 1; all other objects don't define .x.
Then a.b.c.d.x will result in lookups
1. a.b.c.d.x
2. a.b.c.x
3. a.b.x
4. a.x -> 1

This example uses attribute lookup -- the same can be done for
other nested objects by explicitely specifying the nesting
relationship.

Jim's ExtensionClasses allow the above by using a lot of
wrappers around objects -- would be nice if we could come
up with a more general scheme which then also works for
explicit nesting relationships (e.g. dictionaries which
get hooked together -- Jim's MultiMapping does this).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Thu Nov  2 17:06:38 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:06:38 +0100
Subject: [Python-Dev] Python 2.1 tasks
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>
Message-ID: <3A01910E.A19FFBB2@lemburg.com>

Andrew Kuchling wrote:
> 
> On Thu, Nov 02, 2000 at 05:29:55PM +0200, Moshe Zadka wrote:
> >Shouldn't we have a list of goals for [Python 2.1] or something? As a
> >first-order approximation, what PEPs are expected to be included? And,
> 
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.
> 
>   * Write a PEP on using Distutils to build the modules that come with
>     Python, and implement it if accepted.
> 
>   * Work on something CPAN-like.  This may or may not have repercussions for
>     the core; I don't know.

Most important for 2.1 are probably:

1. new C level coercion scheme
2. rich comparisons
3. making the std lib Unicode compatible

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jeremy at alum.mit.edu  Thu Nov  2 17:11:30 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:11:30 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
Message-ID: <14849.37426.860007.989619@bitdiddle.concentric.net>

>>>>> "TP" == Tim Peters <tim_one at email.msn.com> writes:

  TP> [Jeremy Hylton]
  >> ...  Guido once explained that his original reservation about
  >> nested scopes was a reaction to their overuse in Pascal.  In
  >> large Pascal programs he was familiar with, block structure was
  >> overused as an organizing principle for the program, leading to
  >> hard-to-read code.

  TP> Note that this problem will be much worse in Python: in Pascal,
  TP> you could always "look up" for the closest-containing func/proc
  TP> that explicitly declares a referenced vrbl.  In Python, you have
  TP> to indirectly *deduce* which vrbls are local to a def, by
  TP> searching the entire body for an appearance as a binding target.
  TP> So you have to "look up" and "look down" from the reference
  TP> point, and it's easy to miss a binding target.

I agree that visual inspection is a tad harder, but I contend that
existing programs that use the same name for a global variable and a
local variable -- and intend for the global to be visible within a
function nested in the local variable's region -- are confusing.  
It's too hard for a first-time reader of the code to figure out what
is going on.

Incidentally, I have yet to see an example of this problem occurring
in anyone's code.  All the examples seem a bit contrived.  I wonder if
anyone has an example in existing code.

  [My SICP example omitted]

  TP> Unfortunately for proponents, this is exactly the kind of SICP
  TP> example that is much better done via a class.

Indeed, the PEP says exactly that: This kind of program is better
done via a class.  My intent was not to show a compelling use of
mutable state.  Instead it was to show that with read-only access,
people could still modify values on enclosing scopes.  The issue is
whether the language allows the programmer to express this intent
clearly or if she has to jump through some hoops to accomplish it.

  TP>                                                Not only is the
  TP> closure version strained by comparison, but as is usual it
  TP> manages to create a bank account with a write-only balance <0.9
  TP> wink>.

  TP>     def deposit(amount):
  TP>         global bank_account.balance balance += amount

  TP> is one old suggested way to explicitly declare non-local names
  TP> and the enclosing block to which they are local (and in analogy
  TP> with current "global", mandatory if you want to rebind the
  TP> non-local name, optional if you only want to reference it).
  TP> There are subtleties here, but explicit is better than implicit,
  TP> and the subtleties are only subtler if you refuse (like Scheme)
  TP> to make the intent explicit.

I'm still not sure I like it, because it mixes local variables of a
function with attribute access on objects.  I'll add it to the
discussion in the PEP (if Barry approves the PEP <wink>), though.

Do you have any opinion on the subtleties?  The two that immediately
come to mind are: 1) whether the function's local are available as
attributes anywhere or only in nested scopes and 2) whether you can
create new local variable using this notation.

Jeremy



From moshez at math.huji.ac.il  Thu Nov  2 17:09:08 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 2 Nov 2000 18:09:08 +0200 (IST)
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <20001102164239.R12812@xs4all.nl>
Message-ID: <Pine.GSO.4.10.10011021807060.1290-100000@sundial>

On Thu, 2 Nov 2000, Thomas Wouters wrote:

> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

Cries for this sig have been already made in c.l.py.
I'm moving this discussion to meta-sig. Please discuss it there.
I'm willing to champion it, but I'll defer if Andrew or Greg want
to do it.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez





From jeremy at alum.mit.edu  Thu Nov  2 17:13:52 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:13:52 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A015A3A.4EC7DBC6@lemburg.com>
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
	<3A015A3A.4EC7DBC6@lemburg.com>
Message-ID: <14849.37568.510427.834971@bitdiddle.concentric.net>

>>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:

  MAL> It may not look serious, but changing the Python lookup scheme
  MAL> is, since many inspection tools rely and reimplement exactly
  MAL> that scheme. With nested scopes, there would be next to no way
  MAL> to emulate the lookups using these tools.

Can you say more about this issue?  It sounds like it is worth
discussing in the PEP, but I can't get a handle on exactly what the
problem is.  Any tool needs to implement or model Python's name
resolution algorithm, call it algorithm A.  If we change name
resolution to use algorithm B, then the tools need to implement or
model a new algorithm.  I don't see where the impossibility of
emulation comes in.

Jeremy




From guido at python.org  Thu Nov  2 05:24:29 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:24:29 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "01 Nov 2000 18:13:07 GMT."
             <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net>  
            <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> 
Message-ID: <200011020424.XAA07810@cj20424-a.reston1.va.home.com>

> How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> and NCALL_FUNC_KW.
> 
> NCALL_FUNC would pop a function object and a tuple off the stack and
> apply the function to the tuple.
> 
> NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> the moral equivalent of f(*args,**kw).

No, this is a bad idea.  Long, long ago, all calls requird building a
tuple for the arguments first.  This tuple creation turned out to be a
major bottleneck.  That's why the current call opcode exists.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From barry at wooz.org  Thu Nov  2 17:22:32 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:22:32 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38088.903104.936944@anthem.concentric.net>

If we get lexical scoping, there should be a fast (built-in) way to
get at all the accessible names from Python.  I.e. currently I can do

    d = globals().copy()
    d.update(locals())

and know that `d' contains a dictionary of available names, with the
right overloading semantics.  (PEP 42 now includes a feature request
to make vars() do this by default.)

-Barry



From barry at wooz.org  Thu Nov  2 17:23:56 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:23:56 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
Message-ID: <14849.38172.908316.107381@anthem.concentric.net>

This has been added as PEP 227.



From guido at python.org  Thu Nov  2 05:28:00 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 01 Nov 2000 23:28:00 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "Wed, 01 Nov 2000 14:06:46 EST."
             <14848.27078.923932.758419@bitdiddle.concentric.net> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk>  
            <14848.27078.923932.758419@bitdiddle.concentric.net> 
Message-ID: <200011020428.XAA07841@cj20424-a.reston1.va.home.com>

> It certainly wouldn't hurt to implement this, as it would provide some
> practical implementation experience that would inform a PEP on the
> subject.

If it solves the mess with supporting extended call syntax, adding
these opcodes might be a good idea.

But as I said, for the normal (not extended) case, the existing
CALL_FUNCTION opcode is the right thing to use unless you want things
to slow down significantly.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Thu Nov  2 17:31:35 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:31:35 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011021724340.1290-100000@sundial>
Message-ID: <14849.38631.997377.600214@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

    MZ> If MAL means dynamic scoping (which I understood he does),
    MZ> then this simply means:

    MZ> when looking for a variable "foo", you first search for it in
    MZ> the local namespace. If not there, the *caller's* namespace,
    MZ> and so on. In the end, the caller is the __main__ module, and
    MZ> if not found there, it is a NameError.

This is how Emacs Lisp behaves, and it's used all the time in ELisp
programs.  On the one hand it's quite convenient for customizing the
behavior of functions.  On the other hand, it can make documenting the
interface of functions quite difficult because all those dynamically
scoped variables are now part of the function's API.

It's interesting to note that many ELispers really hate dynamic
scoping and pine for a move toward lexical scoping.  I'm not one of
them.

I'm not as concerned about "fixing" nested functions because I hardly
ever use them, and rarely see them much in Python code.  Fixing
lambdas would be nice, but since Guido considers lambdas themselves a
mistake, and given that lamda use /can/ be a performance hit in some
situations, does it make sense to change something as fundamental as
Python's scoping rules to fix this eddy of the language?

-Barry



From jeremy at alum.mit.edu  Thu Nov  2 17:36:45 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:36:45 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <3A0190BA.940FFADA@lemburg.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
	<3A0190BA.940FFADA@lemburg.com>
Message-ID: <14849.38941.59576.682495@bitdiddle.concentric.net>

Moshe's explanation of "dynamic scope" is the definition I've seen in
every programming language text I've ever read.  The essence of the
defintion, I believe, is that a free variable is resolved in the
environment created by the current procedure call stack.

I think it muddles the discussion to use "dynamic scope" to describe
acquistion, though it is a dynamic feature.

Python using dynamic scope for exceptions.  If any exception is
raised, the exception handler that is triggered is determined by the
environment in which the procedure was called.

There are few languages that use dynamic scoping for normal name
resolution.  Many early Lisp implementations did, but I think all the
modern ones use lexical scoping instead.  It is hard to write modular
code using dynamic scope, because the behavior of a function with free
variables can not be determined by the module that defines it.  Not
saying it isn't useful, just that it makes it much harder to reason
about how a particular modular or function works in isolation from the
rest of the system.

Jeremy



From jeremy at alum.mit.edu  Thu Nov  2 17:38:04 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 11:38:04 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <Pine.GSO.4.10.10011021715210.1290-100000@sundial>
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.39020.532855.494649@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

  MZ> you guys are talking about different things.  Jeremy is talking
  MZ> about a tool to warn against incompatible changes Greg is
  MZ> talking about a tool to identify, for each variable, what scope
  MZ> it belongs to.

Not sure we're talking about different things.  The compiler will need
to determine the scope of each variable.  It's a tool.  If it
implements the specifiction for name binding, other tools can too.

Jeremy



From barry at wooz.org  Thu Nov  2 17:36:11 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:36:11 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.34263.310260.404940@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011021715210.1290-100000@sundial>
Message-ID: <14849.38907.808752.186414@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at math.huji.ac.il> writes:

    MZ> you guys are talking about different things.  Jeremy is
    MZ> talking about a tool to warn against incompatible changes Greg
    MZ> is talking about a tool to identify, for each variable, what
    MZ> scope it belongs to.

And Greg's point is well taken, because it /will/ be harder to tell at
a glance where a name is coming from, so programming tools will have
to find ways to help with this.

-Barry



From barry at wooz.org  Thu Nov  2 17:44:06 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:44:06 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
	<20001101234134.O12812@xs4all.nl>
Message-ID: <14849.39382.960359.909365@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:

    >> Update this PEP to current, harsh, reality. It's been rejected
    >> :) If at all possible, the reasoning should be extended to
    >> include the real reasons it was rejected -- this is just
    >> guesswork from my side. (This means you, Guido, or anyone who
    >> can channel Guido enough to call himself Guido.)

    TW> In addition to that, PEP 0 also needs to be updated. Shall I
    TW> do that myself, now that Barry is apparently away ?

I've just done it.
    
    TW> While I was at it, I also noticed PEP 0200 still says
    TW> 'Incomplete', though that might be by design.

I've updated both these too, thanks.

-Barry



From mal at lemburg.com  Thu Nov  2 17:45:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 17:45:12 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>
		<3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>
Message-ID: <3A019A18.20D12FE1@lemburg.com>

Jeremy Hylton wrote:
> 
> >>>>> "MAL" == M -A Lemburg <mal at lemburg.com> writes:
> 
>   MAL> It may not look serious, but changing the Python lookup scheme
>   MAL> is, since many inspection tools rely and reimplement exactly
>   MAL> that scheme. With nested scopes, there would be next to no way
>   MAL> to emulate the lookups using these tools.
> 
> Can you say more about this issue?  It sounds like it is worth
> discussing in the PEP, but I can't get a handle on exactly what the
> problem is.  Any tool needs to implement or model Python's name
> resolution algorithm, call it algorithm A.  If we change name
> resolution to use algorithm B, then the tools need to implement or
> model a new algorithm.  I don't see where the impossibility of
> emulation comes in.

Well first you'd have to change all tools to use the new
scheme (this includes debuggers, inspection tools, reflection
kits, etc.). This certainly is not a smart thing to do since
Python IDEs are just starting to appear -- you wouldn't want
to break all those.

What get's harder with the nested scheme is that
you can no longer be certain that globals() reaches out to
the module namespace. But this is needed by some lazy evaluation
tools. Writing to globals() would not be defined anymore --
where should you bind the new variable ?

Another problem is that there probably won't be a way to access
all the different nesting levels on a per-level basis (could be
that I'm missing something here, but debugging tools would need
some sort of scope() builtin to access the different scopes).
I'm not sure whether this is possible to do without some sort
of link between the scopes. We currently don't need such links.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Thu Nov  2 17:52:50 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 11:52:50 -0500 (EST)
Subject: [Python-Dev] PEP-0217
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
	<3a0186d5.25979536@smtp.worldonline.dk>
Message-ID: <14849.39906.374949.489679@anthem.concentric.net>

>>>>> "FB" == Finn Bock <bckfnn at worldonline.dk> writes:

    FB> I don't see any problems with this. This is already handled by
    FB> a method in the jython runtime (Py.printResult).

Yep, should be easy to add to Jython.
-Barry



From mwh21 at cam.ac.uk  Thu Nov  2 18:00:37 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:00:37 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Wed, 01 Nov 2000 23:24:29 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com>
Message-ID: <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > How about dumping the CALL_FUNCTION* opcodes, and replacing them with
> > two non-argumented opcodes, called for the sake of argument NCALL_FUNC
> > and NCALL_FUNC_KW.
> > 
> > NCALL_FUNC would pop a function object and a tuple off the stack and
> > apply the function to the tuple.
> > 
> > NCALL_FUNC_KW would do the same, then pop a dictionary and then do
> > the moral equivalent of f(*args,**kw).
> 
> No, this is a bad idea.  Long, long ago, all calls requird building a
> tuple for the arguments first.  This tuple creation turned out to be a
> major bottleneck.  That's why the current call opcode exists.

Yes, I realize this now.  I made my suggestion having not actually
looked at the code or thought about it very much.

I still think there is (or at least might be) value in rewriting the
code for the more complex cases, and moving the dictionary creation
out of the implementation of CALL_FUNCTION.

Then the following could be made essentially equivalent:

f(a,b,c=d,e=f)

dict = {c:d,e:f}
f(a,b,**dict)

(modulo evaluation order).

(I've made some changes to get BUILD_MAP using it's argument and
construct literal dictionaries using it, which I'll whack up onto sf
just as soon as I get round to it... ah, it's at

https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102227&group_id=5470

).

Cheers,
M.

-- 
  Those who have deviant punctuation desires should take care of their
  own perverted needs.                  -- Erik Naggum, comp.lang.lisp




From gstein at lyra.org  Thu Nov  2 18:07:29 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 09:07:29 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: <14849.39382.960359.909365@anthem.concentric.net>; from barry@wooz.org on Thu, Nov 02, 2000 at 11:44:06AM -0500
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>
Message-ID: <20001102090729.A1874@lyra.org>

On Thu, Nov 02, 2000 at 11:44:06AM -0500, Barry A. Warsaw wrote:
> 
> >>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:
> 
>     >> Update this PEP to current, harsh, reality. It's been rejected
>     >> :) If at all possible, the reasoning should be extended to
>     >> include the real reasons it was rejected -- this is just
>     >> guesswork from my side. (This means you, Guido, or anyone who
>     >> can channel Guido enough to call himself Guido.)
> 
>     TW> In addition to that, PEP 0 also needs to be updated. Shall I
>     TW> do that myself, now that Barry is apparently away ?
> 
> I've just done it.

Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
Barry's administrative overload.

I mean, geez... this is what source control is about. Let a lot of people in
there, but be able to back up in case somebody totally goofs it.

This goes for adding new PEPs, too. I'm not as convinced here, since some
level of "good enough for a PEP" filtering is probably desirable, but then
again, it would seem that the people with commit access probably have that
filter in their head anyways.

Just a thought... how can we grease things up a bit more...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From pf at artcom-gmbh.de  Thu Nov  2 17:59:21 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 2 Nov 2000 17:59:21 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011021618.IAA15298@slayer.i.sourceforge.net> from Barry Warsaw at "Nov 2, 2000  8:18:27 am"
Message-ID: <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>

Barry Warsaw wrote:
> Update of /cvsroot/python/python/nondist/peps
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv15290
> 
> Added Files:
> 	pep-0227.txt 
> Log Message:
> PEP 227, Statically Nested Scopes, Jeremy Hylton
> 
> 
> ***** Error reading new file: (2, 'No such file or directory')

It was obviously not intended to be mailed out that way again.  
Problem with pathname and/or current directory?  Barry got this 
right once, now it is broken again.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Thu Nov  2 18:07:15 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Nov 2000 18:07:15 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz>  
	            <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <3A019F43.E13604BA@lemburg.com>

Guido van Rossum wrote:
> 
> > It may not look serious, but changing the Python lookup scheme
> > is, since many inspection tools rely and reimplement exactly
> > that scheme. With nested scopes, there would be next to no
> > way to emulate the lookups using these tools.
> 
> So fix the tools.

Eek. Are you proposing to break all the Python IDE that are
just appearing out there ?

> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?

I think the common practice mainly comes from the fact,
that by making globals locals which can benefit from LOAD_FAST
you get a noticable performance boost.

So the "right" solution to these weird looking hacks would
be to come up with a smart way by which the Python compiler
itself can do the localizing.

Nested scopes won't help eliminating the current keyword 
practice.
 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

Hmm, so far the only argument for changing Python lookups
was to allow writing lambdas without keyword hacks. Does this
really warrant breaking code ?

What other advantages would statically nested scopes have ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Thu Nov  2 06:16:43 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:16:43 -0500
Subject: [Python-Dev] PEP-0217
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:49:37 +0200."
             <Pine.GSO.4.10.10011021648390.1290-100000@sundial> 
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial> 
Message-ID: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>

> 1) BDFL pronouncement

I believe Ping has also proposed such a display hook.  I'm not against
the idea, but I'm also not much in favor -- so I'm kind of +/- 0...

I've always thought that the interpreter mainloop should be rewritten
in Python.  That would be another PEP, and this would be a good place
to add a display hook as a feature.

Note that the example curreltly in the PEP has a fatal flaw: it's got
a recursive reference to print.  This is one of the things to consider
when proposing such a feature.

Sorry I can't be of more help...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 18:20:12 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:20:12 -0500 (EST)
Subject: [Python-Dev] PEP-0217
In-Reply-To: <200011020516.AAA08400@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021648390.1290-100000@sundial>
	<200011020516.AAA08400@cj20424-a.reston1.va.home.com>
Message-ID: <14849.41548.308591.119778@bitdiddle.concentric.net>

I think the current draft of PEP 217 is far too thin and vague to be
acceptable.  You ought to re-read the PEP guidlines and make sure
you've covered all the points.  You must address at least:

- motivation
- specification of feature independent of implementation

Jeremy



From claird at starbase.neosoft.com  Thu Nov  2 18:18:20 2000
From: claird at starbase.neosoft.com (Cameron Laird)
Date: Thu, 2 Nov 2000 11:18:20 -0600 (CST)
Subject: [Python-Dev] Tk news you'll want to read
Message-ID: <200011021718.LAA70431@starbase.neosoft.com>

<URL:http://deja.com/=dnc/getdoc.xp?AN=688816476>

Note the clear intention to co-operate with Perl and Python,
the ambition to be considerably more portable and superior in
performance to GTK+, and so on.

I encourage you to e-mail this on to others it might interest.
A lot of the Perl people don't receive anything from me because
my ISP is ORBS-blacklisted (don't ask).



From guido at python.org  Thu Nov  2 06:29:27 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:29:27 -0500
Subject: [Python-Dev] CPyAN
In-Reply-To: Your message of "Thu, 02 Nov 2000 16:42:39 +0100."
             <20001102164239.R12812@xs4all.nl> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us>  
            <20001102164239.R12812@xs4all.nl> 
Message-ID: <200011020529.AAA08549@cj20424-a.reston1.va.home.com>

[Andrew]
> > * Work on something CPAN-like.  This may or may not have repercussions for
> >   the core; I don't know.

[Thomas]
> Probably not, though perhaps a new module would be nice. As for the
> CPAN-like thing, I really got a kick out of Greg S's WebDAV session on
> Apachecon, and I think it would be suited extremely well as the transmission
> protocol for SPAM (or however you want to call the Python CPAN ;). You can
> do the uploading, downloading and searching for modules using WebDAV without
> too much pain, and there's excellent WebDAV support for Apache ;)
> 
> Is anyone working on something like this, or even thinking about it ? I'm
> not deep enough into distutils to join that SIG, but I definately would join
> a CPyAN SIG ;)

This is a nice thing to have, but I don't see why it should be tied to
the 2.1 effort.  Let's not couple projects that can be carried out
independently!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mwh21 at cam.ac.uk  Thu Nov  2 18:29:52 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 17:29:52 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Michael Hudson's message of "02 Nov 2000 17:00:37 +0000"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21 at cam.ac.uk> writes:

> Guido van Rossum <guido at python.org> writes:
> 
> Then the following could be made essentially equivalent:
> 
> f(a,b,c=d,e=f)
> 
> dict = {c:d,e:f}
> f(a,b,**dict)

... except when you have stuff like

f(a,b=c,**kw)

Gruntle.  Maybe Python function calls are just complicated!  (Has
anyone looked at bytecodehacks.xapply? - and that doesn't even handle
*-ed and **-ed arguments...).

Hmm - the interaction of "a=b" style args & *-ed args is a bit
counter-intuitive, particularly as the "a=b" args syntatically have to
come before the *-ed args:

>>> def f(a,b,c,d):
...  return a,b,c,d
... 
>>> f(1,c=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'c' redefined in call to f()
>>> f(1,b=3,*(2,4))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: keyword parameter 'b' redefined in call to f()
>>> f(1,d=4,*(2,3))
(1, 2, 3, 4)

I humbly submit that This Is Wrong.  I haven't seen anybody complain
about it, which suggests to me that noone is using this combination,
and I propose either:

1) banning it
2) demanding that the *-ed arg precede the "a=b" args

Of course, if noone is using this "feature", then maybe this dusty
little corner of the language should be left undisturbed.

And I haven't even thought about default arguments yet...

about-to-make-an-opcode-called-BODGE_KEYWORD_ARGUMENTS-ly y'rs
M.

-- 
  nonono,  while we're making wild  conjectures about the behavior 
  of completely  irrelevant tasks,  we must not  also make serious 
  mistakes, or the data might suddenly become statistically valid.
                                        -- Erik Naggum, comp.lang.lisp




From barry at wooz.org  Thu Nov  2 18:32:13 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:32:13 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
References: <200011012237.OAA06642@slayer.i.sourceforge.net>
	<20001101234134.O12812@xs4all.nl>
	<14849.39382.960359.909365@anthem.concentric.net>
	<20001102090729.A1874@lyra.org>
Message-ID: <14849.42269.297360.615404@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

    GS> Shouldn't we allow other people to tweak PEP 0? It would
    GS> certainly lighten Barry's administrative overload.

I certainly don't mind at the very least, people modifying PEP 0 when
the status of their own peps change.

    GS> I mean, geez... this is what source control is about. Let a
    GS> lot of people in there, but be able to back up in case
    GS> somebody totally goofs it.

    GS> This goes for adding new PEPs, too. I'm not as convinced here,
    GS> since some level of "good enough for a PEP" filtering is
    GS> probably desirable, but then again, it would seem that the
    GS> people with commit access probably have that filter in their
    GS> head anyways.

    GS> Just a thought... how can we grease things up a bit more...

I do like to make a sanity pass through the text before approving it,
just to make sure we've got consistent format throughout the peps.
Also, I know we're all on "internet time" here <wink>, but one day
isn't too much time to let pass before taking action on things. :)
I'd also prefer it if there's /some/ limited editorial review before
these things get added.

That having been said, I'm very happy if someone wants to co-edit the
peps.  The pep 0 re-organization and slacker scolding would definitely
benefit from more than one watchdog.

Volunteers? :)

-Barry



From barry at wooz.org  Thu Nov  2 18:33:11 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 12:33:11 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
References: <200011021618.IAA15298@slayer.i.sourceforge.net>
	<m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>
Message-ID: <14849.42327.23020.553510@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf at artcom-gmbh.de> writes:

    PF> It was obviously not intended to be mailed out that way again.
    PF> Problem with pathname and/or current directory?  Barry got
    PF> this right once, now it is broken again.

Except that AFAIK, I didn't do anything to fix it, or to break it
again.

-Barry



From guido at python.org  Thu Nov  2 06:45:09 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:45:09 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:05:14 +0100."
             <3A0190BA.940FFADA@lemburg.com> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>  
            <3A0190BA.940FFADA@lemburg.com> 
Message-ID: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>

> > > [MAL]
> > > > Dynamic nested scopes is another topic... those are *very*
> > > > useful; especially when it comes to avoiding global variables
> > > > and implementing programs which work using control objects
> > > > instead of global function calls.
> > >
> > > Marc-Andre, what are Dynamic nested scopes?

[Moshe]
> > If MAL means dynamic scoping (which I understood he does), then this
> > simply means:
> > 
> > when looking for a variable "foo", you first search for it in the local
> > namespace. If not there, the *caller's* namespace, and so on. In the
> > end, the caller is the __main__ module, and if not found there, it is
> > a NameError.

Ah, yuck.  For variable namespace, this is a really bad idea.  For
certain other things (e.g. try/except blocks, Jeremy's example) it is
of course OK.

[MAL]
> That would be one application, yes.
> 
> With dynamic scoping I meant that the context of a lookup is
> defined at run-time and by explicitely or implicitely
> hooking together objects which then define the nesting.

But certainly you're not thinking of doing this to something as basic
to Python's semantics as local/global variable lookup!

> Environment acquisition is an example of such dynamic scoping:
> attribute lookups are passed on to the outer scope in case they
> don't resolve on the inner scope, e.g. say you have
> object a with a.x = 1; all other objects don't define .x.
> Then a.b.c.d.x will result in lookups
> 1. a.b.c.d.x
> 2. a.b.c.x
> 3. a.b.x
> 4. a.x -> 1

This seems only remotely related to dynamic scopes.  There are
namespaces here, but not "scopes" as I think of them: a scope defines
the validity for an unadorned variable.  Static scopes mean that this
is determined by the program source structure.  Dynamic scopes means
that that this is determined by run-time mechanisms.  Python uses
static scoping for local variables, but a dynamic scoping mechanism
for non-local variable references: first it does a module-global
lookup, then it looks for a built-in name.

When we're not talking about simple name lookup, we should speak of
namespaces (which can also have lifetimes).

> This example uses attribute lookup -- the same can be done for
> other nested objects by explicitely specifying the nesting
> relationship.
> 
> Jim's ExtensionClasses allow the above by using a lot of
> wrappers around objects -- would be nice if we could come
> up with a more general scheme which then also works for
> explicit nesting relationships (e.g. dictionaries which
> get hooked together -- Jim's MultiMapping does this).

I think we're getting way off track here.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 18:43:03 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 12:43:03 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDIHOAA.tim_one@email.msn.com>

[Jeremy Hylton]
> ...
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.

I believe all early Lisps were dynamically scoped.  Scheme changed that, and
Common Lisp followed.  Many interpreted languages *start* life with dynamic
scoping because it's easy to hack together, but that never lasts.  REBOL
went thru this a couple years ago, switching entirely from dynamic to
lexical before its first public release, and breaking most existing programs
in the process.  Perl also started with dynamic scoping, but, in Perl-like
fashion, Perl5 *added* lexical scoping on top of dynamic ("local" vars use
dynamic scoping; "my" vars lexical; and all Perl5 gotcha guides stridently
recommend never using "local" anymore).  Here's some Perl5:

$i = 1;

sub f {
    local($i) = 2;
    &g();
}

sub g {
    return $i;
}

print "i at start is $i\n";
print "i in g called directly is ", &g(), "\n";
print "i in g called indirectly via f is ", &f(), "\n";
print "i at end is $i\n";

Here's what it prints:

i at start is 1
i in g called directly is 1
i in g called indirectly via f is 2
i at end is 1

> It is hard to write modular code using dynamic scope, because the
> behavior of a function with free variables can not be determined by
> the module that defines it.

As shown above; dynamic scoping is a nightmare even in the absence of nested
functions.

> Not saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

People who spend too much time writing meta-systems <wink> overestimate its
usefulness.  Most programmers who need this kind of effect would be much
better off explicitly passing a dict of name->value mappings explicitly
manipulated, or simply passing the values of interest (if I want a function
that sucks the value of "i" out of its caller, what clearer way than for the
caller to pass i in the arglist?!  dynamic scoping is much subtler, of
course -- it sucks the value of "i" out of *whatever* function up the call
chain happened to define an i "most recently").

Lexical scoping is much tamer, and, indeed, Python is one of the few modern
languages that doesn't support it.  Last time Guido and I hand-wrung over
this, that was the chief reason to implement it:  newcomers are endlessly
surprised that when they write functions that visually nest, their scopes
nevertheless don't nest.  There's certainly nothing novel or unexpected
about lexical scoping anymore.

The problem unique to Python is its rule for determining which vrbls are
local; in Lisps, REBOL and Perl, you have to explicitly name all vrbls local
to a given scope, which renders "how do you rebind a non-local name?" a
non-issue.

has-the-feel-of-inevitability-ly y'rs  - tim





From guido at python.org  Thu Nov  2 06:56:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 00:56:03 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:11:30 EST."
             <14849.37426.860007.989619@bitdiddle.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>  
            <14849.37426.860007.989619@bitdiddle.concentric.net> 
Message-ID: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>

[Jeremy and Tim argue about what to do about write access for
variables at intermediate levels of nesting, neither local nor
module-global.]

I'll risk doing a pronouncement, even though I know that Jeremy (and
maybe also Tim?) disagree.

You don't need "write access" (in the sense of being able to assign)
for variables at the intermediate scopes, so there is no need for a
syntax to express this.  Assignments are to local variables (normally)
or to module-globals (when 'global' is used).  Use references search
for a local, then for a local of the containing function definition,
then for a local in its container, and so forth, until it hits the
module globals, and then finally it looks for a builtin.

We can argue over which part of this is done statically and which part
is done dynamically: currently, locals are done dynamically and
everything else is done statically.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 18:57:09 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 12:57:09 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020545.AAA08743@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial>
	<3A0190BA.940FFADA@lemburg.com>
	<200011020545.AAA08743@cj20424-a.reston1.va.home.com>
Message-ID: <14849.43765.419161.105395@bitdiddle.concentric.net>

I don't think I buy your explanation that Python uses dynamic scope
for resolving globals.  As I understand the mechanism, the module
namespace and builtins namespace are those for the module in which the
function was defined.  If so, this is still static scope.

Here's a quick example that illustrates the difference:

module foo:
-----------------------
a = 12

def add(b):
    return a + b
-----------------------

module bar:
-----------------------
from foo import add

a = -1
print add(1)
-----------------------

If Python used static scope, "python bar.py" should print 13 (it
does).  If it used dynamic scope, I would expect the answer to be 0.

Jeremy



From fdrake at acm.org  Thu Nov  2 18:55:11 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 12:55:11 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14848.39586.832800.139182@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
Message-ID: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I just did a clean configure and make from the latest CVS tree.  It
 > seems to get stuck in a loop calling makesetup over and over again.

  Please do an update and run autoconf, then try again.
  I didn't check in the autoconf output; Guido thinks that we want to
stick with autoconf 2.13, and I've got 2.14.1.  The reason is that
there are enough differences in the m4 expansions that seeing the real
effects of the configure.in changes is hard, just because there are so
many changed lines which are not related to the actual change.
  (If you do have autoconf 2.13, please check in the new configure.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov  2 07:02:10 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:02:10 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:22:32 EST."
             <14849.38088.903104.936944@anthem.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net>  
            <14849.38088.903104.936944@anthem.concentric.net> 
Message-ID: <200011020602.BAA08974@cj20424-a.reston1.va.home.com>

> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
> 
>     d = globals().copy()
>     d.update(locals())
> 
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.  (PEP 42 now includes a feature request
> to make vars() do this by default.)

Note that I just deleted that feature request from PEP 42 -- vars() or
locals() returns the dictionary containing the variables, and you
can't just change the semantics to return a newly copied dictionary
(which could be quite expensive too!).

I don't think you need to have a mechanism to find all accessible
names; I don't see a common use for that.  It's sufficient to have a
mechanism to look up any specific name according to whatever mechanism
we decide upon.  This is needed for internal use of course; it can
also be useful for e.g. variable substitution mechanisms like the one
you recently proposed or Ping's Itmpl.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From jeremy at alum.mit.edu  Thu Nov  2 19:04:10 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:04:10 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44186.494602.297323@bitdiddle.concentric.net>

I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
placed three files in Modules: Makefile.pre, Setup, and Setup.config.

I ran "make" and it immediately reported an error:
"../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
Setup.local: No such file or directory"

Jeremy



From guido at python.org  Thu Nov  2 07:07:04 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:07:04 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:31:35 EST."
             <14849.38631.997377.600214@anthem.concentric.net> 
References: <200011020326.WAA07307@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021724340.1290-100000@sundial>  
            <14849.38631.997377.600214@anthem.concentric.net> 
Message-ID: <200011020607.BAA09013@cj20424-a.reston1.va.home.com>

[Barry, on dynamic scopes]
> This is how Emacs Lisp behaves, and it's used all the time in ELisp
> programs.  On the one hand it's quite convenient for customizing the
> behavior of functions.  On the other hand, it can make documenting the
> interface of functions quite difficult because all those dynamically
> scoped variables are now part of the function's API.
> 
> It's interesting to note that many ELispers really hate dynamic
> scoping and pine for a move toward lexical scoping.  I'm not one of
> them.

I don't care what you pine for in ELisp, but for Python this would be
a bad idea.

> I'm not as concerned about "fixing" nested functions because I hardly
> ever use them, and rarely see them much in Python code.  Fixing
> lambdas would be nice, but since Guido considers lambdas themselves a
> mistake, and given that lamda use /can/ be a performance hit in some
> situations, does it make sense to change something as fundamental as
> Python's scoping rules to fix this eddy of the language?

I referred to this in our group meeting as "fixing lambda" because
that's where others seem to need it most often.  But it is a real
problem that exists for all nested functions.  So let me rephrase
that: "fixing nested function definitions" is useful, if it can be
done without leaking memory, without breaking too much existing code,
and without slowing down code that doesn't use the feature.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Thu Nov  2 19:06:12 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:06:12 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com>
	<14849.37426.860007.989619@bitdiddle.concentric.net>
	<200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <14849.44308.333212.37111@bitdiddle.concentric.net>

Don't know if you saw the discussion in the PEP or not.  I made two
arguments for being able to assign to variables bound in enclosing
scopes.

1. Every other language that supports nested lexical scoping allows
   this.  To the extent that programmers have seen these other
   languages, they will expect it to work.

2. It is possible to work around this limitation by using containers.
   If you want to have an integer that can be updated by nested
   functions, you wrap the interger in a list and make all assignments
   and references refer to list[0].  It would be unfortunate if
   programmers used this style, because it is obscure.  I'd rather see
   the language provide a way to support this style of programming
   directly. 

Jeremy



From esr at thyrsus.com  Thu Nov  2 23:10:16 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:10:16 -0800
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>; from guido@python.org on Wed, Nov 01, 2000 at 10:29:03PM -0500
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
Message-ID: <20001102141016.B1838@thyrsus.com>

Guido van Rossum <guido at python.org>:
> > To be honest, I don't think static nested scopes buy us all that
> > much. You can do the same now, by using keyword arguments which
> > isn't all that nice, but works great and makes the scope clearly
> > visible.
> 
> Yes.  It's a hack that gets employed over and over.  And it has
> certain problems.  We added 'import as' to get rid of a common
> practice that was perceived unclean.  Maybe we should support nested
> scopes to get rid of another unclean common practice?
> 
> I'm not saying that we definitely should add this to 2.1 (there's
> enough on our plate already) but we should at least consider it, and
> now that we have cycle GC, the major argument against it (that it
> causes cycles) is gone...

For whatever it's worth, I agree with both these arguments.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"The power to tax involves the power to destroy;...the power to
destroy may defeat and render useless the power to create...."
	-- Chief Justice John Marshall, 1819.



From fdrake at acm.org  Thu Nov  2 19:04:34 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:04:34 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44186.494602.297323@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
Message-ID: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I created a new directory.  I executed 'OPT="-O3" ../configure'.  It
 > placed three files in Modules: Makefile.pre, Setup, and Setup.config.
 > 
 > I ran "make" and it immediately reported an error:
 > "../../Modules/makesetup Setup.config Setup.local Setup" reports "cat:
 > Setup.local: No such file or directory"

  Very interesting!  The first thing I get is this:

(cd Modules; make -f Makefile.pre Makefile)
make[1]: Entering directory `/home/fdrake/projects/python/temp/Modules'
echo "# Edit this file for local setup changes" >Setup.local
rm -rf ../libpython2.0.a
/bin/sh ../../Modules/makesetup Setup.config Setup.local Setup
make[1]: Leaving directory `/home/fdrake/projects/python/temp/Modules'

  Can you capture stout & stderr from a clean configure & make and
mail it to me?
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov  2 07:10:57 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:10:57 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 11:36:45 EST."
             <14849.38941.59576.682495@bitdiddle.concentric.net> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com>  
            <14849.38941.59576.682495@bitdiddle.concentric.net> 
Message-ID: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>

> Moshe's explanation of "dynamic scope" is the definition I've seen in
> every programming language text I've ever read.  The essence of the
> defintion, I believe, is that a free variable is resolved in the
> environment created by the current procedure call stack.

Ah.  The term "free variable" makes sense here.

> I think it muddles the discussion to use "dynamic scope" to describe
> acquistion, though it is a dynamic feature.
> 
> Python using dynamic scope for exceptions.  If any exception is
> raised, the exception handler that is triggered is determined by the
> environment in which the procedure was called.

Then I think this is also muddles the discussion, since the look for
exception handlers has nothing to do with free variable lookup.

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

I think Python 3000 ought to use totally static scoping.  That will
make it possible to do optimize code using built-in names!

--Guido van Rossum (home page: http://www.python.org/~guido/)




From esr at thyrsus.com  Thu Nov  2 23:12:57 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:12:57 -0800
Subject: [Python-Dev] Python 2.1 tasks
In-Reply-To: <3A01910E.A19FFBB2@lemburg.com>; from mal@lemburg.com on Thu, Nov 02, 2000 at 05:06:38PM +0100
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <3A01910E.A19FFBB2@lemburg.com>
Message-ID: <20001102141257.C1838@thyrsus.com>

M.-A. Lemburg <mal at lemburg.com>:
> Most important for 2.1 are probably:
> 
> 1. new C level coercion scheme
> 2. rich comparisons
> 3. making the std lib Unicode compatible

I'd certainly like to see rich comparisons go in.  I have a "Set" class 
all ready for addition to the standard library except that it's waiting
on this feature in order to do partial ordering properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Every election is a sort of advance auction sale of stolen goods. 
	-- H.L. Mencken 



From esr at thyrsus.com  Thu Nov  2 23:14:24 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Nov 2000 14:14:24 -0800
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 11:36:45AM -0500
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <20001102141424.D1838@thyrsus.com>

Jeremy Hylton <jeremy at alum.mit.edu>:
> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.

Correct.  Based on my LISP experience, I would be strongly opposed to
dymamic scoping.  That path has been tried and found wanting.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"Guard with jealous attention the public liberty.  Suspect every one
who approaches that jewel.  Unfortunately, nothing will preserve it
but downright force.  Whenever you give up that force, you are
inevitably ruined."
	-- Patrick Henry, speech of June 5 1788



From jeremy at alum.mit.edu  Thu Nov  2 19:14:08 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:14:08 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
Message-ID: <14849.44784.479281.162333@bitdiddle.concentric.net>

Despite the error, I did get a successful build this time.  I guess
your configure change worked.

Jeremy



From jeremy at alum.mit.edu  Thu Nov  2 19:16:25 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 13:16:25 -0500 (EST)
Subject: [Python-Dev] Larry Wall talk on Perl 6
Message-ID: <14849.44921.765397.205628@bitdiddle.concentric.net>

http://dev.perl.org/~ask/als/

Interesting reading.

Jeremy



From fdrake at acm.org  Thu Nov  2 19:12:38 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 13:12:38 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44784.479281.162333@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
	<14849.44784.479281.162333@bitdiddle.concentric.net>
Message-ID: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Despite the error, I did get a successful build this time.  I guess
 > your configure change worked.

  I don't think the bug I fixed and the bug you reported were tightly
related.  I don't understand why you got the error after before or
after my change.
  I'f you're running autoconf 2.13, though, please do check in the new
configure script!
  (If anyone else is, you can check it in too!  The winner gets one
free documentation download!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From barry at wooz.org  Thu Nov  2 19:20:06 2000
From: barry at wooz.org (barry at wooz.org)
Date: Thu, 2 Nov 2000 13:20:06 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14848.27102.223001.369662@bitdiddle.concentric.net>
	<14849.38088.903104.936944@anthem.concentric.net>
	<200011020602.BAA08974@cj20424-a.reston1.va.home.com>
Message-ID: <14849.45142.860806.358489@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    >> If we get lexical scoping, there should be a fast (built-in)
    >> way to get at all the accessible names from Python.
    >> I.e. currently I can do d = globals().copy() d.update(locals())
    >> and know that `d' contains a dictionary of available names,
    >> with the right overloading semantics.  (PEP 42 now includes a
    >> feature request to make vars() do this by default.)

    GvR> Note that I just deleted that feature request from PEP 42 --
    GvR> vars() or locals() returns the dictionary containing the
    GvR> variables, and you can't just change the semantics to return
    GvR> a newly copied dictionary (which could be quite expensive
    GvR> too!).

Saw that.  I was just thinking that locals() already does what
vars()-no-args does, so why have two ways to do the same thing?

    GvR> I don't think you need to have a mechanism to find all
    GvR> accessible names; I don't see a common use for that.  It's
    GvR> sufficient to have a mechanism to look up any specific name
    GvR> according to whatever mechanism we decide upon.  This is
    GvR> needed for internal use of course; it can also be useful for
    GvR> e.g. variable substitution mechanisms like the one you
    GvR> recently proposed or Ping's Itmpl.

Ah, something like this then:

-------------------- snip snip --------------------
import sys
from UserDict import UserDict

class NamesDict(UserDict):
    def __init__(self, frame):
        self.__frame = frame
        UserDict.__init__(self)

    def __getitem__(self, key):
        if self.data.has_key(key):
            return self.data[key]
        locals = self.__frame.f_locals
        if locals.has_key(key):
            return locals[key]
        globals = self.__frame.f_globals
        if globals.has_key(key):
            return globals[key]
        raise KeyError, key

def _(s):
    try: raise 'oops'
    except: frame = sys.exc_info()[2].tb_frame.f_back
    return s % NamesDict(frame)

theirs = 'theirs'
def give(mine, yours):
    print _('mine=%(mine)s, yours=%(yours)s, theirs=%(theirs)s')
-------------------- snip snip --------------------
Python 2.0 (#128, Oct 18 2000, 04:48:44) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import dict
>>> dict.give('mine', 'yours')
mine=mine, yours=yours, theirs=theirs
>>> 

-Barry



From guido at python.org  Thu Nov  2 07:28:26 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:28:26 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0204.txt,1.4,1.5
In-Reply-To: Your message of "Thu, 02 Nov 2000 09:07:29 PST."
             <20001102090729.A1874@lyra.org> 
References: <200011012237.OAA06642@slayer.i.sourceforge.net> <20001101234134.O12812@xs4all.nl> <14849.39382.960359.909365@anthem.concentric.net>  
            <20001102090729.A1874@lyra.org> 
Message-ID: <200011020628.BAA09282@cj20424-a.reston1.va.home.com>

> Shouldn't we allow other people to tweak PEP 0? It would certainly lighten
> Barry's administrative overload.
> 
> I mean, geez... this is what source control is about. Let a lot of people in
> there, but be able to back up in case somebody totally goofs it.

Agreed.

> This goes for adding new PEPs, too. I'm not as convinced here, since some
> level of "good enough for a PEP" filtering is probably desirable, but then
> again, it would seem that the people with commit access probably have that
> filter in their head anyways.

Here, common sense and good judgement should be applied.  If there
seems to be consensus that a PEP is needed, there's no need to wait
for Barry.  The update to PEP-0000 commits the assignment of the new
PEP number.  But the new PEP should follow all the rules for a new
PEP!

Having Barry in the loop makes sense for those who aren't sure they
can comply with all the rules, and for those outside the python-dev
community.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 19:29:41 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 13:29:41 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <200011020556.AAA08847@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com>

[Guido]
> [Jeremy and Tim argue about what to do about write access for
> variables at intermediate levels of nesting, neither local nor
> module-global.]
>
> I'll risk doing a pronouncement, even though I know that Jeremy (and
> maybe also Tim?) disagree.
>
> You don't need "write access" (in the sense of being able to assign)
> for variables at the intermediate scopes, so there is no need for a
> syntax to express this.

I can live with that!  Reference-only access to intermediate scopes would
address 99% of current gripes.  Of course, future gripes will shift to that
there's no rebinding access.  If we have to support that someday too, my
preferred way of spelling it in Python requires explicit new syntax, so
adding that later would not break anything.

> Assignments are to local variables (normally) or to module-globals (when
> 'global' is used).  Use references search for a local, then for a local of
> the containing function definition, then for a local in its container,

The Pascal standard coined "closest-containing scope" to describe this
succinctly, and I recommend it for clarity and brevity.

> and so forth, until it hits the module globals, and then finally it looks
> for a builtin.
>
> We can argue over which part of this is done statically and which part
> is done dynamically: currently, locals are done dynamically and
> everything else is done statically.

I'm not sure what you're trying to say there, but to the extent that I think
I grasp it, I believe it's backwards:  locals are static today but
everything else is dynamic (and not in the sense of "dynamic scoping", but
in the operational sense of "requires runtime search to resolve non-local
names", while local names are fully resolved at compile-time today (but in
the absence of "exec" and "import *")).

what's-in-a-name?-a-rose-in-any-other-scope-may-not-smell-as-
     sweet-ly y'rs  - tim





From guido at python.org  Thu Nov  2 07:35:00 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:35:00 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 18:07:15 +0100."
             <3A019F43.E13604BA@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>  
            <3A019F43.E13604BA@lemburg.com> 
Message-ID: <200011020635.BAA09321@cj20424-a.reston1.va.home.com>

[MAL]
> > > It may not look serious, but changing the Python lookup scheme
> > > is, since many inspection tools rely and reimplement exactly
> > > that scheme. With nested scopes, there would be next to no
> > > way to emulate the lookups using these tools.

[GvR]
> > So fix the tools.

[MAL]
> Eek. Are you proposing to break all the Python IDE that are
> just appearing out there ?

Yes.  If a tool does variable scope analysis, it should be prepared
for changes in the rules.  Otherwise we might as well have refused the
syntax changes in 2.0 because they required changes to tools!

> > > To be honest, I don't think static nested scopes buy us all that
> > > much. You can do the same now, by using keyword arguments which
> > > isn't all that nice, but works great and makes the scope clearly
> > > visible.
> > 
> > Yes.  It's a hack that gets employed over and over.  And it has
> > certain problems.  We added 'import as' to get rid of a common
> > practice that was perceived unclean.  Maybe we should support nested
> > scopes to get rid of another unclean common practice?
> 
> I think the common practice mainly comes from the fact,
> that by making globals locals which can benefit from LOAD_FAST
> you get a noticable performance boost.
> 
> So the "right" solution to these weird looking hacks would
> be to come up with a smart way by which the Python compiler
> itself can do the localizing.

Can you elaborate?  I dun't understand what you are proposing here.

> Nested scopes won't help eliminating the current keyword 
> practice.

Why not?  I'd say that

    def create_adder(n):
        def adder(x, n=n): return x+n
	return adder

is a hack and that nested scopes can fix this by allowing you to write


    def create_adder(n):
        def adder(x): return x+n
	return adder
    
like one would expect.  (Don't tell me that it isn't a FAQ why this
doesn't work!)

> > I'm not saying that we definitely should add this to 2.1 (there's
> > enough on our plate already) but we should at least consider it, and
> > now that we have cycle GC, the major argument against it (that it
> > causes cycles) is gone...
> 
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?
> 
> What other advantages would statically nested scopes have ?

Doing what's proper.  Nested scopes are not a bad idea.  They weren't
implemented because they were hard to get right (perhaps impossible
without creating cycles), and I was okay with that because I didn't
like them; but I've been convinced that examples like the second
create_adder() above should reall work.  Just like float+int works
(in Python 0.1, this was a type-error -- you had to case the int arg
to a float to get a float result).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 07:40:19 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:40:19 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:45:12 +0100."
             <3A019A18.20D12FE1@lemburg.com> 
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <14849.37568.510427.834971@bitdiddle.concentric.net>  
            <3A019A18.20D12FE1@lemburg.com> 
Message-ID: <200011020640.BAA09370@cj20424-a.reston1.va.home.com>

> Well first you'd have to change all tools to use the new
> scheme (this includes debuggers, inspection tools, reflection
> kits, etc.). This certainly is not a smart thing to do since
> Python IDEs are just starting to appear -- you wouldn't want
> to break all those.

I've seen a Komodo demo.  Yes, it does this.  But it's soooooooo far
from being done that adding this wouldn't really slow them down much,
I think.  More likely, the toolmakers will have fun competing with
each other to be the first to support this! :-)

> What get's harder with the nested scheme is that
> you can no longer be certain that globals() reaches out to
> the module namespace. But this is needed by some lazy evaluation
> tools. Writing to globals() would not be defined anymore --
> where should you bind the new variable ?

I think globals() should return the module's __dict__, and the global
statement should cause the variable to reach directly into there.

We'll need to introduce a new builtin to do a name *lookup* in the
nested scopes.

> Another problem is that there probably won't be a way to access
> all the different nesting levels on a per-level basis (could be
> that I'm missing something here, but debugging tools would need
> some sort of scope() builtin to access the different scopes).
> I'm not sure whether this is possible to do without some sort
> of link between the scopes. We currently don't need such links.

Correct.  That link may have to be added to the frame object.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From fredrik at effbot.org  Thu Nov  2 19:56:40 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Thu, 2 Nov 2000 19:56:40 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>              <3A019F43.E13604BA@lemburg.com>  <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <000f01c044ff$47b72cb0$3c6340d5@hagrid>

Guido van Rossum wrote:> [MAL]
> > Eek. Are you proposing to break all the Python IDE that are
> > just appearing out there ?
> 
> Yes.  If a tool does variable scope analysis, it should be prepared
> for changes in the rules.

we can live with that...

</F>




From guido at python.org  Thu Nov  2 07:50:45 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 01:50:45 -0500
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Your message of "02 Nov 2000 17:29:52 GMT."
             <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> 
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk>  
            <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011020650.BAA09505@cj20424-a.reston1.va.home.com>

> Hmm - the interaction of "a=b" style args & *-ed args is a bit
> counter-intuitive, particularly as the "a=b" args syntatically have to
> come before the *-ed args:
> 
> >>> def f(a,b,c,d):
> ...  return a,b,c,d
> ... 
> >>> f(1,c=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'c' redefined in call to f()
> >>> f(1,b=3,*(2,4))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: keyword parameter 'b' redefined in call to f()
> >>> f(1,d=4,*(2,3))
> (1, 2, 3, 4)
> 
> I humbly submit that This Is Wrong.  I haven't seen anybody complain
> about it, which suggests to me that noone is using this combination,
> and I propose either:
> 
> 1) banning it
> 2) demanding that the *-ed arg precede the "a=b" args

Interesting.  The *-ed arg cannot precede the a=b args currently, but
I agree that it would have made more sense to do it that way.

I'm not sure that it's worth allowing the combination of kw args and
*tuple, but I'l also not sure that it *isn't* worth it.

> Of course, if noone is using this "feature", then maybe this dusty
> little corner of the language should be left undisturbed.

I think it may be best to leave it alone.

> And I haven't even thought about default arguments yet...

That's unrelated.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Thu Nov  2 19:56:55 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Thu, 2 Nov 2000 12:56:55 -0600 (CST)
Subject: [Python-Dev] PythonLabs -> Digital Creations
Message-ID: <14849.47351.835291.365708@buffalo.fnal.gov>

So, when are the relevant websites going to be updated to reflect the
new reality?  Pythonlabs.com and BeOpen.com still reflect the old
regime...  I'd think the Digital Creations folks would be eager to
announce the changing of the guard...



From gstein at lyra.org  Thu Nov  2 19:57:07 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 2 Nov 2000 10:57:07 -0800
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Thu, Nov 02, 2000 at 01:12:38PM -0500
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>
Message-ID: <20001102105707.H2037@lyra.org>

On Thu, Nov 02, 2000 at 01:12:38PM -0500, Fred L. Drake, Jr. wrote:
> 
> Jeremy Hylton writes:
>  > Despite the error, I did get a successful build this time.  I guess
>  > your configure change worked.
> 
>   I don't think the bug I fixed and the bug you reported were tightly
> related.  I don't understand why you got the error after before or
> after my change.
>   I'f you're running autoconf 2.13, though, please do check in the new
> configure script!

Wouldn't it make sense to do the upgrade to take advantages of the autoconf
fixes? If there are any problems with the change, then now is the time to do
it and to sort them out!

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From tim_one at email.msn.com  Thu Nov  2 20:07:18 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:07:18 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.37426.860007.989619@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>

[Jeremy]
> I agree that visual inspection is a tad harder, but I contend that
> existing programs that use the same name for a global variable and a
> local variable -- and intend for the global to be visible within a
> function nested in the local variable's region -- are confusing.
> It's too hard for a first-time reader of the code to figure out what
> is going on.
>
> Incidentally, I have yet to see an example of this problem occurring
> in anyone's code.  All the examples seem a bit contrived.  I wonder if
> anyone has an example in existing code.

I wasn't the one making the "will break code" argument (I'm sure it will,
but very little, and not at all for most people since most people never nest
functions in Python today apart from default-abusing lambdas).  Visual
inspection stands on its own as a potential problem.

>   [My SICP example omitted]
>
>   TP> Unfortunately for proponents, this is exactly the kind of SICP
>   TP> example that is much better done via a class.
>
> Indeed, the PEP says exactly that: This kind of program is better
> done via a class.  My intent was not to show a compelling use of
> mutable state.  Instead it was to show that with read-only access,
> people could still modify values on enclosing scopes.  The issue is
> whether the language allows the programmer to express this intent
> clearly or if she has to jump through some hoops to accomplish it.

Guido said "jump through some hoops", so I'm dropping it, but first noting
that "the container" in *idiomatic* Python will most often be "self":

    def yadda(self, ...):
        def whatever(amount):
            self.balance += amount
        return whatever

will work to rebind self.balance.  I don't think Guido will ever be
interested in supporting idiomatic Scheme.

>   TP>     def deposit(amount):
>   TP>         global bank_account.balance
>   TP>         balance += amount

> I'm still not sure I like it, because it mixes local variables of a
> function with attribute access on objects.  I'll add it to the
> discussion in the PEP (if Barry approves the PEP <wink>), though.

Actually, no connection to attribute access was intended there:  it was just
a backward-compatible way to spell the pair (name of containing scope, name
of vrbl in that scope).

    global back_account:balance
or
    global balance from bank_account

would do as well (and maybe better as they don't imply attribute access; but
maybe worse as they don't bring to mind attribute access <wink>).

> Do you have any opinion on the subtleties?  The two that immediately
> come to mind are: 1) whether the function's local are available as
> attributes anywhere or only in nested scopes

I didn't intend attribute-like access at all (although we had earlier talked
about that wrt JavaScript, I didn't have that in mind here).

> and 2) whether you can create new local variable using this notation.

Yes, that's the primary one I was thinking about.





From tim_one at email.msn.com  Thu Nov  2 20:10:52 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:10:52 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.38088.903104.936944@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>

[Barry A. Warsaw]
> If we get lexical scoping, there should be a fast (built-in) way to
> get at all the accessible names from Python.  I.e. currently I can do
>
>     d = globals().copy()
>     d.update(locals())
>
> and know that `d' contains a dictionary of available names, with the
> right overloading semantics.

It was long ago agreed (don't you love how I pull this stuff out of thin
historical air <wink>?) that if nested lexical scoping was added, we would
need also to supply a new mapping object that mimicked the full Python
lookup rules (including builtins).  Not necessarily a dictionary, though.





From guido at python.org  Thu Nov  2 08:15:50 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:15:50 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:33:11 EST."
             <14849.42327.23020.553510@anthem.concentric.net> 
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de>  
            <14849.42327.23020.553510@anthem.concentric.net> 
Message-ID: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>

>     PF> It was obviously not intended to be mailed out that way again.
>     PF> Problem with pathname and/or current directory?  Barry got
>     PF> this right once, now it is broken again.
> 
> Except that AFAIK, I didn't do anything to fix it, or to break it
> again.

We have too much to do to try and fix this now.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 20:14:50 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:14:50 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A019F43.E13604BA@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEPHOAA.tim_one@email.msn.com>

[MAL]
> ...
> Hmm, so far the only argument for changing Python lookups
> was to allow writing lambdas without keyword hacks. Does this
> really warrant breaking code ?

*Some* amount of code, sure.  Hard to quantify, but hard to believe there's
much code at risk.

> What other advantages would statically nested scopes have ?

Pythonic obviousness.  Virtually everyone coming to Python from other
languages *expects* visually nested functions to work this way.  That they
don't today is a very frequent source of surprises and complaints.





From barry at wooz.org  Thu Nov  2 20:31:37 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Thu, 2 Nov 2000 14:31:37 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
References: <14849.38088.903104.936944@anthem.concentric.net>
	<LNBBLJKPBEHFEDALKOLCOEEOHOAA.tim_one@email.msn.com>
Message-ID: <14849.49433.197319.973848@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim_one at email.msn.com> writes:

    TP> It was long ago agreed (don't you love how I pull this stuff
    TP> out of thin historical air <wink>?) that if nested lexical
    TP> scoping was added, we would need also to supply a new mapping
    TP> object that mimicked the full Python lookup rules (including
    TP> builtins).  Not necessarily a dictionary, though.

Oh yeah, I vaguely remember now <0.5 scratch-head>.

Works for me, although I'll point out that we needn't wait for lexical
scoping to provide such an object!

-Barry



From guido at python.org  Thu Nov  2 08:34:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:34:53 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:57:09 EST."
             <14849.43765.419161.105395@bitdiddle.concentric.net> 
References: <Pine.GSO.4.10.10011021724340.1290-100000@sundial> <3A0190BA.940FFADA@lemburg.com> <200011020545.AAA08743@cj20424-a.reston1.va.home.com>  
            <14849.43765.419161.105395@bitdiddle.concentric.net> 
Message-ID: <200011020734.CAA09751@cj20424-a.reston1.va.home.com>

> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

That's not what I meant, but I expressed it clumsily.  Perhaps the
terminology is just inadequate.  I simply meant that builtins can be
overridden by module-globals.  But only in the module whose globals
are searched for globals -- that part is still static.

Let's drop this thread...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Thu Nov  2 20:31:59 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Nov 2000 14:31:59 -0500 (EST)
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: <14849.49013.363302.823824@bitdiddle.concentric.net>
References: <14848.39586.832800.139182@bitdiddle.concentric.net>
	<14849.43647.550092.678278@cj42289-a.reston1.va.home.com>
	<14849.44186.494602.297323@bitdiddle.concentric.net>
	<14849.44210.93217.431262@cj42289-a.reston1.va.home.com>
	<14849.45483.304866.556547@bitdiddle.concentric.net>
	<14849.45714.207143.196945@cj42289-a.reston1.va.home.com>
	<14849.47007.862290.364694@bitdiddle.concentric.net>
	<14849.48464.183713.819894@cj42289-a.reston1.va.home.com>
	<14849.49013.363302.823824@bitdiddle.concentric.net>
Message-ID: <14849.49455.97877.379601@cj42289-a.reston1.va.home.com>

[We've uncovered the root of Jeremy's problem here -- this is a
 problem/limitation of make with VPATH.]

Jeremy Hylton writes:
 > There is a Setup.local in the source directory.

  That's why it isn't being built.  Since it's on the VPATH, make
thinks it already exists in a usable form.  Since the command that
uses it doesn't look for the "most local" version but assumes it
exists in the right place, the command fails.
  This is a problem that exists with VPATH; the built files cannot
already exist in the source directories (hence, the build directories
can't refer to source dirs which have been used as build dirs; "make
clobber" isn't good enough for the Setup* files).
  Remove (or move) the existing file, and it should work fine.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From tim_one at email.msn.com  Thu Nov  2 20:41:12 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:41:12 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: <14849.47351.835291.365708@buffalo.fnal.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFBHOAA.tim_one@email.msn.com>

[Charles G Waldman]
> So, when are the relevant websites going to be updated to reflect the
> new reality?

Sorry, we have no idea.

> Pythonlabs.com and BeOpen.com still reflect the old regime...

And they're running on BeOpen.com machines, which we can no longer fiddle
with.  The people remaining at BeOpen.com probably don't view updating those
sites as a priority anymore (you all saw how long it took them to get
pythonlabs.com and Starship merely running again, although at the time we
couldn't tell you *why* we weren't able to fix them ourselves).

> I'd think the Digital Creations folks would be eager to announce the
> changing of the guard...

They're getting the word out in their own way.  Hope you saw the cover story
on LWN today!

    http://www.lwn.net/

There's a lot of good info there, including useful interviews with Paul
Everitt and Guido.





From mwh21 at cam.ac.uk  Thu Nov  2 20:45:47 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 02 Nov 2000 19:45:47 +0000
Subject: [Python-Dev] Python Call Mechanism
In-Reply-To: Guido van Rossum's message of "Thu, 02 Nov 2000 01:50:45 -0500"
References: <200010301715.JAA32564@slayer.i.sourceforge.net> <39FDB5EA.B60EA39A@lemburg.com> <14845.36020.17063.951147@bitdiddle.concentric.net> <m3aebjftn0.fsf_-_@atrus.jesus.cam.ac.uk> <200011020424.XAA07810@cj20424-a.reston1.va.home.com> <m34s1qfgwa.fsf@atrus.jesus.cam.ac.uk> <m31ywuffjj.fsf@atrus.jesus.cam.ac.uk> <200011020650.BAA09505@cj20424-a.reston1.va.home.com>
Message-ID: <m3vgu6duok.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > Hmm - the interaction of "a=b" style args & *-ed args is a bit
> > counter-intuitive, particularly as the "a=b" args syntatically have to
> > come before the *-ed args:
> > 
> > >>> def f(a,b,c,d):
> > ...  return a,b,c,d
> > ... 
> > >>> f(1,c=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'c' redefined in call to f()
> > >>> f(1,b=3,*(2,4))
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: keyword parameter 'b' redefined in call to f()
> > >>> f(1,d=4,*(2,3))
> > (1, 2, 3, 4)
> > 
> > I humbly submit that This Is Wrong.  I haven't seen anybody complain
> > about it, which suggests to me that noone is using this combination,
> > and I propose either:
> > 
> > 1) banning it
> > 2) demanding that the *-ed arg precede the "a=b" args
> 
> Interesting.  The *-ed arg cannot precede the a=b args currently, but
> I agree that it would have made more sense to do it that way.
> 
> I'm not sure that it's worth allowing the combination of kw args and
> *tuple, but I'l also not sure that it *isn't* worth it.
> 
> > Of course, if noone is using this "feature", then maybe this dusty
> > little corner of the language should be left undisturbed.
> 
> I think it may be best to leave it alone.

Oh, I mostly agree and am not going to put any effort into this area -
but if I or someone else manages to make striking simplifications to
the function call code that has the side effect of banning/changing
the syntax of this combination, then this probably wouldn't be a good
enough reason for its rejection.

> > And I haven't even thought about default arguments yet...
> 
> That's unrelated.

I know, it was just something that occurred to me as adding more
complexity.  Hypothetical excercise: Write a description of Python's
current function call behaviour in sufficient detail to allow
reimplementation of it without playing with an existing interpreter in
less than 1000 words.  It's just hairy, and it amazes me that it feels
so natural most of the time...

Cheers,
M.

-- 
  Not only does the English Language borrow words from other
  languages, it sometimes chases them down dark alleys, hits 
  them over the head, and goes through their pockets.   -- Eddy Peters




From guido at python.org  Thu Nov  2 08:51:15 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:51:15 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:20:06 EST."
             <14849.45142.860806.358489@anthem.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <14849.38088.903104.936944@anthem.concentric.net> <200011020602.BAA08974@cj20424-a.reston1.va.home.com>  
            <14849.45142.860806.358489@anthem.concentric.net> 
Message-ID: <200011020751.CAA12539@cj20424-a.reston1.va.home.com>

[Barry]
> Saw that.  I was just thinking that locals() already does what
> vars()-no-args does, so why have two ways to do the same thing?

Not clear -- maybe one of them needs to be made obsolete.

>     GvR> I don't think you need to have a mechanism to find all
>     GvR> accessible names; I don't see a common use for that.  It's
>     GvR> sufficient to have a mechanism to look up any specific name
>     GvR> according to whatever mechanism we decide upon.  This is
>     GvR> needed for internal use of course; it can also be useful for
>     GvR> e.g. variable substitution mechanisms like the one you
>     GvR> recently proposed or Ping's Itmpl.
> 
> Ah, something like this then:
[Example deleted]

I'm not sure that the mechanism provided should follow the mapping
API.  *All* it needs to do it provide lookup capability.  Having
.keys(), .items(), .has_key() etc. just slows it down.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov  2 20:54:05 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 14:54:05 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.43765.419161.105395@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFCHOAA.tim_one@email.msn.com>

[Jeremy]
> I don't think I buy your explanation that Python uses dynamic scope
> for resolving globals.

It's dynamic in the shallow (but real!) sense that it can't be determined to
which of {module scope, builtin scope} a global name resolves today until
runtime.  Indeed, in

def f():
    print len

the resolving scope for "len" may even change with each invocation of f(),
depending on who's playing games with the containing module's __dict__.
That's not "dynamic scoping" in the proper sense of the term, but it's sure
dynamic!

words-lead-to-more-words-so-become-one-with-the-essence-instead<wink>-ly
    y'rs  - tim





From guido at python.org  Thu Nov  2 08:59:47 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:29:41 EST."
             <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCKEEJHOAA.tim_one@email.msn.com> 
Message-ID: <200011020759.CAA12600@cj20424-a.reston1.va.home.com>

> [Guido]
> > [Jeremy and Tim argue about what to do about write access for
> > variables at intermediate levels of nesting, neither local nor
> > module-global.]
> >
> > I'll risk doing a pronouncement, even though I know that Jeremy (and
> > maybe also Tim?) disagree.
> >
> > You don't need "write access" (in the sense of being able to assign)
> > for variables at the intermediate scopes, so there is no need for a
> > syntax to express this.

[Tim]
> I can live with that!  Reference-only access to intermediate scopes would
> address 99% of current gripes.  Of course, future gripes will shift to that
> there's no rebinding access.  If we have to support that someday too, my
> preferred way of spelling it in Python requires explicit new syntax, so
> adding that later would not break anything.

Exactly my point.

> > Assignments are to local variables (normally) or to module-globals (when
> > 'global' is used).  Use references search for a local, then for a local of
> > the containing function definition, then for a local in its container,
> 
> The Pascal standard coined "closest-containing scope" to describe this
> succinctly, and I recommend it for clarity and brevity.

Thanks, that's a good term.  And the principle is totally
uncontroversial.

> > and so forth, until it hits the module globals, and then finally it looks
> > for a builtin.
> >
> > We can argue over which part of this is done statically and which part
> > is done dynamically: currently, locals are done dynamically and
> > everything else is done statically.
> 
> I'm not sure what you're trying to say there, but to the extent that I think
> I grasp it, I believe it's backwards:  locals are static today but
> everything else is dynamic (and not in the sense of "dynamic scoping", but
> in the operational sense of "requires runtime search to resolve non-local
> names", while local names are fully resolved at compile-time today (but in
> the absence of "exec" and "import *")).

Oops, yes, I had it backwards.

As I said elsewhere, in Python 3000 I'd like to do it all more
statically.  So perhaps we should look up nested locals based on
static information too.  Thus:

  x = "global-x"

  def foo():
      if 0:
          x = "x-in-foo"
      def bar():
	  return x
      return bar

  print foo()()

should raise UnboundLocalError, not print "global-x".

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 08:59:55 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 02:59:55 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 13:06:12 EST."
             <14849.44308.333212.37111@bitdiddle.concentric.net> 
References: <14848.27102.223001.369662@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCKECCHOAA.tim_one@email.msn.com> <14849.37426.860007.989619@bitdiddle.concentric.net> <200011020556.AAA08847@cj20424-a.reston1.va.home.com>  
            <14849.44308.333212.37111@bitdiddle.concentric.net> 
Message-ID: <200011020759.CAA12609@cj20424-a.reston1.va.home.com>

> Don't know if you saw the discussion in the PEP or not.

Sorry, I had no time.  I have read it now, but it doesn't change my
point of view.

> I made two
> arguments for being able to assign to variables bound in enclosing
> scopes.
> 
> 1. Every other language that supports nested lexical scoping allows
>    this.  To the extent that programmers have seen these other
>    languages, they will expect it to work.

But we have a unique way of declaring variables, which makes the
issues different.

Your PEP wonders why I am against allowing assignment to intermediate
levels.  Here's my answer: all the syntaxes that have been proposed
to spell this have problems.  So let's not provide a way to spell it.
I predict that it won't be a problem.  If it becomes a problem, we can
add a way to spell it later.  I expect that the mechanism that will be
used to find variables at intermediate levels can also be used to set
them, so it won't affect that part of the implementation much.

> 2. It is possible to work around this limitation by using containers.
>    If you want to have an integer that can be updated by nested
>    functions, you wrap the interger in a list and make all assignments
>    and references refer to list[0].  It would be unfortunate if
>    programmers used this style, because it is obscure.  I'd rather see
>    the language provide a way to support this style of programming
>    directly. 

I don't expect that programmers will use this style.  When they have
this need, they will more likely use a class.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 09:08:14 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:08:14 -0500
Subject: [Python-Dev] PythonLabs -> Digital Creations
In-Reply-To: Your message of "Thu, 02 Nov 2000 12:56:55 CST."
             <14849.47351.835291.365708@buffalo.fnal.gov> 
References: <14849.47351.835291.365708@buffalo.fnal.gov> 
Message-ID: <200011020808.DAA16162@cj20424-a.reston1.va.home.com>

> So, when are the relevant websites going to be updated to reflect the
> new reality?  Pythonlabs.com and BeOpen.com still reflect the old
> regime...  I'd think the Digital Creations folks would be eager to
> announce the changing of the guard...

While technically I have the capability to update the pythonlabs.com
website, ethically I feel I cannot do this.  The site belongs to
BeOpen.  I'll ask them to make some kind of announcement.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 09:16:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 03:16:53 -0500
Subject: [Python-Dev] Modules/makesetup loop
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:57:07 PST."
             <20001102105707.H2037@lyra.org> 
References: <14848.39586.832800.139182@bitdiddle.concentric.net> <14849.43647.550092.678278@cj42289-a.reston1.va.home.com> <14849.44186.494602.297323@bitdiddle.concentric.net> <14849.44210.93217.431262@cj42289-a.reston1.va.home.com> <14849.44784.479281.162333@bitdiddle.concentric.net> <14849.44694.303665.854116@cj42289-a.reston1.va.home.com>  
            <20001102105707.H2037@lyra.org> 
Message-ID: <200011020816.DAA16227@cj20424-a.reston1.va.home.com>

> Wouldn't it make sense to do the upgrade to take advantages of the autoconf
> fixes? If there are any problems with the change, then now is the time to do
> it and to sort them out!

Yeah, but I have too much to do to get to this any time soon -- it
just isn't important enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From skip at mojam.com  Thu Nov  2 22:19:11 2000
From: skip at mojam.com (Skip Montanaro)
Date: Thu, 2 Nov 2000 15:19:11 -0600 (CST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
Message-ID: <14849.55887.283432.505910@beluga.mojam.com>

>>>>> "Tim" == Tim Peters <tim_one at email.msn.com> writes:

    Tim> [Jeremy]
    >> All the examples seem a bit contrived.  I wonder if anyone has an
    >> example in existing code.

    Tim> I wasn't the one making the "will break code" argument ...

Nor was I.  Jeremy (I think) asked MAL how it could break code.  I posted a
(simple, but obviously contrived) example.  I have no particular opinion on
this subject.  I was just trying to answer Jeremy's question.

Skip



From jeremy at alum.mit.edu  Thu Nov  2 21:25:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 15:25:36 -0500 (EST)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.55887.283432.505910@beluga.mojam.com>
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
	<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
	<14849.55887.283432.505910@beluga.mojam.com>
Message-ID: <14849.52672.943439.978785@bitdiddle.concentric.net>

Looks like we need to rehash this thread at least enough to determine
who is responsible for causing us to rehash it.

MAL said it would break code.  I asked how.  Skip and Tim obliged with
examples.  I said their examples exhibited bad style; neither of them
claimed they were good style.

In the end, I observed that while it could break code in theory, I
doubted it really would break much code.  Furthermore, I believe that
the code it will break is already obscure so we needn't worry about
it.

Jeremy



From tim_one at email.msn.com  Thu Nov  2 22:13:47 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 2 Nov 2000 16:13:47 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFKHOAA.tim_one@email.msn.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
>
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.

Ah!  I wasn't trying to give an example of code that would break, but, ya,
now that you mention it, it would.  I was just giving an example of why
visual inspection will be harder in Python than in Pascal.

I expect that the kind of code I showed *will* be common: putting all the
nested "helper functions" at the top of a function, just as was also done in
Pascal.  The killer difference is that in Pascal, the containing function's
locals referenced by the nested helpers can be found declared at the top of
the containing function; but in Python you'll have to search all over the
place, and if you don't find a non-local var at once, you'll be left uneasy,
wondering whether you missed a binding target, or whether the var is truly
global, or what.





From guido at python.org  Thu Nov  2 10:41:01 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 04:41:01 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Thu, 02 Nov 2000 15:25:36 EST."
             <14849.52672.943439.978785@bitdiddle.concentric.net> 
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com>  
            <14849.52672.943439.978785@bitdiddle.concentric.net> 
Message-ID: <200011020941.EAA18905@cj20424-a.reston1.va.home.com>

[Jeremy]
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's quite enough rehashing.  I don't think we'll have to worry
about breakiung much code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Thu Nov  2 22:47:35 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 22:47:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0227.txt,NONE,1.1
In-Reply-To: <200011020715.CAA09616@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 02:15:50AM -0500
References: <200011021618.IAA15298@slayer.i.sourceforge.net> <m13rNhx-000Dm9C@artcom0.artcom-gmbh.de> <14849.42327.23020.553510@anthem.concentric.net> <200011020715.CAA09616@cj20424-a.reston1.va.home.com>
Message-ID: <20001102224734.U12812@xs4all.nl>

On Thu, Nov 02, 2000 at 02:15:50AM -0500, Guido van Rossum wrote:

> >     PF> It was obviously not intended to be mailed out that way again.
> >     PF> Problem with pathname and/or current directory?  Barry got
> >     PF> this right once, now it is broken again.

> We have too much to do to try and fix this now.

But I haven't ;) I've made syncmail give some more information if it can't
find the file, to compensate for the fact that cvs.python.sourceforge.net
runs a Python before 1.5.2, and thus doesn't show the file it couldn't find.
I won't bother with creating files just for the hell of it, so we'll just
wait until Barry triggers it again.

(Not that I'm bored (not much, anyway), but this was a no-brainer.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From MarkH at ActiveState.com  Thu Nov  2 22:55:07 2000
From: MarkH at ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:55:07 +1100
Subject: [Python-Dev] please subscribe me ;-)
Message-ID: <LCEPIIGDJPKCOIHOBJEPAENNCFAA.MarkH@ActiveState.com>

Hi all,
	My skippinet.com.au address is temporarily bouncing, and thus I am no
longer getting python-dev mail.  I re-subscribed on Monday, but am still
waiting moderator approval.

Can someone approve me please?

Thanks,

Mark.




From MarkH at ActiveState.com  Thu Nov  2 22:57:58 2000
From: MarkH at ActiveState.com (Mark Hammond)
Date: Fri, 3 Nov 2000 08:57:58 +1100
Subject: [Python-Dev] Dont bother subscribing me!
Message-ID: <LCEPIIGDJPKCOIHOBJEPEENNCFAA.MarkH@ActiveState.com>

All done already - sorry about the noise.

Thanks,

Mark.



From guido at python.org  Thu Nov  2 11:06:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 05:06:16 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 10:34:42 EST."
             <20001102103442.B5027@kronos.cnri.reston.va.us> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial>  
            <20001102103442.B5027@kronos.cnri.reston.va.us> 
Message-ID: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>

[Andrew]
> Stuff I personally want to get done:
>   * Finish PEP 222, "Web Programming Improvements" and implement whatever
>     emerges from it.

I just skimmed PEP 222.  I agree that the classes defined by cgi.py
are unnecessarily arcane.  I wonder if it isn't better to just start
over rather than trying to add yet another new class to the already
top-heavy CGI module???

Regarding file uploads: you *seem* to be proposing that uploaded files
should be loaded into memory only.  I've got complaints from people
who are uploading 10 Mb files and don't appreciate their process
growing by that much.  Perhaps there should be more options, so that
the caller can control the disposition of uploads more carefully?
What's wrong with subclassing?  Maybe two example subclasses should be
provided?

Regarding templating -- what's wrong with HTMLgen as a starting point?
Just that it's too big?  I've never used it myself, but I've always
been impressed with its appearance. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Thu Nov  2 23:12:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 17:12:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <200011021006.FAA19152@cj20424-a.reston1.va.home.com>; from guido@python.org on Thu, Nov 02, 2000 at 05:06:16AM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>
Message-ID: <20001102171226.A21364@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:06:16AM -0500, Guido van Rossum wrote:
>I just skimmed PEP 222.  I agree that the classes defined by cgi.py
>are unnecessarily arcane.  I wonder if it isn't better to just start
>over rather than trying to add yet another new class to the already
>top-heavy CGI module???

I've wondered about that, too; writing a neat request class that wraps
up field values, cookies, and environment variables, and provides
convenience functions for re-creating the current URL, Something like
the request classes in Zope and Webware.

>Regarding file uploads: you *seem* to be proposing that uploaded files
>should be loaded into memory only.  I've got complaints from people

Mrr...?  The only file upload reference simply says you shouldn't have
to subclass in order to use them; it's not implying files have to be
read into memory.  (We have to deal with whackingly large mask layout
files at work, after all.)

>Regarding templating -- what's wrong with HTMLgen as a starting point?
>Just that it's too big?  I've never used it myself, but I've always
>been impressed with its appearance. :-)

I, personally, am against including templating, but it was suggested.
I'm against it because there are too many solutions with different
tradeoffs.  Do you want a simple regex search-and-replace,
constructing HTML pages as Python objects, or a full-blown
minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
Python-compatible syntax?  Much better just to move templating into
the "Rejected" category and give the above rationale.

--amk




From jeremy at alum.mit.edu  Thu Nov  2 23:07:37 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 2 Nov 2000 17:07:37 -0500 (EST)
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <20001102171226.A21364@kronos.cnri.reston.va.us>
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011021727341.1290-100000@sundial>
	<20001102103442.B5027@kronos.cnri.reston.va.us>
	<200011021006.FAA19152@cj20424-a.reston1.va.home.com>
	<20001102171226.A21364@kronos.cnri.reston.va.us>
Message-ID: <14849.58793.132399.810370@bitdiddle.concentric.net>

Since today has been busy on the meta-sig, I wonder if we should
create a web-sig to thrash out these issues.

Jeremy



From thomas at xs4all.net  Thu Nov  2 23:51:52 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 2 Nov 2000 23:51:52 +0100
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: <200011020500.VAA11872@sf-web2.i.sourceforge.net>; from noreply@sourceforge.net on Wed, Nov 01, 2000 at 09:00:43PM -0800
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>
Message-ID: <20001102235152.N12776@xs4all.nl>

On Wed, Nov 01, 2000 at 09:00:43PM -0800, noreply at sourceforge.net wrote:

> Comment:
> Accepted and assigned back to Thomas.

> Guido approved of this "in theory" before, so go for it!

Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
prefix, etc) and I was about to check it in, when I got cold feet. This
change is going to break programs embedding or extending python, that rely
on getopt() (the C function) being available, one way or another. After this
move, that is no longer necessarily true. I got cold feet because of
Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
optind and optarg, and doesn't bother with #include <getopt.h> or some such.

Before, that would probably always work, depending on which library/archive
was searched first, for the getopt() function. It would only break if
system-libraries were searched first, for some reason, and the system
libraries provide a broken getopt. In all other cases, like there not being
a getopt, getopt needing seperate includes or defines, or separate link or
compile arguments, the Python getopt would be used, and it would do what was
expected (probably, anyway ;)

After the change, people need to use the system getopt(), and do their own
hoop-jumping to find out if it's there, how to call it, etc. Now I myself
have never written anything that would depend on Python providing getopt(),
but that doesn't say anything at all. Is this acceptable breakage ? Or
should we go the whole nine yards, make it an official API, document it, and
provide backwards-compatible symbols on platforms where getopt used to be
used ?

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Thu Nov  2 14:41:53 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:41:53 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: Your message of "Thu, 02 Nov 2000 17:12:26 EST."
             <20001102171226.A21364@kronos.cnri.reston.va.us> 
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com>  
            <20001102171226.A21364@kronos.cnri.reston.va.us> 
Message-ID: <200011021341.IAA19469@cj20424-a.reston1.va.home.com>

> >Regarding file uploads: you *seem* to be proposing that uploaded files
> >should be loaded into memory only.  I've got complaints from people
> 
> Mrr...?  The only file upload reference simply says you shouldn't have
> to subclass in order to use them; it's not implying files have to be
> read into memory.  (We have to deal with whackingly large mask layout
> files at work, after all.)

Mrrauw...?  Do you really have to subclass?  I thought it just says
that you can subclass if you're not happy with the given make_file()
implementation?

> >Regarding templating -- what's wrong with HTMLgen as a starting point?
> >Just that it's too big?  I've never used it myself, but I've always
> >been impressed with its appearance. :-)
> 
> I, personally, am against including templating, but it was suggested.
> I'm against it because there are too many solutions with different
> tradeoffs.  Do you want a simple regex search-and-replace,
> constructing HTML pages as Python objects, or a full-blown
> minilanguage?  HTML/XML-compatibile syntax, ASP-compatible syntax,
> Python-compatible syntax?  Much better just to move templating into
> the "Rejected" category and give the above rationale.

Sure -- I'm perfectly happy with ad-hoc templating solutions myself
(see my FAQ wizard).  I've also heard Tom Christiansen complain that
Perl is slower to start up than Python for CGI work -- because the
templating classes are so big, and are all loaded at startup!

I do see a use for a helper or helpers to creates tables though --
tables are notoriously tag-intensive and hard to get right.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov  2 14:51:14 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 02 Nov 2000 08:51:14 -0500
Subject: [Python-Dev] Re: [Patch #102170] move getopt() to Py_GetOpt() and use it unconditionally
In-Reply-To: Your message of "Thu, 02 Nov 2000 23:51:52 +0100."
             <20001102235152.N12776@xs4all.nl> 
References: <200011020500.VAA11872@sf-web2.i.sourceforge.net>  
            <20001102235152.N12776@xs4all.nl> 
Message-ID: <200011021351.IAA19544@cj20424-a.reston1.va.home.com>

> Well, I changed everything you wanted changed (use _PyOS_ rather than Py_ as
> prefix, etc) and I was about to check it in, when I got cold feet. This
> change is going to break programs embedding or extending python, that rely
> on getopt() (the C function) being available, one way or another. After this
> move, that is no longer necessarily true. I got cold feet because of
> Demo/pysrv/pysrv.c, which just adds the necessary externs for getopt(),
> optind and optarg, and doesn't bother with #include <getopt.h> or some such.
> 
> Before, that would probably always work, depending on which library/archive
> was searched first, for the getopt() function. It would only break if
> system-libraries were searched first, for some reason, and the system
> libraries provide a broken getopt. In all other cases, like there not being
> a getopt, getopt needing seperate includes or defines, or separate link or
> compile arguments, the Python getopt would be used, and it would do what was
> expected (probably, anyway ;)
> 
> After the change, people need to use the system getopt(), and do their own
> hoop-jumping to find out if it's there, how to call it, etc. Now I myself
> have never written anything that would depend on Python providing getopt(),
> but that doesn't say anything at all. Is this acceptable breakage ? Or
> should we go the whole nine yards, make it an official API, document it, and
> provide backwards-compatible symbols on platforms where getopt used to be
> used ?

Don't worry -- getopt has never been something offered by Python.
It's something assumed to be in the system library, and if it isn't we
provide our own -- but we don't provide it for the benefit of those
who embed us.  (In fact, if Python is embedded, it will not *use*
getopt.)  We can't even guarantee to provide it, because it's likely
already in the system library.  (That's why I like that the prefix
begins with an underscore -- those are not provided for use by others,
only for internal use.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Fri Nov  3 04:57:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 2 Nov 2000 22:57:26 -0500
Subject: [Python-Dev] Web Programming Improvements
In-Reply-To: <14849.58793.132399.810370@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Thu, Nov 02, 2000 at 05:07:37PM -0500
References: <200011020329.WAA07330@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011021727341.1290-100000@sundial> <20001102103442.B5027@kronos.cnri.reston.va.us> <200011021006.FAA19152@cj20424-a.reston1.va.home.com> <20001102171226.A21364@kronos.cnri.reston.va.us> <14849.58793.132399.810370@bitdiddle.concentric.net>
Message-ID: <20001102225726.A21509@kronos.cnri.reston.va.us>

On Thu, Nov 02, 2000 at 05:07:37PM -0500, Jeremy Hylton wrote:
>Since today has been busy on the meta-sig, I wonder if we should
>create a web-sig to thrash out these issues.

There's already a python-web-modules at egroups.com list, for authors 
of Python Web frameworks; fairly on-topic for that list, I should think.

--amk



From moshez at math.huji.ac.il  Fri Nov  3 11:18:26 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Fri, 3 Nov 2000 12:18:26 +0200 (IST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011020610.BAA09073@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>

On Thu, 2 Nov 2000, Guido van Rossum wrote:

> I think Python 3000 ought to use totally static scoping.  That will
> make it possible to do optimize code using built-in names!

Isn't that another way of saying you want the builtin names to be
part of the language definition? Part of today's method advantages
is that new builtins can be added without any problems.

no-clear-win-either-way-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Fri Nov  3 11:02:13 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:02:13 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
		<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
		<14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>
Message-ID: <3A028D25.B6A227F2@lemburg.com>

Jeremy Hylton wrote:
> 
> Looks like we need to rehash this thread at least enough to determine
> who is responsible for causing us to rehash it.
> 
> MAL said it would break code.  I asked how.  Skip and Tim obliged with
> examples.  I said their examples exhibited bad style; neither of them
> claimed they were good style.
> 
> In the end, I observed that while it could break code in theory, I
> doubted it really would break much code.  Furthermore, I believe that
> the code it will break is already obscure so we needn't worry about
> it.

That's just what I was trying to say all along: statically
nested scopes don't buy you anything except maybe for lambdas
and nested functions (which is bad style programming, IMHO too).

The only true argument for changing scoping I see is that
of gained purity in language design... without much practical
use.  

Other issues that need sorting out:

x = 2
class C:
  x = 1
  C = 'some string'
  def a(self):
     print x
  def b(self):
     global x
     x = 3

class D(C):
  C = 'some string'
  def a(self):
     C.a(self)
     print C

o = C()
o.a()
o.b()
o.a()

o = D()
o.a()

What would the output look like under your proposal ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 11:46:59 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:46:59 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
Message-ID: <3A0297A3.A778F4FC@lemburg.com>

Moshe Zadka wrote:
> 
> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!
> 
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

+1.

Wouldn't it be more Python-like to provide the compiler with
a set of known-to-be-static global name bindings ?

A simple way of avoiding optimizations like these:

def f(x, str=str):
   return str(x) + '!'

would then be to have the compiler lookup "str" in the globals()
passed to it and assign the found value to the constants of the
function, provided that "str" appears in the list of
known-to-be-static global name bindings (perhaps as optional
addition parameter to compile() with some reasonable default
in sys.staticsymbols).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 11:58:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 11:58:10 +0100
Subject: [Python-Dev] statically nested scopes
References: <200011020037.NAA29447@s454.cosc.canterbury.ac.nz> <3A015A3A.4EC7DBC6@lemburg.com> <200011020329.WAA07330@cj20424-a.reston1.va.home.com>  
	            <3A019F43.E13604BA@lemburg.com> <200011020635.BAA09321@cj20424-a.reston1.va.home.com>
Message-ID: <3A029A42.3C11C774@lemburg.com>

Guido van Rossum wrote:
> 
> > > > To be honest, I don't think static nested scopes buy us all that
> > > > much. You can do the same now, by using keyword arguments which
> > > > isn't all that nice, but works great and makes the scope clearly
> > > > visible.
> > >
> > > Yes.  It's a hack that gets employed over and over.  And it has
> > > certain problems.  We added 'import as' to get rid of a common
> > > practice that was perceived unclean.  Maybe we should support nested
> > > scopes to get rid of another unclean common practice?
> >
> > I think the common practice mainly comes from the fact,
> > that by making globals locals which can benefit from LOAD_FAST
> > you get a noticable performance boost.
> >
> > So the "right" solution to these weird looking hacks would
> > be to come up with a smart way by which the Python compiler
> > itself can do the localizing.
> 
> Can you elaborate?  I dun't understand what you are proposing here.

See my other post... I would like to have the compiler do the
localization for me in case it sees a global which has been
"defined" static.

> > Nested scopes won't help eliminating the current keyword
> > practice.
> 
> Why not?  I'd say that
> 
>     def create_adder(n):
>         def adder(x, n=n): return x+n
>         return adder
> 
> is a hack and that nested scopes can fix this by allowing you to write
> 
>     def create_adder(n):
>         def adder(x): return x+n
>         return adder
> 
> like one would expect.  (Don't tell me that it isn't a FAQ why this
> doesn't work!)

I know... still, I consider function definitions within a
function bad style. Maybe just me, though ;-)
 
> > > I'm not saying that we definitely should add this to 2.1 (there's
> > > enough on our plate already) but we should at least consider it, and
> > > now that we have cycle GC, the major argument against it (that it
> > > causes cycles) is gone...
> >
> > Hmm, so far the only argument for changing Python lookups
> > was to allow writing lambdas without keyword hacks. Does this
> > really warrant breaking code ?
> >
> > What other advantages would statically nested scopes have ?
> 
> Doing what's proper.  Nested scopes are not a bad idea.  They weren't
> implemented because they were hard to get right (perhaps impossible
> without creating cycles), and I was okay with that because I didn't
> like them; but I've been convinced that examples like the second
> create_adder() above should reall work.  Just like float+int works
> (in Python 0.1, this was a type-error -- you had to case the int arg
> to a float to get a float result).

Ok, but how does nested scoping mix with class definitions
and global lookups then ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Fri Nov  3 13:50:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 13:50:17 +0100
Subject: [Python-Dev] statically nested scopes
References: <14849.37426.860007.989619@bitdiddle.concentric.net>
			<LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com>
			<14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net> <3A028D25.B6A227F2@lemburg.com>
Message-ID: <3A02B489.89EF108C@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Jeremy Hylton wrote:
> >
> > Looks like we need to rehash this thread at least enough to determine
> > who is responsible for causing us to rehash it.
> >
> > MAL said it would break code.  I asked how.  Skip and Tim obliged with
> > examples.  I said their examples exhibited bad style; neither of them
> > claimed they were good style.
> >
> > In the end, I observed that while it could break code in theory, I
> > doubted it really would break much code.  Furthermore, I believe that
> > the code it will break is already obscure so we needn't worry about
> > it.
> 
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions (which is bad style programming, IMHO too).
> 
> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.
> 
> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

[Moshe pointed out to me in private mail that the above would
continue to work as it does now due to a difference being made
between class and function scoping categories]

More questions:

How are you going to explain the different
scoping categories to a newbie ? What if you define a class 
within a method ? How can you explicitely attach a dynamically
defined class to a certain scope ?

More problems (?!):

Nested scopes will introduce cycles in all frame objects.
This means that with GC turned off, frame objects will live
forever -- Python will eat up memory at a very fast pace.

BTW, Python's GC only works for a few builtin types (frames
are not among the supported types): what if a frame finds its
way into a user defined type ? Will GC still be able to cleanup
the cycles ?

Perhaps I'm just being silly, but I still don't see the
benefits of breaking todays easy-to-grasp three level
scoping rules... 

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Fri Nov  3 14:05:08 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:05:08 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 12:18:26 +0200."
             <Pine.GSO.4.10.10011031216280.8123-100000@sundial> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> 
Message-ID: <200011031305.IAA22103@cj20424-a.reston1.va.home.com>

> On Thu, 2 Nov 2000, Guido van Rossum wrote:
> 
> > I think Python 3000 ought to use totally static scoping.  That will
> > make it possible to do optimize code using built-in names!

[Moshe]
> Isn't that another way of saying you want the builtin names to be
> part of the language definition? Part of today's method advantages
> is that new builtins can be added without any problems.

The built-in names have always been part of the language definition in
my mind.  The way they are implemented doesn't reflect this, but
that's just an implementation detail.  How would you like it if
something claimed to be Python but didn't support len()?  Or map()?

That doesn't mean you can't add new built-ins, and I don't think that
the new implementation will prevent that -- but it *will* assume that
you don't mess with the definitions of the existing built-ins.

Of course you still will be able to define functions whose name
overrides a built-in -- in that case the compiler can see that you're
doing that (because it knows the scope rules and can see what you are
doing).  But you won't be able to confuse someone else's module by
secretly sticking a replacement built-in into their module's __dict__.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Fri Nov  3 14:12:11 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 08:12:11 -0500
Subject: [Python-Dev] statically nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 11:02:13 +0100."
             <3A028D25.B6A227F2@lemburg.com> 
References: <14849.37426.860007.989619@bitdiddle.concentric.net> <LNBBLJKPBEHFEDALKOLCGEENHOAA.tim_one@email.msn.com> <14849.55887.283432.505910@beluga.mojam.com> <14849.52672.943439.978785@bitdiddle.concentric.net>  
            <3A028D25.B6A227F2@lemburg.com> 
Message-ID: <200011031312.IAA22141@cj20424-a.reston1.va.home.com>

[MAL]
> That's just what I was trying to say all along: statically
> nested scopes don't buy you anything except maybe for lambdas
> and nested functions

That's a tautology -- that's what nested scopes are FOR!

> (which is bad style programming, IMHO too).

Not always.  The keyword argument hack is so common that it must serve
a purpose, and that's what we're trying to fix -- for lambda's *and*
nested functions (which are semantically equivalent anyway).

> The only true argument for changing scoping I see is that
> of gained purity in language design... without much practical
> use.

But there is practical use: get rid of the unintuitive, unobvious and
fragile keyword argument hack.

> Other issues that need sorting out:
> 
> x = 2
> class C:
>   x = 1
>   C = 'some string'
>   def a(self):
>      print x
>   def b(self):
>      global x
>      x = 3
> 
> class D(C):
>   C = 'some string'
>   def a(self):
>      C.a(self)
>      print C
> 
> o = C()
> o.a()
> o.b()
> o.a()
> 
> o = D()
> o.a()
> 
> What would the output look like under your proposal ?

This is a good point!  If we considered the class as a nested scope
here, I think it might break too much code, plus it would allow a new
coding style where you could reference class variables without a self
or <classname> prefix.  I don't like that prospect, so I'm in favor
for ruling this out.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Fri Nov  3 15:16:20 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 09:16:20 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
Message-ID: <14850.51380.7327.977719@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> The built-in names have always been part of the language
    GvR> definition in my mind.  The way they are implemented doesn't
    GvR> reflect this, but that's just an implementation detail.  How
    GvR> would you like it if something claimed to be Python but
    GvR> didn't support len()?  Or map()?

    GvR> That doesn't mean you can't add new built-ins, and I don't
    GvR> think that the new implementation will prevent that -- but it
    GvR> *will* assume that you don't mess with the definitions of the
    GvR> existing built-ins.

    GvR> Of course you still will be able to define functions whose
    GvR> name overrides a built-in -- in that case the compiler can
    GvR> see that you're doing that (because it knows the scope rules
    GvR> and can see what you are doing).  But you won't be able to
    GvR> confuse someone else's module by secretly sticking a
    GvR> replacement built-in into their module's __dict__.

I'm a little confused.  I've occasionally done the following within an
application:

----------driver.py
# need to override built-in open() to do extra debugging
def debuggin_open(filename, mode, bufsize):
    # ...
if EXTRA_DEBUGGING:
    import __builtin__.__dict__['open'] = debugging_open
-------------------- snip snip --------------------

Would this be illegal?  Would other modules in my application (even if
imported from the standard library!) automatically get
debugging_open() for open() like they do now?

-Barry



From gward at mems-exchange.org  Fri Nov  3 15:33:03 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 09:33:03 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sat, Oct 28, 2000 at 12:51:20PM +0200
References: <200010281051.MAA01290@loewis.home.cs.tu-berlin.de>
Message-ID: <20001103093303.A2683@ludwig.cnri.reston.va.us>

On 28 October 2000, Martin v. Loewis said:
> > I don't have access to a Solaris machine, so I can't do anything to
> > help these users.
> 
> The patch in 117606 looks right to me: gcc on Solaris (and on any
> other platform) needs -shared to build shared library; configure
> currently passes -G. I haven't actually tried the patch, since it is a
> pain to extract it from the SF bug report page. What happens is that
> gcc passes -G to the linker, which correctly produces a shared
> library.  However, gcc also links crt1/crti into the library, which
> causes the reference to main.

Well, I do have access to a Solaris machine -- I try not to use it if I
don't have to, and about the only purpose it has these days is
occasionally building Python to make sure it still works.

Incidentally, I'm the one who changed "ld -G" to "$(CC) -G" -- see
revision 1.124 of configure.in:

  revision 1.124
  date: 2000/05/26 12:22:54;  author: gward;  state: Exp;  lines: +6 -2
  When building on Solaris and the compiler is GCC, use '$(CC) -G' to
  create shared extensions rather than 'ld -G'.  This ensures that shared
  extensions link against libgcc.a, in case there are any functions in the
  GCC runtime not already in the Python core.

I think the checkin message there is fairly clear; the context was in
using Robin Dunn's extension for BSDDB 2.x, which does some 64-bit
arithmetic deep down inside.  Turned out that GCC compiled a 64-bit
divide into a function call, and that function is in GCC's own runtime
library.  Using "ld -G" -- that's Sun's linker, which knows nothing
about GCC's runtime library -- the function in question wasn't
available, so loading the extension failed.  I assume that if *Python*
did a 64-bit divide somewhere in *its* guts, that function would have
been available (linked into the python binary), which is why this
problem doesn't come up very often -- Python probably does use most of
GCC's runtime.  ;-)

Anyways, I thought the patch in bug #117606 looked fine, so I tried it
out.  Not so good; here's what happens when I try to build
arraymodule.so (the first extension, alphabetically) with "gcc -shared":

  Text relocation remains                         referenced
      against symbol                  offset      in file
  _PyObject_NewVar                    0x654       arraymodule.o
  <unknown>                           0x26cc      arraymodule.o
  <unknown>                           0x26c8      arraymodule.o
  [...many many <unknown> symbols...]
  PyErr_Occurred                      0x1274      arraymodule.o
  PyErr_Occurred                      0x4a0       arraymodule.o
  PyErr_Occurred                      0x22d8      arraymodule.o
  PyErr_Occurred                      0x115c      arraymodule.o
  PyErr_Occurred                      0x368       arraymodule.o
  PyErr_Occurred                      0x1074      arraymodule.o
  PyErr_Occurred                      0x1f50      arraymodule.o
  PyInt_FromLong                      0x3f4       arraymodule.o
  [...]
  _Py_NoneStruct                      0x19d4      arraymodule.o
  Py_InitModule4                      0x26b8      arraymodule.o
  ld: fatal: relocations remain against allocatable but non-writable sections
  collect2: ld returned 1 exit status

All told, there were 500+ relocation errors in that one extension.
About half where "Py" or "_Py" symbols; a bunch were "<unknown>", and
the rest were 'malloc', 'memcpy', 'sprintf', and other standard library
functions.  So apparently "-shared" tells GCC to forget everything it
knows about linking C code and be as stupid as it can.  Hmmm.

I tried adding "../libpython2.0.a", and "-L.. -lpython2.0", and instead
got 20,000 relocation errors, since of course libpython2.0.a needs a lot
more symbols than arraymodule.o.  Hmmm.

I have no idea what's going on here.  I've updated the bug report, and I
am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless, of
course, there are other linker options that make it work right...

        Greg



From gmcm at hypernet.com  Fri Nov  3 15:48:29 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 09:48:29 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14850.51380.7327.977719@anthem.concentric.net>
Message-ID: <3A0289ED.6247.EB4DD0E@localhost>

> 
> >>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

>     GvR> Of course you still will be able to define functions whose
>     GvR> name overrides a built-in -- in that case the compiler can
>     GvR> see that you're doing that (because it knows the scope rules
>     GvR> and can see what you are doing).  But you won't be able to
>     GvR> confuse someone else's module by secretly sticking a
>     GvR> replacement built-in into their module's __dict__.

[A Craven Dog overrides builtin open...]

> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

I have always found it very elegant & powerful that keyword 
"import" calls builtin __import__, and that well-behaved C goes 
through the same hook. In a similar vein, I have for quite 
awhile wanted to experiment with mountable virtual file 
systems so that I can "mount" a (for example) MetaKit 
database at /cheesewhiz and other modules in my app, when 
navigating into /cheesewhiz will, unbeknownst to them, be 
reading from the MetaKit DB (these ideas leaked from tcl-land 
by Jean-Claude).

I most definitely appreciate that these facilities are dangerous 
and this type of stuff tends to be abused gratuitously (eg, 
most import hacks), but disallowing them might be considered 
a gratuitous limitation (heck - Barry knows how to bypass the 
governor on every cheezewhizzer ever manufactured).

- Gordon



From guido at python.org  Fri Nov  3 16:36:22 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 10:36:22 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 09:16:20 EST."
             <14850.51380.7327.977719@anthem.concentric.net> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com>  
            <14850.51380.7327.977719@anthem.concentric.net> 
Message-ID: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>

[Barry]
> I'm a little confused.  I've occasionally done the following within an
> application:
> 
> ----------driver.py
> # need to override built-in open() to do extra debugging
> def debuggin_open(filename, mode, bufsize):
>     # ...
> if EXTRA_DEBUGGING:
>     import __builtin__.__dict__['open'] = debugging_open
> -------------------- snip snip --------------------
> 
> Would this be illegal?  Would other modules in my application (even if
> imported from the standard library!) automatically get
> debugging_open() for open() like they do now?

That's up for discussion.  Note that the open() function is special in
this respect -- I don't see you doing the same to range() or hash().

If this is deemed a useful feature (for open()), we can make a rule
about which built-ins you cannot override like this and which ones you
can.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Fri Nov  3 16:45:05 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Fri, 3 Nov 2000 10:45:05 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
	<14850.51380.7327.977719@anthem.concentric.net>
	<200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14850.56705.120643.110016@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> That's up for discussion.  Note that the open() function is
    GvR> special in this respect -- I don't see you doing the same to
    GvR> range() or hash().

Me neither, but special casing overridability seems like a fragile
hack.

    GvR> If this is deemed a useful feature (for open()), we can make
    GvR> a rule about which built-ins you cannot override like this
    GvR> and which ones you can.

Hmm, maybe we need __open__() and an open-hook? ;)

-Barry



From guido at python.org  Fri Nov  3 17:07:13 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 11:07:13 -0500
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
Message-ID: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>

At our PythonLabs group meeting last Tuesday we decided that we needed
to set up a Patches page and a FAQ for Python 2.0.  (This because we
don't see a reason yet to issue a bugfix release, but we do need to
answer some common questions and point people to some patches.)

I figured that we could do ourselves a favor by making this a set of
dynamic pages maintained wiki-style, rather than static pages
(especially since I've managed to personally become the bottleneck for
editing the static pages, until the move of python.org to a DC hosted
machine is complete :-).

So, when I saw the announcement of MoinMoin, a Wiki clone written in
Python as a single CGI script, I decided to try it -- instead of
ZWiki, which might be the obvious choice given my new employer.  This
is not because I don't like Zope or ZWiki, but because we can learn
from using different implementations of the same idea.

So, I humbly present the Python 2.0 Info Area: 

    http://www.python.org/cgi-bin/moinmoin

I've added one critical patch, three non-critical (misc) patches,
answers to two frequent bug reports to the FAQ page, and a bunch of
links to the front page.

Note that to generate patches, I use SourceForge cvsweb's diff
feature.  The URLs are ugly, but only the page editors see these, and
it saves having to store copies of the patches.

I'd like to get some feedback from the python-dev group before I link
it into the 2.0 home page.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Fri Nov  3 17:12:04 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:12:04 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 10:36:22AM -0500
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <20001103171203.G28658@xs4all.nl>

On Fri, Nov 03, 2000 at 10:36:22AM -0500, Guido van Rossum wrote:
> [Barry]

> > Would this [replacing builtins] be illegal?

> That's up for discussion.  Note that the open() function is special in
> this respect -- I don't see you doing the same to range() or hash().

Eep. I don't care much about scoping, so I'm not going to say much about
that, but I certainly do care about Python's flexibility. One of the great
things is that there are so little special cases, that (nearly) everything
is delightfully consistent. Being able to override open() but not hash() or
range() sounds directly contrary to that, at least to me. Being able to
change __builtins__ *just like any other dict* strikes me as terribly
Pythonic, though I realize this is probably contrary to Guido's view of
Python :-) It also makes for instance 'exec' and 'eval' terribly obvious.

I understand where the wish to restrict replacement and shadowing of
builtins comes from, but I'm hoping here that this is going to be
'optional', like Perl's -w and 'use strict'. Defaulting to maximum
simplicity and maximum flexibility (in that order:) but with optional
warnings (when shadowing/modifying a builtin) and optimizations (using
constants for builtins, when not modifying or shadowing them, for instance.)

Just-my-fl.0,04-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Fri Nov  3 17:31:07 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 3 Nov 2000 17:31:07 +0100
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 11:07:13AM -0500
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <20001103173107.H28658@xs4all.nl>

On Fri, Nov 03, 2000 at 11:07:13AM -0500, Guido van Rossum wrote:

> So, I humbly present the Python 2.0 Info Area: 

>     http://www.python.org/cgi-bin/moinmoin

> I've added one critical patch, three non-critical (misc) patches,

Looks good. I'd never seen a Wiki thing before, and the naming requirements
takes a bit getting used to, but I think it looks great, regardless :)

It also reminds me that we still need to fix the erroneous conclusion by
configure that BSDI has large file support, just because it has an off_t
type that is 64 bit. (It does have that, it just doesn't use that in any of
the seek/tell functions available.) Trent, you were making noises about
looking at it, when I left for ApacheCon. Did you ever get to look at it ?
If not, I'll see if I can figure it out ;P Once it's fixed, I think it
should be added to CriticalPatches, but it might not be as straightforward
as pointing to a cvsweb URL ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From loewis at informatik.hu-berlin.de  Fri Nov  3 19:51:49 2000
From: loewis at informatik.hu-berlin.de (Martin von Loewis)
Date: Fri, 3 Nov 2000 19:51:49 +0100 (MET)
Subject: [Python-Dev] build problems under Solaris
Message-ID: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>

>  Text relocation remains                         referenced
>      against symbol                  offset      in file
[...]

> I have no idea what's going on here.  I've updated the bug report,
> and I am definitely -1 on "gcc -shared" for GCC on Solaris!  Unless,
> of course, there are other linker options that make it work right...

That happens only when using the system linker (/usr/ccs/bin/ld). The
GNU linker won't complain, and the resulting executables will run
fine.

To make the system linker happy, you also need to compile the modules
with -fPIC - which, according to Sun, we should have done all the time
when building shared libraries. The linker complains about relocations
in the text segment, which shouldn't be there - shared libraries
should be position independent.

Alternatively, you can use 'gcc -shared -mimpure-text'; that does not
request that remaining relocations cause errors.

Regards,
Martin



From mal at lemburg.com  Fri Nov  3 20:08:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 20:08:17 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
Message-ID: <3A030D21.413B8C8A@lemburg.com>

I'm having trouble opening ZIP files created using InfoZIP's
zip utility (which uses zlib) with zipfile.py:

>>> x = z.read('README')
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "/home/lemburg/lib/zipfile.py", line 242, in read
    bytes = dc.decompress(bytes)
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Is this due to the installed zlib on my system being incompatible,
or is this a bug in zipfile.py ? I have libz version 1.1.3 and
zip version 2.2.

Also, I wonder why zipfile forces the mode flag to be 'r',
'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
to the mode flag instead ?!

The ZipFile is also missing some kind of method which
extracts files in the ZIP archive to a file-like object. This
would be very useful for extracting large files from a ZIP
archive without having to first read in the whole file into
memory.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From skip at mojam.com  Fri Nov  3 22:04:44 2000
From: skip at mojam.com (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:04:44 -0600 (CST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011031536.KAA22620@cj20424-a.reston1.va.home.com>
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
	<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
	<14850.51380.7327.977719@anthem.concentric.net>
	<200011031536.KAA22620@cj20424-a.reston1.va.home.com>
Message-ID: <14851.10348.715861.488845@beluga.mojam.com>

    Guido> If this is deemed a useful feature (for open()), we can make a
    Guido> rule about which built-ins you cannot override like this and
    Guido> which ones you can.

I thought we were all adults...

For Py3k I think it should be sufficient to define the semantics of the
builtin functions so that if people want to override them they can, but that
overriding them in incompatible ways is likely to create some problems.
(They might have to run with a "no optimize" flag to keep the compiler from
assuming semantics, for instance.)  I see no particular reason to remove the
current behavior unless there are clear instances where something important
is not going to work properly.

Modifying builtins seems to me to be akin to linking a C program with a
different version of malloc.  As long as the semantics of the new functions
remain the same as the definition, everyone's happy.  You can have malloc
leave a logfile behind or keep histograms of allocation sizes.  If someone
links in a malloc library that only returns a pointer to a region that's
only half the requested size though, you're likely to run into problems.

Skip




From skip at mojam.com  Fri Nov  3 22:17:12 2000
From: skip at mojam.com (Skip Montanaro)
Date: Fri, 3 Nov 2000 15:17:12 -0600 (CST)
Subject: [Python-Dev] New Wiki-based Python 2.0 Info Area
In-Reply-To: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com>
Message-ID: <14851.11096.337480.434853@beluga.mojam.com>

    Guido> I'd like to get some feedback from the python-dev group before I
    Guido> link it into the 2.0 home page.

Looks good to me.

Skip



From mal at lemburg.com  Fri Nov  3 21:14:17 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:14:17 +0100
Subject: [Python-Dev] Re: Dynamic nested scopes
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial>
		<200011031305.IAA22103@cj20424-a.reston1.va.home.com>
		<14850.51380.7327.977719@anthem.concentric.net>
		<200011031536.KAA22620@cj20424-a.reston1.va.home.com> <14851.10348.715861.488845@beluga.mojam.com>
Message-ID: <3A031C99.7F7DD8E2@lemburg.com>

[Making builtins static...]

What about the idea to have the compiler make the decision
whether a global symbol may be considered static based on a 
dictionary ?

In non-optimizing mode this dictionary would be empty, with -O
it would include all builtins which should never be overloaded
and with -OO even ones which can be overloaded such as open()
in addition to some standard modules which are known to only
contain static symbols.

Perhaps this needs some additional help of the "define" statement
we planned as dynamic compiler interface ?! ...

define static_symbols = *            # all globals in this module
define static_symbols = func1, func2 # only these two
etc.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Fri Nov  3 21:31:51 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 15:31:51 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:04:44 CST."
             <14851.10348.715861.488845@beluga.mojam.com> 
References: <Pine.GSO.4.10.10011031216280.8123-100000@sundial> <200011031305.IAA22103@cj20424-a.reston1.va.home.com> <14850.51380.7327.977719@anthem.concentric.net> <200011031536.KAA22620@cj20424-a.reston1.va.home.com>  
            <14851.10348.715861.488845@beluga.mojam.com> 
Message-ID: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>

>     Guido> If this is deemed a useful feature (for open()), we can make a
>     Guido> rule about which built-ins you cannot override like this and
>     Guido> which ones you can.

[Skip]
> I thought we were all adults...

And consenting as well... :-)

> For Py3k I think it should be sufficient to define the semantics of the
> builtin functions so that if people want to override them they can, but that
> overriding them in incompatible ways is likely to create some problems.
> (They might have to run with a "no optimize" flag to keep the compiler from
> assuming semantics, for instance.)  I see no particular reason to remove the
> current behavior unless there are clear instances where something important
> is not going to work properly.
> 
> Modifying builtins seems to me to be akin to linking a C program with a
> different version of malloc.  As long as the semantics of the new functions
> remain the same as the definition, everyone's happy.  You can have malloc
> leave a logfile behind or keep histograms of allocation sizes.  If someone
> links in a malloc library that only returns a pointer to a region that's
> only half the requested size though, you're likely to run into problems.

Actually, the C standard specifically says you are *not* allowed to
override standard library functions like malloc().

I'm thinking of the example of the rules in Fortran for intrinsic
functions (Fortran's name for built-ins).  Based on what Tim has told
me, I believe that Fortran by default assumes that you're not doing
anything funky with intrinsics (like sin, cos, tan) it can use a
shortcut, e.g. inline them.  But there are also real functions by
these names in the Fortran standard library, and you can call those by
declaring e.g. "external function sin".  (There may also be an
explicit way to say that you're happy with the intrinsic one.)  I
believe that when you use the external variant, they may be overridden
by the user.

I'm thinking of something similar here for Python.  If the bytecode
compiler knows that the builtins are vanilla, it can generate better
(== more efficient) code for e.g.

   for i in range(10):
       ...

Ditto for expressions like len(x) -- the len() operation is typically
so fast that the cost is dominated by the two dict lookup operations
(first in globals(), then in __builtins__).

Why am I interested in this?  People interested in speed routinely use
hacks that copy a built-in function into a local variable so that they
don't have dictlookups in their inner loop; it's really silly to have
to do this, and if certain built-ins were recognized by the compiler
it wouldn't be necessary.  There are other cases where this is not so
easy without much more analysis; but the built-ins (to me) seem
low-hanging fruit.  (Search the archives for that term, I've used it
before in this context.)

I assume that it's *really* unlikely that there are people patching
the __builtin__ module to replace the functions that are good inline
candidates (range, len, id, hash and so on).  So I'm interesting in
complicating the rules here.  I'd be happy to make an explicit list of
those builtins that should not be messed with, as part of the language
definition.  Program that *do* mess with these have undefined
semantics.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Fri Nov  3 21:50:48 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Nov 2000 21:50:48 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test/output
 test_unicodedata,1.3,1.4
References: <200011032024.MAA26242@slayer.i.sourceforge.net>
Message-ID: <3A032528.D743FC0A@lemburg.com>

Fredrik Lundh wrote:
> 
> Update of /cvsroot/python/python/dist/src/Lib/test/output
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv25791/lib/test/output
> 
> Modified Files:
>         test_unicodedata
> Log Message:
> 
> Added 38,642 missing characters to the Unicode database (first-last
> ranges) -- but thanks to the 2.0 compression scheme, this doesn't add
> a single byte to the resulting binaries (!)
> 
> Closes bug #117524

Cool :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gmcm at hypernet.com  Fri Nov  3 21:56:56 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Fri, 3 Nov 2000 15:56:56 -0500
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 03 Nov 2000 15:04:44 CST."             <14851.10348.715861.488845@beluga.mojam.com> 
Message-ID: <3A02E048.17812.10062D77@localhost>

> >     Guido> If this is deemed a useful feature (for open()), we can make a
> >     Guido> rule about which built-ins you cannot override like this and
> >     Guido> which ones you can.

I think I would be satisfied with just those builtins which involve 
interfaces to the external world. Where Java allows such 
deviance, they tend to provide an API whereby you can supply 
or register a factory to override or extend the default behavior.

In principle this seems less hackish than stomping on builtins; 
in practice I doubt it makes much difference ;-).
 
> [Skip]
> > I thought we were all adults...

Snort. One of my kids will be voting on Tuesday, but I *still* 
don't know what I want to be when I grow up.

- Gordon



From trentm at ActiveState.com  Fri Nov  3 21:51:41 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 3 Nov 2000 12:51:41 -0800
Subject: 64-bit stuff on BSDI (was: Re: [Python-Dev] New Wiki-based Python 2.0 Info Area)
In-Reply-To: <20001103173107.H28658@xs4all.nl>; from thomas@xs4all.net on Fri, Nov 03, 2000 at 05:31:07PM +0100
References: <200011031607.LAA22863@cj20424-a.reston1.va.home.com> <20001103173107.H28658@xs4all.nl>
Message-ID: <20001103125141.I20329@ActiveState.com>

On Fri, Nov 03, 2000 at 05:31:07PM +0100, Thomas Wouters wrote:
> It also reminds me that we still need to fix the erroneous conclusion by
> configure that BSDI has large file support, just because it has an off_t
> type that is 64 bit. (It does have that, it just doesn't use that in any of
> the seek/tell functions available.) Trent, you were making noises about
> looking at it, when I left for ApacheCon. Did you ever get to look at it ?

No. I was in Washington D.C. for the past week at SD 2000 (and getting
occassionally verbally abused by Tim about 64-bit stuff).

> If not, I'll see if I can figure it out ;P 

I can answer questions about fixing it but you can test it, so better for you
to submit the patch.

> Once it's fixed, I think it
> should be added to CriticalPatches, but it might not be as straightforward
> as pointing to a cvsweb URL ;P
>

Sure. Yes, it will probably be more than one file (configure.in, configure,
and fileobject.c).  Whatever, I don't think it should be a problem to put up
the actually checkin patch rather than a link.

Thanks!
Trent


-- 
Trent Mick
TrentM at ActiveState.com



From gward at mems-exchange.org  Fri Nov  3 23:45:00 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 17:45:00 -0500
Subject: [Python-Dev] build problems under Solaris
In-Reply-To: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>; from loewis@informatik.hu-berlin.de on Fri, Nov 03, 2000 at 07:51:49PM +0100
References: <200011031851.TAA10703@pandora.informatik.hu-berlin.de>
Message-ID: <20001103174500.B6683@ludwig.cnri.reston.va.us>

On 03 November 2000, Martin von Loewis said:
> That happens only when using the system linker (/usr/ccs/bin/ld). The
> GNU linker won't complain, and the resulting executables will run
> fine.

I'm not sure which linker my GCC on Solaris is using.  (Even though,
umm, I built that GCC.  Errr...)  But ISTR that the choice of linker is
made when you build GCC, so it's not really an option here.  Bad enough
to tell people that they need to reinstall Python because they need some
extension; requiring them to rebuild GCC -- !

But it's not necessary at all, because...

> To make the system linker happy, you also need to compile the modules
> with -fPIC - which, according to Sun, we should have done all the time
> when building shared libraries. The linker complains about relocations
> in the text segment, which shouldn't be there - shared libraries
> should be position independent.

...compiling everything with -fPIC is exactly what the doctor ordered.
I added "-fPIC" to OPT in the Makefile and rebuilt, and everything went
smoothly when linking the extensions with "gcc -shared".  No problems
compiling or linking, and the tests are running right now without a
hitch.  Yaayyyyh!

So here's an update of the patch: this changes LDSHARED to "$(CC)
-shared" when using GCC under Solaris, and it adds "-fPIC" to OPT when
using GCC *anywhere*.  This seems like a good thing to me when building
shared objects, but if anyone is aware of a platform where "gcc
... -fPIC" is a bad idea, speak up now!

*** configure.in	2000/11/03 08:18:36	1.177
--- configure.in	2000/11/03 22:43:50
***************
*** 308,315 ****
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes";;
  		esac
  		;;
  	*)	OPT="-O";;
--- 308,315 ----
  	case $GCC in
  	yes)
  		case $ac_cv_prog_cc_g in
! 		yes)	OPT="-g -O2 -Wall -Wstrict-prototypes -fPIC";;
! 		*)	OPT="-O2 -Wall -Wstrict-prototypes -fPIC";;
  		esac
  		;;
  	*)	OPT="-O";;
***************
*** 564,570 ****
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -G'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;
--- 564,570 ----
  	SunOS/4*) LDSHARED="ld";;
  	SunOS/5*) 
  		if test "$GCC" = "yes"
! 		then LDSHARED='$(CC) -shared'
  		else LDSHARED="ld -G";
  		fi ;;
  	hp*|HP*) LDSHARED="ld -b";;


I'll go update the bug report now.  Thanks, Martin!

        Greg



From guido at python.org  Sat Nov  4 00:01:27 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 18:01:27 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
Message-ID: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>

Moshe and Andrew (the impatient youth :-) have asked what's on our
plate for Python 2.1.


Schedule
--------

The current plan for the 2.1 release is in PEP 226:

    http://python.sourceforge.net/peps/pep-0226.html

According to that PEP, the tentative schedule is:

    16-Dec-2000: 2.1 PEPs ready for review
    01-Feb-2001: First 2.1 beta release
    16-Mar-2001: 2.1 final release

But the PEP doesn't really say much about which PEPs we're going to
work on.


Overview of the PEPs
--------------------

In my recent reworking of PEP 0 (the PEP index) I created a category
of Active PEPs -- these are PEPs that I definitely want to consider in
Python 2.1:

 I    42  pep-0042.txt  Small Feature Requests                 Hylton
 S   207  pep-0207.txt  Rich Comparisons                       Ascher
 S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
 S   217  pep-0217.txt  Display Hook for Interactive Use       Zadka
 S   222  pep-0222.txt  Web Library Enhancements               Kuchling
 I   226  pep-0226.txt  Python 2.1 Release Schedule            Hylton
 S   227  pep-0227.txt  Statically Nested Scopes               Hylton

Note that I said *consider*: I'm sure that not all feature requests
from PEP 42 (a grab-bag of stuff that may or may not be feasible to
implement) will be implemented, and I'm not sure that all the other
PEPs on the list will be (especially nested scopes still seems iffy).

Two of these (207, 208) haven't been written yet -- but I know roughly
what they will contain, so they are included in this list, and not in
the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
PEPs).  These may be reconsidered for Python 2.1 if their authors care
to follow the PEP guidelines.

There are a bunch of other PEPs that I moved to the Pie-in-the-sky
category -- these are form-complete PEPs, but they are controversial
(e.g. there are two competing matrix op PEPs); some others I think are
not important; for yet others I think it's unrealistic to expect them
to be implemented by Python 2.1.  (There are also other things I'd
like to do that fit in the pie-in-the-sky category, like breaking up
the interpreter in several replaceable pieces.)


Other issues to work on
-----------------------

These aren't PEPs yet, but I think they need to become PEPs soon --
I'd like to see work on them go into Python 2.1:

- The buffer interface needs a revamping or simplification; this was
  discussed here previously.

- A warning framework.  I've got some ideas already.

- Integer division.  If we want to play by Paul Prescod's Language
  Evolution rules (PEP 5), we better get started on the first stage.
  E.g. we could introduce a // operator in 2.1 for integer division,
  and issue a warning when / is used for integers.  Then a year later
  (i.e., March 2002!) we could change / so that it returns a floating
  point number.

- Case sensitivity???

- Class/type dichotomy???

- Weak references.  This *is* a PEP, but there's no contents yet.  We
  could also try to implement (just) weak dictionaries.

- Internationalization.  Barry knows what he wants here; I bet Martin
  von Loewis and Marc-Andre Lemburg have ideas too.

- Arbitrart attributes on functions?  This would be a generalization
  of docstrings; with the intent that you don't have to put semantics
  in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
  This could possibly lead to implementing private, protected, public
  attributes too.

- Whatever you want to work on.  If you have an idea for a PEP that
  you think should be implemented in Python 2.1, or if you want to
  revive a PEP that's currently listed in one of the "unattainable"
  categories, now's the time to make a plea!


--Guido van Rossum (home page: http://www.python.org/~guido/)



From gstein at lyra.org  Sat Nov  4 00:13:50 2000
From: gstein at lyra.org (Greg Stein)
Date: Fri, 3 Nov 2000 15:13:50 -0800
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103151350.C12266@lyra.org>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>...
> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'd like to write up a free-threading PEP, but with all of my work on
Subversion right now, I'm not sure when that writing will occur or the
coding. I'll certainly do the PEP first because it could also be used as a
development roadmap for somebody do some of the work.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From gward at mems-exchange.org  Sat Nov  4 00:17:35 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Fri, 3 Nov 2000 18:17:35 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
Message-ID: <20001103181734.A6809@ludwig.cnri.reston.va.us>

Hi all --

since "-Wall -Wstrict-prototypes" were added to OPT by default, a bunch
of warnings are showing up with GCC on Solaris.  (I haven't seen them on
Linux, so I assume this is OS-specific.)  See PR#121479 for a complete
list:
   https://sourceforge.net/bugs/?func=detailbug&bug_id=121479&group_id=5470)

Is anyone else looking into these?  Some of them are easy to fix,
eg. cast char to int when using 'isalnum()' and friends.  Some of them
are easy but unnecessary -- eg. sham variable initializations to shut up
the "might be used uninitialized" warning.  Others seem to require
#define'ing XOPEN_SOURCE or __EXTENSIONS__, which was a bit
controversial when it was done on Linux... but may be needed on Solaris
too.  I'm not sure yet; I've just started looking at it.  And I have to
run now.

Anyways, if anyone has guidelines/hints/suggestions for the best way to
fix these warnings, I'm all ears.  Also, if there are warnings we're not
going to worry about (eg. incorrect "might be used uninitialized"),
lemme know.

Thanks --

        Greg



From DavidA at ActiveState.com  Sat Nov  4 00:38:13 2000
From: DavidA at ActiveState.com (David Ascher)
Date: Fri, 3 Nov 2000 15:38:13 -0800 (Pacific Standard Time)
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>

>  S   207  pep-0207.txt  Rich Comparisons                       Ascher
>  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

> Two of these (207, 208) haven't been written yet -- but I know roughly
> what they will contain, so they are included in this list, and not in
> the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> PEPs).  These may be reconsidered for Python 2.1 if their authors care
> to follow the PEP guidelines.

I would _love_ to find someone to take over PEP ownership of these.  I
feel terrible that I haven't been able to find the time to do those right
(although I only feel moral ownership of 207, not 208, where others have
much more in-depth knowledge).

If someone wants to take over, please speak up and grab them.  I'll try to
find the time to share the information I have, would gladly give the very
early and now obsolete patches I wrote up.

--david
 




From akuchlin at mems-exchange.org  Sat Nov  4 02:31:05 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 3 Nov 2000 20:31:05 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Nov 03, 2000 at 06:01:27PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <20001103203105.A25097@kronos.cnri.reston.va.us>

On Fri, Nov 03, 2000 at 06:01:27PM -0500, Guido van Rossum wrote:
>- Whatever you want to work on.  If you have an idea for a PEP that
>  you think should be implemented in Python 2.1, or if you want to

Using Distutils to compile most of the extension modules, taking the
first step to eliminating the Setup/Makefile.pre.in scheme.  I'll
begin drafting a PEP.

--amk




From guido at python.org  Sat Nov  4 04:38:17 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 03 Nov 2000 22:38:17 -0500
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
In-Reply-To: Your message of "Fri, 03 Nov 2000 15:38:13 PST."
             <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a> 
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a> 
Message-ID: <200011040338.WAA26703@cj20424-a.reston1.va.home.com>

> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher

[David]
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).

I will take care of these myself if noone else picks them up.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Sat Nov  4 08:07:43 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Sat, 4 Nov 2000 02:07:43 -0500
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <20001103181734.A6809@ludwig.cnri.reston.va.us>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>

[Greg Ward]
> ...
> Also, if there are warnings we're not going to worry about (eg.
> incorrect "might be used uninitialized"), lemme know.

If a compiler is afraid something might be uninitialized, the code is too
clever for people to be sure it's correct at a glance too.  Note that right
before 2.0 was released, a bogus "uninitalized" msg got repaired, which
turned up a *legitimate* "uninitialized" msg that the bogus msg was covering
up.  The effort needed to fix one of these is too minor to be worth even
considering not fixing.

I'm not sure what gcc is complaining about in many of the cases; others are
quite clear (e.g., "confstr" apparently has no prototype in scope by the
time it's called in posixmodule.c, and that is indeed not good).

unix-should-be-shot-not-that-windows-shouldn't-ly y'rs  - tim





From py-dev at zadka.site.co.il  Sat Nov  4 19:00:49 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:00:49 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>

Here's a draft PEP.
Have I already mentioned how much it irks me to cater for other
editors in the PEP text itself?

PEP: Unassigned
Title: Reworking Python's Numeric Model
Version: $Revision$
Author: pep at zadka.site.co.il (Moshe Zadka)
Status: Draft
Type: Standards Track
Created: 4-Nov-2000
Post-History:

Abstract

    Today, Python's numerical model is similar to the C numeric model: 
    there are several unrelated numerical types, and when operations 
    between numerical types are requested, coercions happen. While the C 
    rational for the numerical model is that it is very similar to what
    happens on the hardware level, that rational does not apply to Python.
    So, while it is acceptable to C programmers that 2/3 == 0, it is very
    surprising to Python programmers.

Rationale

    In usability studies, one of Python hardest to learn features was
    the fact integer division returns the floor of the division. This
    makes it hard to program correctly, requiring casts to float() in
    various parts through the code. Python numerical model stems from
    C, while an easier numerical model would stem from the mathematical
    understanding of numbers.

Other Numerical Models

    Perl's numerical model is that there is one type of numbers -- floating
    point numbers. While it is consistent and superficially non-suprising,
    it tends to have subtle gotchas. One of these is that printing numbers
    is very tricky, and requires correct rounding. In Perl, there is also
    a mode where all numbers are integers. This mode also has its share of
    problems, which arise from the fact that there is not even an approximate
    way of dividing numbers and getting meaningful answers.

Suggested Interface For Python Numerical Model

    While coercion rules will remain for add-on types and classes, the built
    in type system will have exactly one Python type -- a number. There
    are several things which can be considered "number methods":

    1. isnatural()
    2. isintegral()
    3. isrational()
    4. isreal()
    5. iscomplex()

    a. isexact()

    Obviously, a number which answers m as true, also answers m+k as true.
    If "isexact()" is not true, then any answer might be wrong. (But not
    horribly wrong: it's close the truth).

    Now, there is two thing the models promises for the field operations
    (+, -, /, *): 

        If both operands satisfy isexact(), the result satisfies isexact()

        All field rules are true, except that for not-isexact() numbers,
        they might be only approximately true.      

    There is one important operation, inexact() which takes a number
    and returns an inexact number which is a good approximation.

    Several of the classical Python operations will return exact numbers
    when given inexact numbers: e.g, int().

Inexact Operations

    The functions in the "math" module will be allowed to return inexact
    results for exact values. However, they will never return a non-real
    number. The functions in the "cmath" module will return the correct
    mathematicl result.

Numerical Python Issues

    People using Numerical Python do that for high-performance
    vector operations. Therefore, NumPy should keep it's hardware
    based numeric model.

Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 10:50:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:50:10 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>
Message-ID: <3A03DBD2.7E024978@lemburg.com>

David Ascher wrote:
> 
> >  S   207  pep-0207.txt  Rich Comparisons                       Ascher
> >  S   208  pep-0208.txt  Reworking the Coercion Model           Ascher
> 
> > Two of these (207, 208) haven't been written yet -- but I know roughly
> > what they will contain, so they are included in this list, and not in
> > the lists of vaporware PEPs later in PEP 0 (Incomplete and Empty
> > PEPs).  These may be reconsidered for Python 2.1 if their authors care
> > to follow the PEP guidelines.
> 
> I would _love_ to find someone to take over PEP ownership of these.  I
> feel terrible that I haven't been able to find the time to do those right
> (although I only feel moral ownership of 207, not 208, where others have
> much more in-depth knowledge).
> 
> If someone wants to take over, please speak up and grab them.  I'll try to
> find the time to share the information I have, would gladly give the very
> early and now obsolete patches I wrote up.

I can take over the coercion PEP: I've been working
on this before (see the proposal on my Python Pages).

I would also like to know whether the PEP-0224 will be considered
for 2.1 if I update the patch to make it a little more robust
w/r to the problems mentioned in that PEP -- I'd really like
to see this in Python soon, since it makes documenting Python
programs so much easier.

Note that I won't get around to do much work on these before
January... way too busy at the moment :-/

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From py-dev at zadka.site.co.il  Sat Nov  4 18:42:48 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 19:42:48 +0200
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Fri, 03 Nov 2000 18:01:27 EST." <200011032301.SAA26346@cj20424-a.reston1.va.home.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> 
Message-ID: <E13s7Lz-0002m1-00@darjeeling.zadka.site.co.il>

> Moshe and Andrew (the impatient youth :-) have asked what's on our
> plate for Python 2.1.

Yeah, sure. They're not making enough fun of my age where I work, <wink>.

> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

I'm working on that one, and will submit a PEP draft ASAP. 
I do want to know what kind of changes are acceptable: it seems you have
no problem with just patching over the numerical model, while I think
that solving this problem without recreating a host of others needs
total reworking of the numerical model.

> - Case sensitivity???

Writing a PEP for this seems trivial, but I thought the consensus was
that this should be solved by tools, not the language.

> - Whatever you want to work on.  If you have an idea for a PEP that
>   you think should be implemented in Python 2.1, or if you want to
>   revive a PEP that's currently listed in one of the "unattainable"
>   categories, now's the time to make a plea!

I'm not sure I want to work on it, but in the past, we through around
ideas for pluggable nanny architecture. This is related to both
the case-sensitivity discussion above, and we also had lots of FAQ-stopping
nannies. (self-nanny, e.g., was designed to be a FAQ stopper)
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 10:58:49 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 10:58:49 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03DDD9.8527F7BE@lemburg.com>

Moshe Zadka wrote:
> 
> Here's a draft PEP.
> Have I already mentioned how much it irks me to cater for other
> editors in the PEP text itself?
> 
> PEP: Unassigned
> Title: Reworking Python's Numeric Model
> Version: $Revision$
> Author: pep at zadka.site.co.il (Moshe Zadka)
> Status: Draft
> Type: Standards Track
> Created: 4-Nov-2000
> Post-History:
> 
> Abstract
> 
>     Today, Python's numerical model is similar to the C numeric model:
>     there are several unrelated numerical types, and when operations
>     between numerical types are requested, coercions happen. While the C
>     rational for the numerical model is that it is very similar to what
>     happens on the hardware level, that rational does not apply to Python.
>     So, while it is acceptable to C programmers that 2/3 == 0, it is very
>     surprising to Python programmers.
> 
> Rationale
> 
>     In usability studies, one of Python hardest to learn features was
>     the fact integer division returns the floor of the division. This
>     makes it hard to program correctly, requiring casts to float() in
>     various parts through the code. Python numerical model stems from
>     C, while an easier numerical model would stem from the mathematical
>     understanding of numbers.
> 
> Other Numerical Models
> 
>     Perl's numerical model is that there is one type of numbers -- floating
>     point numbers. While it is consistent and superficially non-suprising,
>     it tends to have subtle gotchas. One of these is that printing numbers
>     is very tricky, and requires correct rounding. In Perl, there is also
>     a mode where all numbers are integers. This mode also has its share of
>     problems, which arise from the fact that there is not even an approximate
>     way of dividing numbers and getting meaningful answers.
> 
> Suggested Interface For Python Numerical Model
> 
>     While coercion rules will remain for add-on types and classes, the built
>     in type system will have exactly one Python type -- a number. 

While I like the idea of having the numeric model in Python
based on a solid class hierarchy, I don't think that this model
is implementable in Python 2.x without giving away performance.

> There
>     are several things which can be considered "number methods":
> 
>     1. isnatural()
>     2. isintegral()
>     3. isrational()
>     4. isreal()
>     5. iscomplex()

+1.

I would like to see methods on Python numbers too (after having
made some really good experiences with methods on strings ;-).
There's one problem though: how would you call these on
numeric literals ? ... 1.2.isreal() ?!

>     a. isexact()
> 
>     Obviously, a number which answers m as true, also answers m+k as true.
>     If "isexact()" is not true, then any answer might be wrong. (But not
>     horribly wrong: it's close the truth).

Not sure what you mean here: perhaps .isexact() <=> can be
represented in IEEE ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From py-dev at zadka.site.co.il  Sat Nov  4 19:19:13 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:19:13 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Sat, 04 Nov 2000 10:58:49 +0100." <3A03DDD9.8527F7BE@lemburg.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com> 
Message-ID: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>

> While I like the idea of having the numeric model in Python
> based on a solid class hierarchy, I don't think that this model
> is implementable in Python 2.x without giving away performance.

I think they are, using a similar trick to Fred's automorphing dictionaries.

> +1.
> 
> I would like to see methods on Python numbers too (after having
> made some really good experiences with methods on strings ;-).
> There's one problem though: how would you call these on
> numeric literals ? ... 1.2.isreal() ?!

Ummmm....how would you say you want to add 3 and 4, and multiply the result
by 5? 3+4*5?

No, you use parens:

(3+4)*5
(1.2).isreal()

> >     a. isexact()
> > 
> >     Obviously, a number which answers m as true, also answers m+k as true.
> >     If "isexact()" is not true, then any answer might be wrong. (But not
> >     horribly wrong: it's close the truth).
> 
> Not sure what you mean here: perhaps .isexact() <=> can be
> represented in IEEE ?

No, I meant "not represented exactly". The real meaning for that (one
that we might or might not promise) is that it's a float. It's a place 
where the numeric model takes the easy way out <wink>.

-- 
Moshe Zadka <sig at zadka.site.co.il>



From mal at lemburg.com  Sat Nov  4 11:21:52 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:21:52 +0100
Subject: [Python-Dev] Gathering Python 2.1 goals and non-goals
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
Message-ID: <3A03E340.38F78FA2@lemburg.com>

Guido van Rossum wrote:
> 
> Other issues to work on
> -----------------------
> 
> These aren't PEPs yet, but I think they need to become PEPs soon --
> I'd like to see work on them go into Python 2.1:
> 
> - The buffer interface needs a revamping or simplification; this was
>   discussed here previously.
> 
> - A warning framework.  I've got some ideas already.

This would be a *cool* thing. I have a need for such a framework
in mx.ODBC where ODBC often issues warnings. Currently these
turn out as exceptions which is not really all that useful
because some warnings can safely be ignored.
 
> - Integer division.  If we want to play by Paul Prescod's Language
>   Evolution rules (PEP 5), we better get started on the first stage.
>   E.g. we could introduce a // operator in 2.1 for integer division,
>   and issue a warning when / is used for integers.  Then a year later
>   (i.e., March 2002!) we could change / so that it returns a floating
>   point number.

+0... and then only, if there will be a tool to check Python
source code for integer divides.
 
> - Case sensitivity???

Should be left to Py3k. It could then be implemented by using
a special dictionary subclass as instance dictionary.
 
> - Class/type dichotomy???

One thing that would probably be implementable is a way to
maintain "instance" dictionaries for types (which are created
on-demand whenever an assignment is made). This would enable
extending types with new methods and attributes. "Subclassing"
could then be emulated by using new contructors which add the
new or changed methods to each created type instance, e.g.

class myclose:

    def __init__(self, object, basemethod):
        self.object = object
        self.basemethod = basemethod

    def __call__(self):
        print 'Closed file %s' % self.object
        self.basemethod()

def myfile(filename):
    f = open(filename)
    # add/override attributes
    f.newattribute = 1
    # add/override methods
    f.close = myclose(f, f.close)
    return f

Types would have to be made aware of this possibility. Python
could provide some helping APIs to make life easier for the
programmer.

> - Weak references.  This *is* a PEP, but there's no contents yet.  We
>   could also try to implement (just) weak dictionaries.

These already exist... http://www.handshake.de/~dieter/weakdict.html

mx.Proxy also has an implementation which support weak references.

BTW, are these still needed now that we have GC ?
 
> - Internationalization.  Barry knows what he wants here; I bet Martin
>   von Loewis and Marc-Andre Lemburg have ideas too.

We'd need a few more codecs, support for the Unicode compression,
normalization and collation algorithms.

> - Arbitrart attributes on functions?  This would be a generalization
>   of docstrings; with the intent that you don't have to put semantics
>   in docstrings (like SPARK and Zope).  Issue: what syntax to choose?
>   This could possibly lead to implementing private, protected, public
>   attributes too.

Perhaps the above "trick" could help with this ?!
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov  4 11:31:01 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Nov 2000 11:31:01 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <3A03E565.9D23559D@lemburg.com>

Moshe Zadka wrote:
> 
> > While I like the idea of having the numeric model in Python
> > based on a solid class hierarchy, I don't think that this model
> > is implementable in Python 2.x without giving away performance.
> 
> I think they are, using a similar trick to Fred's automorphing dictionaries.

You mean numbers "morph" to become floats, complex numbers, etc. 
on demand ? E.g. math.sqrt(-1) would return 1j ?!

> > +1.
> >
> > I would like to see methods on Python numbers too (after having
> > made some really good experiences with methods on strings ;-).
> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!
> 
> Ummmm....how would you say you want to add 3 and 4, and multiply the result
> by 5? 3+4*5?
> 
> No, you use parens:
> 
> (3+4)*5
> (1.2).isreal()

Ah. Of course :-) Cool !
 
> > >     a. isexact()
> > >
> > >     Obviously, a number which answers m as true, also answers m+k as true.
> > >     If "isexact()" is not true, then any answer might be wrong. (But not
> > >     horribly wrong: it's close the truth).
> >
> > Not sure what you mean here: perhaps .isexact() <=> can be
> > represented in IEEE ?
> 
> No, I meant "not represented exactly". The real meaning for that (one
> that we might or might not promise) is that it's a float. It's a place
> where the numeric model takes the easy way out <wink>.

Uhm, that's what I meant. I don't see much use for this though:
the whole meaning of "exact" is void w/r to floats. It should
be replaced by "accurate to n digits".

Note that there is a whole mathematical theory that tries to
deal with this problem: interval calculus. A package to support
this would probably make sense... a nice side-effect of 
interval calculus is that it allows "automatic" optimization
of functions within certain bounds. Numbers are replaced with
intervals and calculus is then done on the interval bounds.

This is just about as close as you can get to floating point
values with computer machinery ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at zadka.site.co.il  Sat Nov  4 19:52:35 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sat, 04 Nov 2000 20:52:35 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Sat, 04 Nov 2000 11:31:01 +0100." <3A03E565.9D23559D@lemburg.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <3A03E565.9D23559D@lemburg.com> 
Message-ID: <E13s8Qf-0002ok-00@darjeeling.zadka.site.co.il>

[Moshe Zadka, about efficient and honest numbers]
> I think they are, using a similar trick to Fred's automorphing dictionaries

[MAL]
> You mean numbers "morph" to become floats, complex numbers, etc. 
> on demand ? E.g. math.sqrt(-1) would return 1j ?!

math.sqrt has been dealt elsewhere in the PEP. It has been suggested
that math.foo will accept and return real numbers, and that
cmath.foo will accept and return complex numbers. If you 
want to always deal with complex numbers, put this in your site.py

import cmath
import sys
sys.modules['math']=cmath

> > No, I meant "not represented exactly". The real meaning for that (one
> > that we might or might not promise) is that it's a float. It's a place
> > where the numeric model takes the easy way out <wink>.
> 
> Uhm, that's what I meant. I don't see much use for this though:
> the whole meaning of "exact" is void w/r to floats. It should
> be replaced by "accurate to n digits".

I'm just promising that floats are inexact. I don't see a need for
"accuracy to n digits" (interval mathematics, etc.) in core Python.
This should be a new module if anyone needs it. Since inexact numbers
will only come about via external modules, you can just use:

import imath # interval math module
import sys
sys.modules['math']=imath.

I'm not repeating myself.

> This is just about as close as you can get to floating point
> values with computer machinery ;-)

I thought floats are the way to get to floating point values with computer
machinery?
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sat Nov  4 12:12:22 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 12:12:22 +0100
Subject: [Python-Dev] 2.1 tasks (Was: statically nested scopes)
Message-ID: <200011041112.MAA01135@loewis.home.cs.tu-berlin.de>

>  * Work on something CPAN-like.  This may or may not have repercussions for
>    the core; I don't know.

At a minimum, I think we need to include somethingCPANlike.py into the
core (library) to make something CPAN-like useful.

Regards,
Martin




From thomas at xs4all.net  Sat Nov  4 14:54:17 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 14:54:17 +0100
Subject: [Python-Dev] Compiler warnings on Solaris
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>; from tim_one@email.msn.com on Sat, Nov 04, 2000 at 02:07:43AM -0500
References: <20001103181734.A6809@ludwig.cnri.reston.va.us> <LNBBLJKPBEHFEDALKOLCCEJBHOAA.tim_one@email.msn.com>
Message-ID: <20001104145417.I28658@xs4all.nl>

On Sat, Nov 04, 2000 at 02:07:43AM -0500, Tim Peters wrote:
> [Greg Ward]
> > ...
> > Also, if there are warnings we're not going to worry about (eg.
> > incorrect "might be used uninitialized"), lemme know.

> If a compiler is afraid something might be uninitialized, the code is too
> clever for people to be sure it's correct at a glance too.

> I'm not sure what gcc is complaining about in many of the cases; others are
> quite clear (e.g., "confstr" apparently has no prototype in scope by the
> time it's called in posixmodule.c, and that is indeed not good).

There are a few messages that certainly should be looked at. The
'uninitialized usage' messages, for instances, might seriously be problems.
In this case, though, the "'ord' might be used uninitialized" warning isn't
a real problem. 'ord' is indeed only set when 'size == 1' is true, but it's
also only used if 'size == 1', and size isn't changed inbetween those
checks. Whether it should be fixed or not is another issue, but at least
it isn't causing problems.

The 'subscript has type char' message I'm not so sure about -- what is the
problem with those ? I assume it has something to do with char's signedness
being undefined, but I'm not sure. I also thought 'up'casting, such as in
functioncalls (function needs int, you give it char) was done automatically,
as part of the language, and thus shouldn't be a problem.

But the -Wstrict-prototypes seems to detect a lot more 'errors' in system
headerfiles than in Python. For instance, all the "function declaration
isn't a prototype" warnings in signalmodule.c and intrcheck.c seem to be
caused by the SIG_ERR, SIG_DFL and SIG_IGN #defines, which Python can do
nothing about. (those SIG_ #defines are apparently defined as function
declarations without prototypes.) I've seen the same effect on BSDI 4.0.1,
where a few system include files define or declare functions without
prototypes. We can't fix those errors, except by complaining to the OS
vendor. Maybe we should just disable -Wstrict-prototypes (but not -Wall) for
releases, to avoid confusion. (Developers should still use
-Wstrict-prototypes, to catch warnings Python *can* do something about, me
thinks.)

And the 'confstr' message, well, ewww ;) The manpage on Linux seems to
indicate you need to define either _POSIX_C_SOURCE as 2, or define
_XOPEN_SOURCE, for it to work. But I've never understood how those defines
are supposed to interact, so I'm hesitant to go there ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 15:08:43 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:08:43 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>; from py-dev@zadka.site.co.il on Sat, Nov 04, 2000 at 08:19:13PM +0200
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <20001104150842.J28658@xs4all.nl>

On Sat, Nov 04, 2000 at 08:19:13PM +0200, Moshe Zadka wrote:

> > There's one problem though: how would you call these on
> > numeric literals ? ... 1.2.isreal() ?!

> you use parens:
> (1.2).isreal()

Why ? There is exactly no problem with this example :)

>>> 1.2.isreal()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'float' object has no attribute 'isreal'

If float objects had attributes, it would already work. The real problem
isn't with floats, but with nonfloats:

>>> 1.isreal()
  File "<stdin>", line 1
    1.isreal()
           ^
SyntaxError: invalid syntax

And the limitation is just the parser, currently. Whether we want to allow
that syntax is something that will have to be figured out. And the parser
would have to be fixed (either rewritten into at least an LL(2) parser, or
the metagrammar hacked so that 'NUMBER' doesn't eat the . after a number if
it isn't followed by another number or whitespace.) (At least IMHO, this
isn't an insurmountable problem, or even a medium-sized problem. It was just
never necessary to fix it.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 15:46:00 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 15:46:00 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104150842.J28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:08:43PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl>
Message-ID: <20001104154559.L28658@xs4all.nl>

On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:

> And the parser would have to be fixed (either rewritten into at least an
> LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> after a number if it isn't followed by another number or whitespace.) (At
> least IMHO, this isn't an insurmountable problem, or even a medium-sized
> problem. It was just never necessary to fix it.)

Actually, no, it isn't easily fixable, if at all. The problem is mostly the
scientific notation:

1.e5

Even if it was parsed properly, it is definately going to confuse people.
They wouldn't be able to say, for instance, 

1.e()

;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From cgw at fnal.gov  Sat Nov  4 16:21:02 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:21:02 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
	<3A03DDD9.8527F7BE@lemburg.com>
	<E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
Message-ID: <14852.10590.837218.382614@buffalo.fnal.gov>

Moshe Zadka writes:

 > MAL > Not sure what you mean here: perhaps .isexact() <=> can be
 > MAL > represented in IEEE ?
 > 
 > No, I meant "not represented exactly". The real meaning for that (one
 > that we might or might not promise) is that it's a float. It's a place 
 > where the numeric model takes the easy way out <wink>.

Hmm, I like almost everything about your proposal.  The above point
bothers me slightly.  Are you saying (1.0).isexact() == 0?

Also, how about long integers?  Will they, under your new proposal, be
indistinguisable from regular ints?  While this has some appeal to it
it would be problematic for C extension modules.

Finally, although I'm no Schemer, the hierarchy you suggest sounds
very Schemish to me - I know they have a similar hierarchy of numeric
types with some of these same predicates to test for integrality,
rationality, reality, exactness - maybe there is something to be
learned by studying the Scheme model?





From cgw at fnal.gov  Sat Nov  4 16:25:37 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sat, 4 Nov 2000 09:25:37 -0600 (CST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>
	<3A03DDD9.8527F7BE@lemburg.com>
	<mal@lemburg.com>
	<E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>
	<20001104150842.J28658@xs4all.nl>
	<20001104154559.L28658@xs4all.nl>
Message-ID: <14852.10865.96590.740569@buffalo.fnal.gov>

Thomas Wouters writes:

 > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
 > scientific notation:
 > 
 > 1.e5

You could strongly encourage people to spell this 1.0e5



From bckfnn at worldonline.dk  Sat Nov  4 17:21:48 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 16:21:48 GMT
Subject: [Python-Dev] Three argument slices.
Message-ID: <3a04376a.28016074@smtp.worldonline.dk>

Hi,

While updating the difference page in the Jython documentation, I came
across this:

- JPython sequences support three argument slices. i.e. 
  range(3)[::-1] == [2,1,0]. 
  CPython should be fixed.

Is this actually true? Should (and will cpython) change in this respect?

regards,
finn



From py-dev at zadka.site.co.il  Sun Nov  5 01:57:12 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 02:57:12 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Charles G Waldman <cgw@fnal.gov> 
   of "Sat, 04 Nov 2000 09:21:02 CST." <14852.10590.837218.382614@buffalo.fnal.gov> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il>  <14852.10590.837218.382614@buffalo.fnal.gov> 
Message-ID: <E13sE7V-0002v1-00@darjeeling.zadka.site.co.il>

> Hmm, I like almost everything about your proposal.  The above point
> bothers me slightly.  Are you saying (1.0).isexact() == 0?

Yes. 1.0 is not an exact number. What's wrong with that?
(Consider stuff like 0.333333333*3: this shouldn't be exact!)

> Also, how about long integers?  Will they, under your new proposal, be
> indistinguisable from regular ints?

Yes.

> While this has some appeal to it
> it would be problematic for C extension modules.

I haven't mentioned anything about implementation, so I haven't dealt
with the C level at all. Currently, a Python-level API is under 
consideration. I believe I can keep current day C API almost unchanged.

> Finally, although I'm no Schemer, the hierarchy you suggest sounds
> very Schemish to me

I shamelessly stole it from Scheme, with only minor changes -- most
of them about hardening some things Scheme left for implementations
to decide.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From mwh21 at cam.ac.uk  Sat Nov  4 17:47:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 04 Nov 2000 16:47:31 +0000
Subject: [Python-Dev] Three argument slices.
In-Reply-To: bckfnn@worldonline.dk's message of "Sat, 04 Nov 2000 16:21:48 GMT"
References: <3a04376a.28016074@smtp.worldonline.dk>
Message-ID: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>

bckfnn at worldonline.dk (Finn Bock) writes:

> Hi,
> 
> While updating the difference page in the Jython documentation, I
> came across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this
> respect?

Well, there's a patch I wrote on sf to add this to CPython, but it was
too late for 2.0 and it got postponed.

Does J[P]ython allow

l = range(10)
l[::3] = range(4)

?  That's one of the odder bits of the behaviour of my patch.

awaiting-pronouncement-(or-even-consensus)-ly y'rs
m.

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From bckfnn at worldonline.dk  Sat Nov  4 19:41:59 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Sat, 04 Nov 2000 18:41:59 GMT
Subject: [Python-Dev] Three argument slices.
In-Reply-To: <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
References: <3a04376a.28016074@smtp.worldonline.dk> <m3zojfd6qk.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3a045808.36365811@smtp.worldonline.dk>

>> - JPython sequences support three argument slices. i.e. 
>>   range(3)[::-1] == [2,1,0]. 
>>   CPython should be fixed.
>> 
>> Is this actually true? Should (and will cpython) change in this
>> respect?
>
>Well, there's a patch I wrote on sf to add this to CPython, but it was
>too late for 2.0 and it got postponed.
>
>Does J[P]ython allow
>
>l = range(10)
>l[::3] = range(4)
>
>?  That's one of the odder bits of the behaviour of my patch.

No:

Jython 2.0 pre-alpha on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> l = range(10)
>>> l[::3] = range(4)
Traceback (innermost last):
  File "<console>", line 1, in ?
ValueError: step size must be 1 for setting list slice

regards,
finn



From gstein at lyra.org  Sat Nov  4 19:56:08 2000
From: gstein at lyra.org (Greg Stein)
Date: Sat, 4 Nov 2000 10:56:08 -0800
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104154559.L28658@xs4all.nl>; from thomas@xs4all.net on Sat, Nov 04, 2000 at 03:46:00PM +0100
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl>
Message-ID: <20001104105608.A10135@lyra.org>

Oh, this is just simple:

  (1.2).isreal()
  1.2 .isreal()

As Thomas said, fixing the grammar/parser would be rather difficult, so just
expect people to use parens or an extra space if they want to use it on a
constant.

[ of course, it is very silly to make *any* changes to the grammar just to
  allow people to use these on a constant; that is quite a silly "usage"
  that we don't need to pander to; the above "workarounds", if you will, are
  sufficient for the bozos who use it on a constant. ]

Cheers,
-g

On Sat, Nov 04, 2000 at 03:46:00PM +0100, Thomas Wouters wrote:
> On Sat, Nov 04, 2000 at 03:08:43PM +0100, Thomas Wouters wrote:
> 
> > And the parser would have to be fixed (either rewritten into at least an
> > LL(2) parser, or the metagrammar hacked so that 'NUMBER' doesn't eat the .
> > after a number if it isn't followed by another number or whitespace.) (At
> > least IMHO, this isn't an insurmountable problem, or even a medium-sized
> > problem. It was just never necessary to fix it.)
> 
> Actually, no, it isn't easily fixable, if at all. The problem is mostly the
> scientific notation:
> 
> 1.e5
> 
> Even if it was parsed properly, it is definately going to confuse people.
> They wouldn't be able to say, for instance, 
> 
> 1.e()
> 
> ;P
> 
> -- 
> Thomas Wouters <thomas at xs4all.net>
> 
> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/



From thomas at xs4all.net  Sat Nov  4 22:29:28 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:29:28 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <14852.10865.96590.740569@buffalo.fnal.gov>; from cgw@fnal.gov on Sat, Nov 04, 2000 at 09:25:37AM -0600
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <14852.10865.96590.740569@buffalo.fnal.gov>
Message-ID: <20001104222928.M28658@xs4all.nl>

On Sat, Nov 04, 2000 at 09:25:37AM -0600, Charles G Waldman wrote:
> Thomas Wouters writes:

>  > Actually, no, it isn't easily fixable, if at all. The problem is mostly the
>  > scientific notation:

>  > 1.e5

> You could strongly encourage people to spell this 1.0e5

Oh, sure, but that isn't going to solve anything, unless you are proposing
to break the common practice of not requiring zeros before or after decimal
points entirely, and thus breaking gobs of code. The problem is simply that
the syntax is truly ambiguous, and there is no way to tell whether the
statement

  x = 1.e5

is meant to assign 100000 (as a float) to 'x', or assign the 'e5' attribute
of the object '1' to 'x'. Not even full context-based parsing is going to
solve that. This is an edge case, and not likely to happen in real life, but
I don't think it's really worth the trouble, all in all. We'd have to
rewrite the parser into something other than a look-ahead parser to be able
to correctly parse the cases where the syntax isn't really ambiguous, such
as

x = 1.e42e

or some such, and that would still leave unparseable syntax. And all that
just to avoid forcing people to use parentheses around 'normal' integer
literals when directly following them with an attribute dereference. After
all,

 t = 1
 x = t.e5

is perfectly valid, and probably a lot more common. Who needs to run a
method on a literal anyway [don't mention the .join() war!]

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From thomas at xs4all.net  Sat Nov  4 22:32:39 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 22:32:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <20001104105608.A10135@lyra.org>; from gstein@lyra.org on Sat, Nov 04, 2000 at 10:56:08AM -0800
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <3A03DDD9.8527F7BE@lemburg.com> <mal@lemburg.com> <E13s7uM-0002ng-00@darjeeling.zadka.site.co.il> <20001104150842.J28658@xs4all.nl> <20001104154559.L28658@xs4all.nl> <20001104105608.A10135@lyra.org>
Message-ID: <20001104223239.N28658@xs4all.nl>

On Sat, Nov 04, 2000 at 10:56:08AM -0800, Greg Stein wrote:
> Oh, this is just simple:

>   (1.2).isreal()
>   1.2 .isreal()

Very observant, Greg ! I hadn't even realized that, but it makes perfect
sense if you think about it ;) '1.e5' is a single token, to the parser (a
NUMBER), so it can't have whitespace inbetween. But '1 .e5' is naturally
broken up into at least two tokens (three, in this case), and thus is
'correctly' parsed even in todays parser:

>>> 1 .e5
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'int' object has no attribute 'e5'

Y'all just need to quit thinking about floats as the problem... the problem
is ints, not floats ;)

I-shut-up-now-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From martin at loewis.home.cs.tu-berlin.de  Sat Nov  4 22:15:13 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Nov 2000 22:15:13 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>

> Yes. 1.0 is not an exact number. What's wrong with that?
> (Consider stuff like 0.333333333*3: this shouldn't be exact!)

It's not inherently wrong. It just points out an omission in the PEP:
it doesn't talk about the meaning of number literals. Since the new
model is going to follow algebraic principles more closely, I had
expected that

  0.333333333 == 333333333 / 1000000000

where, as I understand the proposal, the right-hand side is an exact
number (so 0.333333333*3 would be 999999999 / 1000000000).

One of the more-frequent questions on python-help is confusion about
floating-point numbers, e.g. why is the result of 1.99+4.99 printed
as 6.9800000000000004; users often report that as a bug.

Of course, spelling the number you had in mind as inexact(0.333333333)
is hardly acceptable, either.

Regards,
Martin



From skip at mojam.com  Sat Nov  4 23:55:44 2000
From: skip at mojam.com (Skip Montanaro)
Date: Sat, 4 Nov 2000 16:55:44 -0600 (CST)
Subject: [Python-Dev] Accessing RH 7.0 on SF?
Message-ID: <14852.37872.843069.514782@beluga.mojam.com>

Someone posted a bug about the bsddb config stuff related to RH7.0, which I
don't have direct access to.  I've seen others access different Linux
dialects on the SF site.  Can someone explain how I can access RH7.0 there?
All I really need to do at the moment is peruse the /usr/include/db3 stuff.

Thx,

Skip



From thomas at xs4all.net  Sat Nov  4 23:10:32 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 4 Nov 2000 23:10:32 +0100
Subject: [Python-Dev] Accessing RH 7.0 on SF?
In-Reply-To: <14852.37872.843069.514782@beluga.mojam.com>; from skip@mojam.com on Sat, Nov 04, 2000 at 04:55:44PM -0600
References: <14852.37872.843069.514782@beluga.mojam.com>
Message-ID: <20001104231031.P28658@xs4all.nl>

On Sat, Nov 04, 2000 at 04:55:44PM -0600, Skip Montanaro wrote:
> Someone posted a bug about the bsddb config stuff related to RH7.0, which I
> don't have direct access to.  I've seen others access different Linux
> dialects on the SF site.  Can someone explain how I can access RH7.0 there?
> All I really need to do at the moment is peruse the /usr/include/db3 stuff.

I don't think they have it, yet. RH7 is pretty new after all. I can give you
an account on my home machine if you want, though, it's RH7. Only accessible
through ssh currently, but if you want I can turn on telnet. Just send me an
email with login/pw you want ;) Or if you'd rather I peruse the db3 stuff
for you, that's fine too.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From moshez at zadka.site.co.il  Sun Nov  5 09:20:45 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 10:20:45 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sat, 04 Nov 2000 22:15:13 +0100." <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> It's not inherently wrong. It just points out an omission in the PEP:
> it doesn't talk about the meaning of number literals.

That's right -- but I did mean that floating-point literals will be
inexact.

> Since the new
> model is going to follow algebraic principles more closely, I had
> expected that
> 
>   0.333333333 == 333333333 / 1000000000
> 
> where, as I understand the proposal, the right-hand side is an exact
> number (so 0.333333333*3 would be 999999999 / 1000000000).
> 
> One of the more-frequent questions on python-help is confusion about
> floating-point numbers, e.g. why is the result of 1.99+4.99 printed
> as 6.9800000000000004; users often report that as a bug.

That's one thing my PEP is not meant to help with -- floating point
numbers will remain hard. Hopefully, people will use them less often
when they'll have rational arithmetic.

-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 09:28:06 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 09:28:06 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 10:20:45 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
Message-ID: <200011050828.JAA00672@loewis.home.cs.tu-berlin.de>

> > It's not inherently wrong. It just points out an omission in the PEP:
> > it doesn't talk about the meaning of number literals.
> 
> That's right -- but I did mean that floating-point literals will be
> inexact.

Remind you that your model has no notion of floating-point numbers -
then what the heck are floating-point literals? The numbers that you
can write in a base-10 notation are all rational numbers, and the
point doesn't really float in them...

Regards,
Martin



From py-dev at zadka.site.co.il  Sun Nov  5 18:25:33 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 19:25:33 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sun, 05 Nov 2000 09:28:06 +0100." <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>

[Martin v. Loewis]
> Remind you that your model has no notion of floating-point numbers -
> then what the heck are floating-point literals? The numbers that you
> can write in a base-10 notation are all rational numbers, and the
> point doesn't really float in them...

Well, of course they are rational numbers. The only question is whether 1.0
should be inexact or exact. While that is not specified in the PEP (which
was meant for Barry to assign me a PEP number primarily...), I think
the principle of least suprise would be to treat 1.0 as inexact.
(IOW, not to promise that (1.0/3.0)*3.0 == 1.0)
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 11:24:47 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 11:24:47 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 19:25:33 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>  <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>

> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

To long-term Python users, that would be the least surprise.

To new users, the entire notion of inexact numbers is surprising; more
so that something as simple as 1.0 is inexact.

To computer numerics fanatics, it is surprising that 1.0 is inexact,
since the common representations of floating point numbers are well
capable of representing it exactly.

Regards,
Martin



From py-dev at zadka.site.co.il  Sun Nov  5 20:04:10 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Sun, 05 Nov 2000 21:04:10 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model 
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> 
   of "Sun, 05 Nov 2000 11:24:47 +0100." <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> 
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> 
Message-ID: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>

[Moshe Zadka]
> Well, of course they are rational numbers. The only question is whether 1.0
> should be inexact or exact. While that is not specified in the PEP (which
> was meant for Barry to assign me a PEP number primarily...), I think
> the principle of least suprise would be to treat 1.0 as inexact.

[Martin v. Loewis]
> To long-term Python users, that would be the least surprise.

And to long term users of C/Perl/etc., once they map the numerical 
concepts correctly. But I hardly thing we should be arguing about this
at this stage: I'm willing to leave this as an open issue in the PEP,
if this is all you find wrong with it...

And a request to all Python-Devvers: please direct comments directly to
me, and I promise I'll summarize them all in the PEP. As soon as I
get a PEP number, I'll publish an updated version, with all objections and open
issues sumarized.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From martin at loewis.home.cs.tu-berlin.de  Sun Nov  5 12:21:14 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Nov 2000 12:21:14 +0100
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> (message from
	Moshe Zadka on Sun, 05 Nov 2000 21:04:10 +0200)
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il>  <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
Message-ID: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>

> And to long term users of C/Perl/etc., once they map the numerical 
> concepts correctly. But I hardly thing we should be arguing about this
> at this stage: I'm willing to leave this as an open issue in the PEP,
> if this is all you find wrong with it...

Sorry I didn't mention it: Overall, I like your proposal very well.
I'm missing the section on implementation strategies, though.

> And a request to all Python-Devvers: please direct comments directly
> to me, and I promise I'll summarize them all in the PEP. As soon as
> I get a PEP number, I'll publish an updated version, with all
> objections and open issues sumarized.

Yes, that is a tricky part of the PEP procedure: not commenting in the
public initially. I think PEP authors can contribute by not posting
the text of their PEP publically.

Regards,
Martin




From gstein at lyra.org  Sun Nov  5 12:40:44 2000
From: gstein at lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 03:40:44 -0800
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sun, Nov 05, 2000 at 12:21:14PM +0100
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de> <E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il> <200011050828.JAA00672@loewis.home.cs.tu-berlin.de> <E13sTXy-0004av-00@darjeeling.zadka.site.co.il> <200011051024.LAA01015@loewis.home.cs.tu-berlin.de> <E13sV5U-0004cp-00@darjeeling.zadka.site.co.il> <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <20001105034044.M10135@lyra.org>

On Sun, Nov 05, 2000 at 12:21:14PM +0100, Martin v. Loewis wrote:
>...
> > And a request to all Python-Devvers: please direct comments directly
> > to me, and I promise I'll summarize them all in the PEP. As soon as
> > I get a PEP number,

Euh... is process getting in the way of progress?

Allocate yourself a PEP number and publish the darn thing. If you don't feel
comfortable grabbing a PEP number, then just post it to the list or
something.

One of the worst things that can happen is to feel locked in by a cumbersome
process. Open Source works because people can flow at their own speed,
independent of what is happening with others. If Guido and company are too
busy to update syncmail... no problem! Thomas has time and inclination and
jumps in to fix it. Somebody too busy to revamp the headers for ANSI C? No
worries... we have a lot of volunteers just ready and able to do that.

But you throw in the gates? The locks? The process? It halts.

> > I'll publish an updated version, with all
> > objections and open issues sumarized.
> 
> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

I'm not sure what you're saying here. That a PEP author should develop the
PEP entirely in private? Only when it is fully-cooked, that it should be
published?

Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
a first draft. Revise. Revise. Revise.

Comments on PEPs are generated when people feel their are listened to. If a
PEP author develops a PEP entirely in secret, then the feedback is going to
be diminished because it is hard for people to really know if their comments
and ideas are being captured and considered. When you have that feedback
loop and the positive reinforcement, then you will engender more commentary.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From nhodgson at bigpond.net.au  Sun Nov  5 12:59:03 2000
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sun, 5 Nov 2000 22:59:03 +1100
Subject: [Python-Dev] What to choose to replace Tkinter?
Message-ID: <049d01c0471f$d7899450$8119fea9@neil>

Andrew Kuchling:

> I believe the GNOME (not GTk's, but GNOME's) canvas widget began as a
> fork of the Tk widget that was then substantially enhanced to be a
> general-purpose display engine, with antialiasing, alpha compositing,
> more attention to performance, and other fancy features.  I don't know
> if the corresponding text widget (which is Pango, www.pango.org, I
> think) is equally featureful.

   There is a port of the Tk text widget to GTK+ by Havoc Pennington which
doesn't require Pango. Its aims are a little muddled as a high quality Pango
based text widget is also under long term development.

   Pango isn't just a text widget but a layered set of capabilities allowing
development of internationalised layout and rendering (the equivalent of
Microsoft's Uniscribe). Scintilla for GTK+ will use Pango to implement
Unicode once Pango is released.

   Neil




From moshez at math.huji.ac.il  Sun Nov  5 13:12:09 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:12:09 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
Message-ID: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>

On Sun, 5 Nov 2000, Martin v. Loewis wrote:

> Yes, that is a tricky part of the PEP procedure: not commenting in the
> public initially. I think PEP authors can contribute by not posting
> the text of their PEP publically.

Perhaps, but that is part of the PEP flow: public post->number
Well, thanks a lot for the feedback.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Sun Nov  5 13:16:46 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 5 Nov 2000 14:16:46 +0200 (IST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
Message-ID: <Pine.GSO.4.10.10011051412530.28025-100000@sundial>

On Sun, 5 Nov 2000, Greg Stein wrote:

> Euh... is process getting in the way of progress?

Perhaps. Well, I won't let it get in the way more then a few more hours --
I'll allocate myself a PEP. It's  easier to get forgiveness then
permission <wink>

> I'm not sure what you're saying here. That a PEP author should develop the
> PEP entirely in private? Only when it is fully-cooked, that it should be
> published?

Far from it. Only tht the discussion tends to clutter up Python-Dev too
much, so I want to moderate it by way of private mail to me-> checking
to the PEP. I hope everyone here trust me to be honest enough not to
censor competing points.

> Bleh. A PEP should be a work-in-progress. Publish an empty version. Publish
> a first draft. Revise. Revise. Revise.

I intend to. 

> Comments on PEPs are generated when people feel their are listened to. If a
> PEP author develops a PEP entirely in secret

All future revisions will be in the Python CVS. Only security through
obscurity can keep me secret there <wink>
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Sun Nov  5 17:45:55 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 11:45:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011051411170.28025-100000@sundial>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
Message-ID: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Perhaps, but that is part of the PEP flow: public post->number
 > Well, thanks a lot for the feedback.

  Don't hesitate to publish an updated version just because Barry
hasn't assigned a number.  If you don't want to publish the full text
too often, assign a number to yourself in the PEP index and check that
in, then add the actual PEP with the right number.
  (Make sure you run "./pep2html.py -i 0 <num>" to update the index
and your PEP on python.sourceforge.net once you've made the checkins.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From thomas at xs4all.net  Sun Nov  5 20:18:51 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 20:18:51 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <200011051655.IAA13536@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Sun, Nov 05, 2000 at 08:55:27AM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
Message-ID: <20001105201851.C27208@xs4all.nl>

On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:

> Added Files:
> 	pep-0228.txt 
> Log Message:
> Added first revision of numerical model pep.

> ***** Error reading new file: (2, 'No such file or directory')
> ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461

Thank you, Moshe ;) Now to figure out why it does work for some people, or
in some cases ;P I'm guessing it's a race condition of some sort, but damned
if I know what triggers it.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From gstein at lyra.org  Sun Nov  5 22:54:39 2000
From: gstein at lyra.org (Greg Stein)
Date: Sun, 5 Nov 2000 13:54:39 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105201851.C27208@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 08:18:51PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl>
Message-ID: <20001105135439.V10135@lyra.org>

On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:
> On Sun, Nov 05, 2000 at 08:55:27AM -0800, Moshe Zadka wrote:
> 
> > Added Files:
> > 	pep-0228.txt 
> > Log Message:
> > Added first revision of numerical model pep.
> 
> > ***** Error reading new file: (2, 'No such file or directory')
> > ***** file: pep-0228.txt cwd: /tmp/cvs-serv13461
> 
> Thank you, Moshe ;) Now to figure out why it does work for some people, or
> in some cases ;P I'm guessing it's a race condition of some sort, but damned
> if I know what triggers it.

It certainly could be a race condition. Just look at blast_mail(). It forks
off the operation, and it might run before the file arrives in the
repository. (??)

Or is it possible that the script looks for pep-0228.txt rather than getting
a copy from pep-0228.txt,v ?? My script does "cvs -Qn update -p -r1.1 FILE"
for new files, and pipes that to the output.

Note that the log_accum.pl script that I use for my CVS repository (which
came from Apache, which came from BSD) does not fork. It seems to work quite
fine with added files.

[ and the point in blast_mail() about holding the CVS lock too long is
  rather silly given the horsepower of the SourceForge boxes ]

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From thomas at xs4all.net  Sun Nov  5 23:12:35 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:12:35 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105135439.V10135@lyra.org>; from gstein@lyra.org on Sun, Nov 05, 2000 at 01:54:39PM -0800
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org>
Message-ID: <20001105231235.X12776@xs4all.nl>

On Sun, Nov 05, 2000 at 01:54:39PM -0800, Greg Stein wrote:
> On Sun, Nov 05, 2000 at 08:18:51PM +0100, Thomas Wouters wrote:

> It certainly could be a race condition. Just look at blast_mail(). It forks
> off the operation, and it might run before the file arrives in the
> repository. (??)

No... that's not it.

> Or is it possible that the script looks for pep-0228.txt rather than getting
> a copy from pep-0228.txt,v ??

Almost correct :) The problem is that the loginfo process is run in the
server-specific /tmp dir, and the filename is not prefixed by the path to
the CVSROOT or some such. I guess it's pure coincidence that the file is
still there when the syncmail script arrives at the 'open()' call.

> My script does "cvs -Qn update -p -r1.1 FILE" for new files, and pipes
> that to the output.

Yes... I just wrote a small patch to syncmail which does exactly that
(actually, it uses 'newrev' rather than a hardcoded '1.1', and it uses -f
and -n but not -Q -- if we add -Q, we should add it to the diff commands as
well, and it might confuse developers that are used to reading the debug
info ;)

The main reason I'm delaying the checkin is to test it on the only CVS
repository I can play with, which is over a slow link to an american
highschool. Now if only you had mailed an hour earlier, Greg, I wouldn't
have had to go through that trouble ;)

> [ and the point in blast_mail() about holding the CVS lock too long is
>   rather silly given the horsepower of the SourceForge boxes ]

Well, syncmail was written to manage the Python CVS tree on a slow Sun (I
believe) and did an rsync-over-ssh to another machine as well. That can
definately take long ;) If we just remove the fork, the rest of syncmail
might just work, even with new files. In the mean time, I'll check in my
change. It might be the best thing to do anyway, since it shouldn't
interfere unless the file isn't there.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jeremy at alum.mit.edu  Sun Nov  5 22:08:55 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:08:55 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <20001105034044.M10135@lyra.org>
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
	<200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
	<E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
	<200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<20001105034044.M10135@lyra.org>
Message-ID: <14853.52327.355764.528782@bitdiddle.concentric.net>

[Comments from Greg and Moshe on the PEP process.]

I don't see the PEP process creating any impediments here.  There is
almost no process -- Barry assigns numbers to PEPs and Guido rules on
them.  We've got some rules about what must be in the PEP before it is
approved, but almost none about how the PEP is created.

There is nothing about the PEP process that prevents a healthy
discussion of issues, in private mail or on a mailing list (python-dev
or otherwise).  We had lots of comments on the statically nested
scopes PEP before Barry assigned it a number.  An entire PEP could be
created and discussed before it gets a number.

Someone may want to work on a PEP in privacy and wait to share it
until the entire document is complete; that's fine too, provided that
revisions are made based on feedback.

One goal we had when setting up the PEP process was to limit the
amount of repeated discussion on an issue.  It's not too uncommon for
email discussions to go in circles or to endlessly rehash the same few
issues.  We hoped that PEP authors would incorporate a discussion of
such issues in the PEP and reduce the amount of wasted bandwidth on
repetitive discussion.

Let's not waste time discussing how to create PEPs and instead
actually create them.  The clock is ticking for new features in 2.1;
the tentative deadline for new PEPs is mid-December.

Jeremy



From jeremy at alum.mit.edu  Sun Nov  5 22:14:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 16:14:36 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
Message-ID: <14853.52668.117844.28459@bitdiddle.concentric.net>

>>>>> "FLD" == Fred L Drake, <fdrake at acm.org> writes:

  FLD> Moshe Zadka writes:
  >> Perhaps, but that is part of the PEP flow: public post->number
  >> Well, thanks a lot for the feedback.

  FLD>   Don't hesitate to publish an updated version just because
  FLD>   Barry
  FLD> hasn't assigned a number.  If you don't want to publish the
  FLD> full text too often, assign a number to yourself in the PEP
  FLD> index and check that in, then add the actual PEP with the right
  FLD> number.

I thought we discussed this earlier and agreed that a little bit of
control over the process was healthy.  I would prefer to see all PEP
creation go through Barry.  We can circulate drafts in email before
that.

Jeremy



From thomas at xs4all.net  Sun Nov  5 23:29:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 5 Nov 2000 23:29:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
In-Reply-To: <20001105231235.X12776@xs4all.nl>; from thomas@xs4all.net on Sun, Nov 05, 2000 at 11:12:35PM +0100
References: <200011051655.IAA13536@slayer.i.sourceforge.net> <20001105201851.C27208@xs4all.nl> <20001105135439.V10135@lyra.org> <20001105231235.X12776@xs4all.nl>
Message-ID: <20001105232921.E27208@xs4all.nl>

On Sun, Nov 05, 2000 at 11:12:35PM +0100, Thomas Wouters wrote:

> In the mean time, I'll check in my change. It might be the best thing to
> do anyway, since it shouldn't interfere unless the file isn't there.

Since changes to files in CVSROOT go to python-checkins-admin rather than
python-checkins, here's the diff I just checked in:

Index: syncmail
===================================================================
RCS file: /cvsroot/python/CVSROOT/syncmail,v
retrieving revision 3.14
retrieving revision 3.15
diff -c -c -r3.14 -r3.15
*** syncmail    2000/11/02 21:44:32     3.14
--- syncmail    2000/11/05 22:24:29     3.15
***************
*** 85,91 ****
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             fp = open(file)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')
--- 85,95 ----
          return '***** Bogus filespec: %s' % filespec
      if oldrev == 'NONE':
          try:
!             if os.path.exists(file):
!                 fp = open(file)
!             else:
!                 update_cmd = 'cvs -fn update -r %s -p %s' % (newrev, file)
!                 fp = os.popen(update_cmd)
              lines = fp.readlines()
              fp.close()
              lines.insert(0, '--- NEW FILE ---\n')

See the manpage for 'cvs' for an explanation of the options ;) This should
fix 99.95% or so of the problem (there is still a tiny window for the file
being removed inbetween the os.path.exists and the actual open) and is
probably best even if we do remove the fork() from syncmail.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From greg at cosc.canterbury.ac.nz  Mon Nov  6 01:56:16 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 13:56:16 +1300 (NZDT)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
Message-ID: <200011060056.NAA29965@s454.cosc.canterbury.ac.nz>

"Martin v. Loewis" <martin at loewis.home.cs.tu-berlin.de>:

> To computer numerics fanatics, it is surprising that 1.0 is inexact,
> since the common representations of floating point numbers are well
> capable of representing it exactly.

I suppose in principle one could meticulously keep track
of which floating point numbers in a calculation were
exact and which weren't. But you'd lose the property that
any arithmetic operation on exact operands produces an
exact result.

Also, it would be very tedious and inefficient to have
to keep track of exactness so exactly!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 02:03:06 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:03:06 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011032031.PAA23630@cj20424-a.reston1.va.home.com>
Message-ID: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz>

Guido:

> I'd be happy to make an explicit list of
> those builtins that should not be messed with

There's a precedent for this in Scheme, which has a notion
of "integrable procedures".

As for the rest, with static scoping it will be possible to
make access to builtins just as efficient as locals, while
still allowing them to be rebound, so there's no reason why
__builtin__.__dict__.open = foo can't continue to work,
if so desired.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 02:16:27 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 14:16:27 +1300 (NZDT)
Subject: [Python-Dev] statically nested scopes
In-Reply-To: <3A02B489.89EF108C@lemburg.com>
Message-ID: <200011060116.OAA29972@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> Nested scopes will introduce cycles in all frame objects.

It doesn't have to be that way. A static link is only
needed if a function actually uses any variables from
an outer scope. In the majority of cases, it won't.

And it's possible to do even better than that. You can
separate out variables referred to in an inner scope
and store them separately from the rests of the frame,
so you only keep what's really needed alive.

> This means that with GC turned off, frame objects will live
> forever

Don't allow GC to be turned off, then! (Presumably this
feature would only be considered once GC has become a
permanent feature of Python.)

> BTW, Python's GC only works for a few builtin types (frames
> are not among the supported types)

But they could become so if necessary, surely?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 02:19:02 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:19:02 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Sat, 04 Nov 2000 20:00:49 +0200."
             <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>

So I go offline for a couple of days to entertain guests and have my
body kicked around in a dance class, and I have 25 messages discussing
Python's numeric model waiting for me...

I was hoping that Tim would chime in, but he's apparently taken the
weekend off -- very much out of character. :-)

I like the idea of a PEP to rework the numeric model.  I think that
Moshe, being a mathematician by training, will make a good editor.  I
think that this has a snowball's chance in hell to make it into Python
2.1 however -- there are many deep issues, and it's a bad idea to
experiment too much in the released version of the language.

Some of the issues are more usability issues than of a mathematical
nature.  For example, Tim has conjectured that using binary floating
point will always be a problem for the "unwashed masses" -- the only
thing they might understand is decimal floating point, not rational
numbers.  There are at least two issues here:

(1) As long as the internal representation is not the same as what is
    commonly printed, there will be surprises -- with rationals just
    as much as with floating point.  There are issues with decimal
    floating point too, but they are only the issues having to do with
    loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
    1.0) and not with loss of precision in the printing, where most of
    the "bug reports" we get seem to concentrate.

(2) Rational numbers have the unpleasant property of growing
    unboundedly during many innocent calculations, thereby using up
    exorbitant amounts of memory and slowing down the calculation --
    often mysteriously, because what is displayed is truncated.

Another issue that I might bring up is that there are no inexact
numbers (each floating point number is perfectly exact and rational)
-- there are only inexact operations.  I'm not sure what to do with
this though.

If we take its meaning literally, the isreal() function should only
return true for numbers for which isrational() is also true:
mathematically speaking, real numbers that aren't also rational don't
have an easy finite representation, since they are numbers like
sqrt(2) or pi.

I'll leave it to Tim to explain why inexact results may not be close
to the truth.

Tim may also break a lance for IEEE 754.

Finally, the PEP doesn't talk about how the numeric model can be
extended, and how other numeric types can be blended in.  E.g. I've
heard of wild floating point representations that make multiplication
and division really cheap but addition a pain, rather than the other
way around; some people may want to have long ints implemented using
the GNU mp library, and so on.  Such custom types should be supported
as well as native types -- like they are now.

--Guido van Rossum (home page: http://www.python.org/~guido/)

PS.  The "1.isreal()" problem is a non-problem.  This question is only
interesting to ask about variables.



From guido at python.org  Mon Nov  6 02:25:08 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 20:25:08 -0500
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
In-Reply-To: Your message of "Sat, 04 Nov 2000 10:50:10 +0100."
             <3A03DBD2.7E024978@lemburg.com> 
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>  
            <3A03DBD2.7E024978@lemburg.com> 
Message-ID: <200011060125.UAA03986@cj20424-a.reston1.va.home.com>

Marc-Andre:
> I can take over the coercion PEP: I've been working
> on this before (see the proposal on my Python Pages).

Thanks, excellent (although I haven't seen your proposal yet).

> I would also like to know whether the PEP-0224 will be considered
> for 2.1 if I update the patch to make it a little more robust
> w/r to the problems mentioned in that PEP -- I'd really like
> to see this in Python soon, since it makes documenting Python
> programs so much easier.

I "kinda" like the idea of having attribute docstrings (meaning it's
not of great importance to me) but there are two things I don't like
in your current proposal:

1. The syntax you propose is too ambiguous: as you say, stand-alone
   string literal are used for other purposes and could suddenly
   become attribute docstrings.

2. I don't like the access method either (__doc_<attrname>__).

> Note that I won't get around to do much work on these before
> January... way too busy at the moment :-/

That's a problem -- we really want to have the PEPs ready for review
by mid December.  This will also be a problem for the coercion PEP --
if you think you won't be able to work on it before then, I'd prefer
to find another (co-)author.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:05:20 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:05:20 -0500
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:03:06 +1300."
             <200011060103.OAA29969@s454.cosc.canterbury.ac.nz> 
References: <200011060103.OAA29969@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>

> Guido:
> > I'd be happy to make an explicit list of
> > those builtins that should not be messed with

[Greg Ewing]
> There's a precedent for this in Scheme, which has a notion
> of "integrable procedures".

Good!

> As for the rest, with static scoping it will be possible to
> make access to builtins just as efficient as locals, while
> still allowing them to be rebound, so there's no reason why
> __builtin__.__dict__.open = foo can't continue to work,
> if so desired.

I'm not sure what you mean.  With integrable procedures (whatever they
may be :-) I believe this is possible.  Without them, the lookup in
globals() can be skipped for builtins, but a local is accessed with
*zero* dict lookups -- how would you do this while still supporting
__builtin__.__dict__.open = foo?  have "hookable" dictionaries?
(Those would solve a bunch of problems, but are not under
consideration at the moment.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gmcm at hypernet.com  Mon Nov  6 03:21:47 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Sun, 5 Nov 2000 21:21:47 -0500
Subject: [Python-Dev] Stackless pages
Message-ID: <3A05CF6B.32107.2CAF777@localhost>

I have put up 6 pages of information about stackless at

http://www.mcmillan-inc.com/stackless.html

The first page attempts to give a conceptual overview of 
stackless. Notice I said "conceptual" - I make no attempt to 
be technically accurate!

Next follow 4 pages of tutorial. Mainly this is a discussion of 
implementing generators and coroutines through the 
continuation module. It includes rewrites of 2 samples that 
Tim used to demonstrate his coroutines-implemented-via-
threads madness.

Finally, the last page is about SelectDispatcher, which is kind 
of Medusa using coroutines. Included as a demonstration is a 
full FTPServer that will run on Windows. This is not just demo 
quality code - it's at the core of a couple commercial apps I'm 
doing for clients, at least one of which will make something of 
a splash in its (large, prosperous) industry. SelectDispatcher 
and friends are released under the McMillan Enterprises 4 line 
license (do what thou wilt; maintain the copyright notice; no 
warranty).

While these are not the PEPs I owe on stackless, they are 
part of the background material for those PEPs, particularly in 
demonstrating why some of us are so interested in seeing 
these facilities within core Python.

I apologize in advance to Christian for any misunderstandings 
or misinformation these pages may contain.

Enjoy!

- Gordon



From guido at python.org  Mon Nov  6 03:34:40 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:34:40 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>

[Guido]
> > - Integer division.  If we want to play by Paul Prescod's Language
> >   Evolution rules (PEP 5), we better get started on the first stage.
> >   E.g. we could introduce a // operator in 2.1 for integer division,
> >   and issue a warning when / is used for integers.  Then a year later
> >   (i.e., March 2002!) we could change / so that it returns a floating
> >   point number.

[MAL]
> +0... and then only, if there will be a tool to check Python
> source code for integer divides.

Hm.  I don't believe it's possible to write a tool to check for
integer divides by inspection of the source code only -- you have to
actually execute the code (with specific input, etc.).

However, with the right warnings framework in place (I'll post some
ideas about this under a separate subject), the Python interpreter
itself can be the perfect tool to do the checking.  Given that it's
pretty uncontroversial that 1/2 in Py3K should equal 0.5, I'd rather
get this started sooner than later.

Let me state some requirements:

- We don't want to break code in 2.1 that works in 2.0.

- It's okay to issue warnings though (my warnings proposal will limit
  the warnings to once per source line).

- In Py3K, 1/2 will yield 0.5 and users must use a different way to
  spell floor(x/y).

- Starting in 2.1, we want to issue warnings that encourage users to
  make their code Py3K-ready.

- We want (almost) all users to have converted their code to using
  1//2 instead of 1/2 by the time 2.n (the last 2.x version before
  Py3K is released) comes out, because unchanged code will silently
  change its meaning at the Py3K transition.

- Code that is Py3K-ready (in this respect) should trigger no warnings
  in Python 2.1.

Note: it is open for debate whether the result of x/y for integer (or
long integer) arguments should yield an integer (or long integer) in
those cases where the result *is* representable as such (e.g. 4/2).
It is possible that the numeric tower will render this problem moot --
but that depends on what happens to Moshe's PEP 228, and that's a much
longer story.  However, I think we can decide on the transition path
from / to // independent from the outcome of that discussion, since in
all cases it is clear that 1/2 will change in meaning.

Here's a concrete proposal (could be PEPped pretty easily):

- In Py3K, there will be two division operators:

  - x/y will always yield the mathematically expected result (possibly
    inexact, depending on how the numeric model is changed).

  - x//y will always yield the floor of the mathematical result, even
    for float arguments.  For complex arguments, this should raise an
    exception, just as int(1j) does today.

- Starting with Python 2.1, x//y will do the right thing (1//2 == 0).

- In 2.1, using x/y for ints and longs will issue a warning.

- In Py3K, there *might* be a special "backward incompatibility
  warning mode" that issues warnings when x/y is used for ints; 1/2
  will still yield 0.5 in this mode, but a warning will be issued.

This is a minimal proposal.  If Python were to grow a syntax for
pragmas, it would be nice to have a pragma saying "I want int/int to
do float division" in code that is Py3K ready; otherwise, this always
has to be spelled as float(x)/y to ensure proper working in 2.x as
well as in Py3K.

David Scherer proposed to spell this pragma as a "magical import"
(http://www.python.org/pipermail/idle-dev/2000-April/000138.html). This
is an OK idea (+0) because it can be assumed to fail in pre-2.1
installations and doesn't require new syntax.  I don't give it a +1
because it's a hack -- "import" doesn't quite convey the intention.
(Perl's "use" is better for this purpose!)

Tim didn't seem to like this idea much
(http://www.python.org/pipermail/python-dev/2000-April/010029.html).
His dislike seems based on the assumption that such annotations would
mention specific language (or interpreter) version numbers, which
could be interpreted as resisting progress (one moment 1.7 is a
forward looking version, but the next moment it is backward looking).
However if we use directives (I don't want to call them pragmas
because pragmas are supposed to be ignorable) to select specify
specific features, especially features for which there are only two
versions (the old way and the new way) then it seems okay to use such
a mechanism -- if we can agree on a syntax for directives.

Hm, reading Tim's post again it seems he's mostly objecting against
defaulting to an old version.  I have to agree with him there.
However what I'm proposing here is defaulting to the current version,
and allowing a way to select a "future version" as an alternative.

If we don't adopt directives, all we need to do (in Python 2.1) is add
a new opcode for //, keeping the opcode for / unchanged.  If we do
adopt directives, we'll need to introduce two new opcodes: one for the
new (always float) /, one for the new //, still keeping the old /
opcode with the 2.0 meaning.  The latter is what David Scherer
proposes (and what he needs for his students).  Note that it would be
okay to introduce directives in a later 2.x version -- this won't
break any previous code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:44:57 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:44:57 -0500
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>

[me]
> > - Class/type dichotomy???

[MAL]
> One thing that would probably be implementable is a way to
> maintain "instance" dictionaries for types (which are created
> on-demand whenever an assignment is made).
> 
> This would enable
> extending types with new methods and attributes. "Subclassing"
> could then be emulated by using new contructors which add the
> new or changed methods to each created type instance, e.g.
> 
> class myclose:
> 
>     def __init__(self, object, basemethod):
>         self.object = object
>         self.basemethod = basemethod
> 
>     def __call__(self):
>         print 'Closed file %s' % self.object
>         self.basemethod()
> 
> def myfile(filename):
>     f = open(filename)
>     # add/override attributes
>     f.newattribute = 1
>     # add/override methods
>     f.close = myclose(f, f.close)
>     return f
> 
> Types would have to be made aware of this possibility. Python
> could provide some helping APIs to make life easier for the
> programmer.

But this would require an extra pointer field for *all* built-in
types.  That would seriously impact the space requirements for ints
and floats!

As long as we're proposing hacks like this that don't allow smooth
subclassing yet but let you get at least some of the desired effects,
I'd rather propose to introduce some kind of metaclass that will allow
you to use a class statement to define this.  Thinking aloud:

import types
filemetaclass = metaclass(types.FileType)

class myfile(filemetaclass):

      def __init__(self, filename):
	  filemetaclass.__init__(filename)
	  self.newattribute = 1

      def close(self):
          myclose(self)
	  filemetaclass.close(self)

I'm not quite sure what kind of object "filemetaclass" here should be
or what exactly "metaclass()" should do, but it could create a new
type that has the lay-out of an existing file object, with an instance
dictionary (for newattribute) tacked on the end.  Later maybe (I'm
going to brainstorm with Jim Fulton about types and classes).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 03:48:47 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 21:48:47 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>

[me]
> > - Weak references.  This *is* a PEP, but there's no contents yet.  We
> >   could also try to implement (just) weak dictionaries.

[MAL]
> These already exist... http://www.handshake.de/~dieter/weakdict.html
> 
> mx.Proxy also has an implementation which support weak references.

Thanks.  For Fred to read...

> BTW, are these still needed now that we have GC ?

Yes, definitely.  Weak dicts are sometimes needed for situations where
a regular dict would keep objects alive forever.  E.g. we were made
aware of a "leak" in JPython that could only be fixed with weak dicts:
the Swing wrapper code has a global dict mapping widgets to callback
functions, and this keeps all widgets alive forever.  The Java GC
doesn't destroy the widgets, because they are still referenced from
the dict.  A weak dict solves this problem nicely (if it weren't that
JDK 1.1 doesn't support weak dicts).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Mon Nov  6 03:49:25 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 21:49:25 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
In-Reply-To: <14853.52668.117844.28459@bitdiddle.concentric.net>
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
	<14853.52668.117844.28459@bitdiddle.concentric.net>
Message-ID: <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I thought we discussed this earlier and agreed that a little bit of
 > control over the process was healthy.  I would prefer to see all PEP
 > creation go through Barry.  We can circulate drafts in email before

  I think I hadn't actually noticed some of that email, or perhaps
there was a conversation I've forgotten.  Fine.  I still don't see a
problem for people creating PEPs; there's always email and the files
can be pubished at alternate locations before a number has been
assigned.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Mon Nov  6 04:06:29 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Nov 2000 22:06:29 -0500 (EST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com>
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>
	<3A03E340.38F78FA2@lemburg.com>
	<200011060248.VAA04306@cj20424-a.reston1.va.home.com>
Message-ID: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > Yes, definitely.  Weak dicts are sometimes needed for situations where
 > a regular dict would keep objects alive forever.  E.g. we were made
 > aware of a "leak" in JPython that could only be fixed with weak dicts:
 > the Swing wrapper code has a global dict mapping widgets to callback

  That's a perfect example.  I've started working on some text
describing the motivation; hopefully I'll have that fleshed out and
checked in later this week.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:19:28 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:19:28 +1300 (NZDT)
Subject: [Python-Dev] Class/type dichotomy thoughts
In-Reply-To: <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <200011060319.QAA00004@s454.cosc.canterbury.ac.nz>

Guido:

> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types

> But this would require an extra pointer field for *all* built-in
> types.

Ruby has an interesting solution to this. It keeps such
"extra" instance variables in a global data structure.

The Python version of this would be to have a special global
dict which maps instances of built-in types to dicts holding
their extra instance variables.

The keys in this dict would have to be weak references,
so that they wouldn't keep the objects alive. A flag would
be set in the object header so that, when the object was
deleted, the corresponding entry in the global dict
could be cleaned up.

The overhead would then be one bit in each object, and
one extra test when deleting an object.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:20:37 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:20:37 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <200011060320.QAA00007@s454.cosc.canterbury.ac.nz>

Guido:

> Here's a concrete proposal (could be PEPped pretty easily):

Looks good to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 04:35:26 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:35:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>

Before I fall asleep let me write up my ideas about the warning
framework.


Requirements:

- A C-level API that lets C code issue a warning with a single call
  taking one or two arguments, e.g. Py_Warning(level, message).  (The
  'level' argument is an example only; I'm not sure what if any we
  need.)

- After the first time a specific warning is issued for a given source
  code location, the overhead of calling Py_Warning() should be
  minimal.

- An equivalent Python level API, e.g. sys.warning(level, message).

- Flexible control over which warnings are printed or not; there
  should be a way to set this up from within the Python program but
  also from the command line or possible using an environment
  variable.

- Control over the disposition of warnings; by default they should be
  printed to sys.stderr but an alternative disposition should be
  supported (the mechanism could be either a different file or a
  different callback function).

- By default, a warning is printed once (the first time it is issued)
  for each source line where it is issued.

- For a specific warning at a specific source code location, it should
  be possible to specify the following alternatives:

  - Turn it into an exception

  - Don't print it at all

  - Print it each time it is issued

- It should also be possible to specify these alternatives:

  - For all warnings

  - For all warnings in a specific module or file

  - For all warnings at a specific source code location

  - For a specific warning everywhere in a specific module or file

  - For a specific warning everywhere in the program

  - For all warnings at/above/below (?) a specific level, if we use
    warning levels


Possible implementation:

- Each module can has a dictionary __warnings__ in its global
  __dict__, which records the state of warnings.  It is created as an
  emprt dict if it doesn't exist when it is needed.  The keys are
  (message, linenumber) tuples (the module or file is implicit through
  the use of the module's __dict__).  The value is None if no more
  action is needed for this particular warning and location.  Some
  other values may indicate the options "always print warning" (1?)
  and "raise an exception" (-1?).

- There's a list of "filters" in the sys module
  (e.g. sys.warningfilters) that is checked whenever a warning doesn't
  have a hit in the __warnings__ dict.  Entries in the filter list are
  (file, line, message, action) tuples.  (If we decide to implement
  warning levels, these must also be represented here somewhere.)

  - The file must be None or a shell matching pattern, e.g. "*foo";
    the ".py" suffix is optional; a partial pathname may be given too.
    So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
    "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
    the filter applies regardless of the file.

  - The line must be None or an integer.  If the file is None or "*"
    (indicating all files) the line must be None and is ignored.

  - The message must be a None or a string.  If it is None, the filter
    applies to all messages.  The message string may end in "*" to
    match all messages with the given text (up to the "*").

  - The action must be one of the following strings:

    - "ignore" -- the warning is never printed

    - "always" -- the warning is always printed

    - "once" -- the warning is printed for the first occurrence
      matching the filter

    - "module" -- the warning is printed for the first occurrence in
      each module matching the filter

    - "location" -- the warning is printed for the first occurrence at
      each source code location (module + line) matching the filter

    - "exception" -- the warning is turned into an exception whenever
      it matches the filter

    Note: implementation of "once" and "module" require additional
    state per filter entry; I'm not sure if that's worth the effort.

  - When the warning system decides to print a warning, it is given to
    sys.displaywarning(file, line, message), which by default does
    something like print >>sys.stderr, file, ":", line, ":", message

  - There should be a function sys.addwarningfilter(file, line,
    message, action) that appends items to sys.warningfilters after
    some sanity checking.

  - There should be command line options to specify the most common
    filtering actions, which I expect to include at least:

    - suppress all warnings

    - suppress a particular warning message everywhere

    - suppress all warnings in a particular module

    - turn all warnings into exceptions

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Mon Nov  6 04:34:26 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 16:34:26 +1300 (NZDT)
Subject: [Python-Dev] Static scoping of builtins (Re: Dynamic nested scopes)
In-Reply-To: <200011060205.VAA04176@cj20424-a.reston1.va.home.com>
Message-ID: <200011060334.QAA00011@s454.cosc.canterbury.ac.nz>

Guido:

> the lookup in
> globals() can be skipped for builtins, but a local is accessed with
> *zero* dict lookups -- how would you do this while still supporting
> __builtin__.__dict__.open = foo?  have "hookable" dictionaries?

With fully static scoping, I envisage that all three kinds
of scope (local, module and builtin) would be implemented
in essentially the same way, i.e. as arrays indexed by
integers.

That being the case, all you need to do is arrange for the
__builtin__ module and the global scope to be one and the
same thing, and __builtin__.open = foo will work just fine
(assuming open() isn't one of the special inlinable functions).

Getting __builtin__.__dict__['open'] = foo to work as well
may require some kind of special dictionary-like object.
But then you're going to need that anyway if you want to
continue to support accessing module namespaces as if they
are dictionaries.

Whether it's worth continuing to support that in Py3k is
something that can be debated separately.

> integrable procedures (whatever they may be :-)

In the Revised^n Report, some builtin procedures are
declared to be "integrable", meaning that the compiler is
allowed to assume that they have their usual definitions
and optimise accordingly.

(This is quite important in Scheme, even more so than
in Python, when you consider that almost every operation in
Scheme, including '+', is a procedure call!)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Mon Nov  6 04:40:33 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:40:33 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Sat, 04 Nov 2000 11:21:52 +0100."
             <3A03E340.38F78FA2@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
            <3A03E340.38F78FA2@lemburg.com> 
Message-ID: <200011060340.WAA04479@cj20424-a.reston1.va.home.com>

[me]
> > - Internationalization.  Barry knows what he wants here; I bet Martin
> >   von Loewis and Marc-Andre Lemburg have ideas too.

[MAL]
> We'd need a few more codecs, support for the Unicode compression,
> normalization and collation algorithms.

Hm...  There's also the problem that there's no easy way to do Unicode
I/O.  I'd like to have a way to turn a particular file into a Unicode
output device (where the actual encoding might be UTF-8 or UTF-16 or a
local encoding), which should mean that writing Unicode objects to the
file should "do the right thing" (in particular should not try to
coerce it to an 8-bit string using the default encoding first, like
print and str() currently do) and that writing 8-bit string objects to
it should first convert them to Unicode using the default encoding
(meaning that at least ASCII strings can be written to a Unicode file
without having to specify a conversion).  I support that reading from
a "Unicode file" should always return a Unicode string object (even if
the actual characters read all happen to fall in the ASCII range).

This requires some serious changes to the current I/O mechanisms; in
particular str() needs to be fixed, or perhaps a ustr() needs to be
added that it used in certain cases.  Tricky, tricky!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 04:48:22 2000
From: guido at python.org (Guido van Rossum)
Date: Sun, 05 Nov 2000 22:48:22 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: Your message of "Sun, 05 Nov 2000 21:21:47 EST."
             <3A05CF6B.32107.2CAF777@localhost> 
References: <3A05CF6B.32107.2CAF777@localhost> 
Message-ID: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>

> I have put up 6 pages of information about stackless at
> 
> http://www.mcmillan-inc.com/stackless.html

Gordon, thanks for doing this.  I still have a review of Stackless on
my TODO list.  It takes a serious chunk of my time to do it justice,
and this continues to be a problem, but the existience of your
overview certainly helps.  I still think that the current Stackless
implementation is too complex, and that continuations aren't worth the
insanity they seem to require (or cause :-), but that microthreads and
coroutines *are* worth having and that something not completely unlike
Stackless will be one day the way to get there...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Mon Nov  6 04:55:17 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun, 5 Nov 2000 22:55:17 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
References: <3A05CF6B.32107.2CAF777@localhost>
	<200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <14854.11173.601039.883893@bitdiddle.concentric.net>

[Changed discussion list from general python-list to specific
stackless.] 

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

  >> I have put up 6 pages of information about stackless at
  >> http://www.mcmillan-inc.com/stackless.html

  GvR> Gordon, thanks for doing this.  I still have a review of
  GvR> Stackless on my TODO list.  It takes a serious chunk of my time
  GvR> to do it justice, and this continues to be a problem, but the
  GvR> existience of your overview certainly helps.  I still think
  GvR> that the current Stackless implementation is too complex, and
  GvR> that continuations aren't worth the insanity they seem to
  GvR> require (or cause :-), but that microthreads and coroutines
  GvR> *are* worth having and that something not completely unlike
  GvR> Stackless will be one day the way to get there...

I tend to agree with you, Guido.  I think we would do well to
purposefully omit continuations from the Python language.  There seems
to be little need for a facility to implement arbitrary control
structures in Python.  If Python support coroutines and microthreads,
I am not sure what else would be needed.

It would be very helpful if the PEPs on Stackless could address this
issue.  One way to address it is to ask these questions: What new
control structures do users want in Python?  How best can they be
implemented?  Are continuations necessary to implement them or are
there other options?

The sort of implementation complexity that I worry about with
Stackless is, e.g. clean interaction with the C stack.  If a Python C
API call is made that pushes a C stack frame, e.g. PyObject_Compare,
then a continuation stored before that call can no longer be invokved.
The problem is that continuations break the notion a C API call will
always return an answer; they create a situation in which the C call
that is made should never return, because control is transferred to
the continuation.  I assume Stackless raises an error in this case,
but this seems pretty messy: How do we right a language spec that
explains when an error will occur without appealing to the
language implementation?

Jeremy




From petrilli at amber.org  Mon Nov  6 05:06:35 2000
From: petrilli at amber.org (Christopher Petrilli)
Date: Sun, 5 Nov 2000 23:06:35 -0500
Subject: [Python-Dev] Weak references
In-Reply-To: <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Sun, Nov 05, 2000 at 10:06:29PM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060248.VAA04306@cj20424-a.reston1.va.home.com> <14854.8245.959258.340132@cj42289-a.reston1.va.home.com>
Message-ID: <20001105230635.A18694@trump.amber.org>

Fred L. Drake, Jr. [fdrake at acm.org] wrote:
> 
> Guido van Rossum writes:
>  > Yes, definitely.  Weak dicts are sometimes needed for situations where
>  > a regular dict would keep objects alive forever.  E.g. we were made
>  > aware of a "leak" in JPython that could only be fixed with weak dicts:
>  > the Swing wrapper code has a global dict mapping widgets to callback
> 
>   That's a perfect example.  I've started working on some text
> describing the motivation; hopefully I'll have that fleshed out and
> checked in later this week.

Another example is some of the things in Zope use back-references for
ease of traversability (or worse keep weird counts hanging around).
Alot of these are negated by ZODB's ability to break cycles, but... a
lot of data structures would be hugely better from an architecture
perspective if we had a native weak reference.  

Chris
-- 
| Christopher Petrilli
| petrilli at amber.org



From est at hyperreal.org  Mon Nov  6 05:27:52 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Sun, 5 Nov 2000 20:27:52 -0800 (PST)
Subject: [Python-Dev] Weak references
In-Reply-To: <200011060248.VAA04306@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 5, 2000 09:48:47 pm"
Message-ID: <20001106042752.25360.qmail@hyperreal.org>

Guido van Rossum discourseth:
> [MAL]
> > These already exist... http://www.handshake.de/~dieter/weakdict.html
> > 
> > mx.Proxy also has an implementation which support weak references.
> 
> Thanks.  For Fred to read...

He may want to also see my http://www.hyperreal.org/~est/python/weak
(also registered in the vaults), an implementation of weak dicts and
proxy maps that doesn't require the contained objects to be
subclassed.  It even has documentation in standard format (!)
including some words about motivation.

One simple implementation approach to make it work for all objects
might be to have:

int PyObject_PushWeakHandler(PyObject *o,
                             void (*handler)(PyObject *o, PyObject *data),
                             PyObject *data);

When an object is deallocated all its handlers would be called on the
object and the data that was registered with the handler.  I believe
this would make weak dicts a very simple extension module.  With this
approach I suspect DATA should be incref'd by the call to
PyObject_PushWeakHandler() and decref'd after the associated handler
is called.

Best,

Eric



From est at hyperreal.org  Mon Nov  6 05:39:29 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Sun, 5 Nov 2000 20:39:29 -0800 (PST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net> from Jeremy
 Hylton at "Nov 5, 2000 10:55:17 pm"
Message-ID: <20001106043929.2515.qmail@hyperreal.org>

Jeremy Hylton discourseth:
> 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

This point is particularly worrisome to me because of a common pattern
I see in my own Python development work.  I'll define a class which is
parameterized with some callbacks.  Sometimes, when profiling reveals
the need, I'll move these classes to C.  If the client of the class is
using continuations via its callbacks, it may suddenly break.  This
seems a step back in the modularity I can count on in my program
components.

..and I say this as a long-time, die-hard Schemer. :)

I definitely pine for micro-threads in some of my application domains
though.

Eric



From greg at cosc.canterbury.ac.nz  Mon Nov  6 05:50:09 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2000 17:50:09 +1300 (NZDT)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy at alum.mit.edu>:

> Are continuations necessary to implement them or are
> there other options?

I think you'll find that any implementation of microthreads
or coroutines or whatever you want to call them, that
doesn't rely on playing nonportable tricks with the C
stack, will be just as mindbending as Stackless.

> The problem is that continuations break the notion a C API call will
> always return an answer;

So do threads or coroutines. As soon as you have multiple
threads of control, you have the chance that one of them
will switch to another and never get back.

> I assume Stackless raises an error in this case,
> but this seems pretty messy

This messiness isn't the fault of Stackless itself, but of the
large amount of code which *hasn't* been converted to the
Stackless Way. If the whole of Python and everything that
it calls were made truly stackless, the problem would not
arise.

Doing so, however, would not be fun. It wouldn't be fun
for any future extension writers, either.

I can't see any way out of this. Continuations/coroutines/
microthreads are all basically the same thing underneath, and
they imply an execution model that just doesn't fit well
with C.

Maybe we need to reimplement Python in Scheme, and then
feed it through a good Scheme compiler. SPython, anyone? 

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From moshez at zadka.site.co.il  Mon Nov  6 14:52:55 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 06 Nov 2000 15:52:55 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Sun, 05 Nov 2000 20:19:02 EST." <200011060119.UAA03952@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il>  <200011060119.UAA03952@cj20424-a.reston1.va.home.com> 
Message-ID: <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>

[GvR]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

I think the solution is obvious -- stop going offline to entertain guests.

[GvR]
> I was hoping that Tim would chime in

Me too. I even tried to drag him in by mentioning 754.

[GvR]
> I like the idea of a PEP to rework the numeric model.  I think that
> Moshe, being a mathematician by training, will make a good editor.  I
> think that this has a snowball's chance in hell to make it into Python
> 2.1 however -- there are many deep issues, and it's a bad idea to
> experiment too much in the released version of the language.

The reason the PEP was written now was because you started making sounds
of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
to show a less dangerous (and less innovative) way of getting similar
usability.


[GvR]
> (1) As long as the internal representation is not the same as what is
>     commonly printed, there will be surprises -- with rationals just
>     as much as with floating point.  There are issues with decimal
>     floating point too, but they are only the issues having to do with
>     loss of precision in the calculation (e.g. 1.0 - 1e-20 yielding
>     1.0) and not with loss of precision in the printing, where most of
>     the "bug reports" we get seem to concentrate.

My PEP does not yet deal with either written or inputted representation.

[GvR]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

Those calculations, if performed with floating points, would often build
up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
to be wildly inaccurate.

[GvR]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

You'll note that my PEP does not mention floating point explicitly --
and once again I mention that my PEP does not yet deal with number
literals. All it allows (not requires) is for things like math.sqrt()
to return inexact results. Naive implementations (which we might use)
would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
course, good implementations of math.sqrt() would realize that 1 has
an exact root, but that might be too hard to do for not enough gain.

[GvR]
> If we take its meaning literally, the isreal() function should only
> return true for numbers for which isrational() is also true:

Correct -- in my current model. If you later add things like constructive
reals, that is no longer true: if I have a constructive Pi, it's not 
rational. 

[GvR]
> mathematically speaking, real numbers that aren't also rational don't
> have an easy finite representation, since they are numbers like
> sqrt(2) or pi.

But numbers don't have to have a finite (periodic really) representation
to be representable in the computer: what about infinite continued fractions,
for example?

[GvR]
> Finally, the PEP doesn't talk about how the numeric model can be
> extended

That's because its rich enough not to need it. 
Let me explain exactly what I mean: as long as all field operations between
Python numbers give honest to god Python numbers, then everything else
can be solved with Python's current model of coercions, and can be solved
well when the coercion PEP will be written.

[GvR]
> I've
> heard of wild floating point representations that make multiplication
> and division really cheap but addition a pain, rather than the other
> way around;

Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
that instead of inexact() and math.{...}. How these numbers interact
with builtin Python numbers is your responsibility -- and that's what
the coercions are for.

Same goes for gmp: as long as you're not expecting to be able to change
10000000000+10000000000 to be a gmp long rather then a Python long, then
there shouldn't be a problem.
-- 
Moshe Zadka <sig at zadka.site.co.il>



From paulp at ActiveState.com  Mon Nov  6 07:24:27 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Sun, 5 Nov 2000 22:24:27 -0800 (PST)
Subject: [Python-Dev] Warnings PEP
Message-ID: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com>

Abstract

    This PEP describes a generalized warning mechanism for Python 2.1. The
    primary purpose of this mechanism is to alert the programmer or user
    of a program to potential or likely errors which, for whatever reason,
    are not considered exception-worthy. For example, this might be done
    to keep old code working during a transitional period or to alert the
    programmer or user of a recoverable error.

Syntax

    assert >> cls, test[[[, arg], arg]...]

    "cls" may be any callable object that takes a list as a single
    argument argument list and returns an object with the required
    attributes "get_action" and "format_message"
    
       * get_action() -> "warn"|"error"|"suppress"
       * format_message() -> string

    A provided base class implements these methods in a reusable
    fashion. Warning creators are encouraged to merely subclass.

    This extended form of the assertion statement calls the assertion
    handler code in the new "assertions" module. 

    The semantics of the built-in assertion handler are defined by the
    following code. It should be exposed in a new "assertions" module.

    def handle_assertion(cls, message = ""):
        "This code is called when an assertion fails and cls is not None"

	obj = cls(message)
	action = obj.get_action()
            
        if action=="error":
            *** existing assertion code ***
        elif action=="warn":
	    sys.stderr.write(obj.format_message())
        elif action=="suppress":
            pass
        else:
            assert action in ["warn","error","suppress"]

    Even if handle_assertion is implemented in C, it should be exposed as
    assertions.handle_assertion so that it may be overriden. 

    The generic warning base class is defined below:

    class Assertion:
        def __init__(self, *args):
	    if len(args) == 1:
	       self.args = args[0]
	    else:
	        self.args = args

	def format_message(self):
            sys.stderr.write("%s: %s" %(obj.__name__, self.args))

	def get_action(self):
	    return (self.get_user_request(self.__class__) 
                                or sys.default_warning_action)

        def get_user_request(self, cls):
            if cls.__name__ in sys.errors:
                return "error"
	    elif cls.__name__ in sys.warnings:
                return "warn"
            elif cls.__name__ in sys.disabled_warnings:
                return "suppress"
	    for base in cls.__bases__:
	        rc = self.get_user_request(base)
		if rc:
		    return rc
            else:
                return None

    The Assertion class should be implemented in Python so that it can be 
    used as a base class.

    Because this code inherently implements "warning state inheritance",
    it would be rare to override any of the methods, but this is possible
    in exceptional circumstances.

Command line

    By default the special variables have the following contents:

    sys.warnings = []
    sys.errors = []
    sys.suppress = []
    sys.default_warning_action = "warn"
    
    These variables may be changed from the command line. The command line
    arguments are interpreted as described below:

    -w XXX => sys.warnings.append("XXX")
    -e XXX => sys.errors.append("XXX")
    -no-w XXX => sys.suppress.append("XXX")
    -wall => sys.default_warning_action => "warn"
    -eall => sys.default_warning_action => "error"
    -no-wall => sys.default_warning_action => "suppress"

    As per the code above, errors take precedence over warnings and
    warnings over suppressions unless a particular assertion class
    specifies otherwise.

Built-in warning objects:

    class exception_in_del(Assertion):
        "An exception was ignored in an __del__ method"

    class deprecation(Assertion):
        "This feature may be removed in a future version of Python."

    class dubious(Assertion):
        "There is a common error associated with this feature."

    These class definitions are part of the "Assertion" module. They
    should only ever be used when there exists a way for the programmer to 
    accomplish the same thing without triggering the warning. For instance
    the way to suppress the deletion exception is to trap exceptions in
    __del__ methods with a try/except block.





From tim_one at email.msn.com  Mon Nov  6 07:34:38 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 01:34:38 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011060119.UAA03952@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>

[Guido]
> So I go offline for a couple of days to entertain guests and have my
> body kicked around in a dance class, and I have 25 messages discussing
> Python's numeric model waiting for me...

The scary thing is which one of those you clearly enjoyed more <wink>.

> I was hoping that Tim would chime in, but he's apparently taken the
> weekend off -- very much out of character. :-)

Exactly in character, alas:  I was obsessed with my new cable modem
connection.  I had years of stuff to learn about firewalls in two days --
not to mention years of pornography to download in one <wink>.

Some quickies for now:

+ Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
numeric model forever.

+ Everyone has IEEE-754 fp hardware today; some people actually want to use
it; Moshe doesn't, but whatever revamping we get needs to allow others their
delusions too.

> ...
> For example, Tim has conjectured that using binary floating point will
> always be a problem for the "unwashed masses" -- the only thing they
> might understand is decimal floating point,

At first glance, yes.  Complaints traced to the "binary" part of "binary fp"
vastly outnumber complaints due to the "fp" part *and* integer division
combined, on both Python-Help and the Tutor list.  So if we want to know
what actually trips up newbies, they've been telling us for years.  Decimal
fp would silence most of those complaints; but rationals would silence them
too (provided they're *displayed* in rounded decimal fp notation (restart
"str" vs "repr" rant, and that the interactive prompt uses the wrong one,
and ditto str(container))), plus a few more (non-obvious example:

    (1/49)*49

does not equal 1 in either decimal or IEEE-754 binary double fp, but does
equal 1 using rationals).

Note that Mike Cowlishaw (REXX's dad) has been working on a scheme to merge
REXX's decimal fp with IEEE-854 (the decimal variant of IEEE-754):

    http://www2.hursley.ibm.com/decimal/

I'll talk to Jim Fulton about that, since Cowlishaw is pushing a BCD variant
and Jim was wondering about that (around the change of the year, for use-- I
presume --in Zope).

Note also IBM's enhanced BigDecimal class for Java:

    http://www2.hursley.ibm.com/decimalj/

> ...
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)
> -- there are only inexact operations.  I'm not sure what to do with
> this though.

IEEE-754 defines exactly what to do with this, for binary floats (and your
hardware has an "inexact result" flag set or not after every fp operation).
Conversion of the string "1.0" to float must not set it; conversion of "0.1"
must set it; and similarly for + - * / sqrt:  "inexact result" gets set
whenever the infinitely precise result differs from the computed result.  So
inexactness there is neither a property of types nor of numbers, but of
specific computations.  Extreme example:

    x = 1./3.   # inexact
    y = x-x     # exact result (from inexact inputs!)

I know that this version (operation-based) of inexactness can be useful.  I
see almost no earthly use for calling every number of a given type inexact.
Tagging individual numbers with an exact/inexact bit is an extremely crude
form of interval arithmetic (where the intervals are single points or
infinite).

> ...
> I'll leave it to Tim to explain why inexact results may not be close
> to the truth.

> Tim may also break a lance for IEEE 754.

Somebody else on c.l.py offered to write a 754 PEP; delighted to let them
have it.

if-you-ever-approximate-people-will-get-confused-
    but-if-you-don't-they'll-run-out-of-time-or-memory-ly y'rs  - tim





From paul at prescod.net  Mon Nov  6 08:17:52 2000
From: paul at prescod.net (Paul Prescod)
Date: Sun, 05 Nov 2000 23:17:52 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A065B20.BBD1C1E3@prescod.net>

It's just coincidence that I was working on warnings at the same time
you were.

Our proposals seem to have almost no overlap. I think mine does less,
but is also much simpler. I'm always nervous about over-engineering
rather than incremental development.

-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From pf at artcom-gmbh.de  Mon Nov  6 10:05:12 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Mon, 6 Nov 2000 10:05:12 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 5, 2000  9:34:40 pm"
Message-ID: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

[Guido]:
> David Scherer proposed to spell this pragma as a "magical import"
> (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
at module level as a backward compatible method to introduce module level
pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
I still like David Scherers proposal very much.

[David]:
> I actually implemented 1/2==0.5 in Python 1.5.2, complete with a
> module-level backward compatibility flag.  The flag made an unusual use of
> the "global" statement, which appears to be accepted at toplevel by 1.5.2
> without any effect.  Therefore a statement like "global olddivision" will be
> silently ignored by 1.5.2 and earlier, and will result in the old behavior
> under my patch.  "global" even has the right sort of sound for module-level
> options :)
> 
> An outline of what I did:
> 
> 1.  Add new opcode BINARY_FRACTION to opcode.h and dis.py
> 2.  Add new flag "int c_olddivision" to struct compiling in compile.c
> 3.  Set c_olddivision to base->c_olddivision or 0 in jcompile
> 4.  Check for "global olddivision" outside a function definition in
> com_global_stmt, and set c_olddivision=1
> 5.  Check c_olddivision in com_term, and generate either BINARY_DIVISION or
> BINARY_FRACTION
> 6.  Add PyNumber_Fraction to abstract.h, and define it in abstract.c to
> explicitly check for a pair of integers and do float division
> 7.  Add a BINARY_FRACTION case to ceval.c, which calls PyNumber_Fraction
> instead of PyNumber_Divide.

BTW:  I think the "symbol" '//' is incredible ugly and starting with
IBMs JCL years ago all languages I encountered, that used this symbol
for something, did suck in some way or another.  I would appreaciate
very much, if it could be avoided alltogether to add a symbol '//'
to Python.  '//' will look like a comment delimiter to most people today.

Using a new keyword like 'div' in the tradition of languages like 
Modula-2 looks far more attractive to me.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Mon Nov  6 10:14:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 10:14:12 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>
Message-ID: <3A067664.22D09D03@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Internationalization.  Barry knows what he wants here; I bet Martin
> > >   von Loewis and Marc-Andre Lemburg have ideas too.
> 
> [MAL]
> > We'd need a few more codecs, support for the Unicode compression,
> > normalization and collation algorithms.
> 
> Hm...  There's also the problem that there's no easy way to do Unicode
> I/O.  I'd like to have a way to turn a particular file into a Unicode
> output device (where the actual encoding might be UTF-8 or UTF-16 or a
> local encoding), which should mean that writing Unicode objects to the
> file should "do the right thing" (in particular should not try to
> coerce it to an 8-bit string using the default encoding first, like
> print and str() currently do) and that writing 8-bit string objects to
> it should first convert them to Unicode using the default encoding
> (meaning that at least ASCII strings can be written to a Unicode file
> without having to specify a conversion).  I support that reading from
> a "Unicode file" should always return a Unicode string object (even if
> the actual characters read all happen to fall in the ASCII range).
> 
> This requires some serious changes to the current I/O mechanisms; in
> particular str() needs to be fixed, or perhaps a ustr() needs to be
> added that it used in certain cases.  Tricky, tricky!

It's not all that tricky since you can write a StreamRecoder
subclass which implements this. AFAIR, I posted such an implementation
on i18n-sig.

BTW, one of my patches on SF adds unistr(). Could be that it's
time to apply it :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Mon Nov  6 10:45:21 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 6 Nov 2000 11:45:21 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011061142050.4175-100000@sundial>

On Sun, 5 Nov 2000, Guido van Rossum wrote:

>   - The file must be None or a shell matching pattern, e.g. "*foo";
>     the ".py" suffix is optional; a partial pathname may be given too.
>     So "foo/bar" matches "/usr/lib/python2.0/foo/bar.py" but also
>     "/home/guido/libp/tralala/foo/bar.py".  If the file is None or "*"
>     the filter applies regardless of the file.

How about "file" must be None or a callable, and if it's a callable,
it will be called to check whether to print? If I'll want fnmatch,
I know where to find it.

>   - The message must be a None or a string.  If it is None, the filter
>     applies to all messages.  The message string may end in "*" to
>     match all messages with the given text (up to the "*").

Same remark. If I want re, I know where to find it.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Mon Nov  6 11:42:34 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 6 Nov 2000 12:42:34 +0200 (IST)
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEMKHOAA.tim_one@email.msn.com>
Message-ID: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>

[Tim Peters]
> Some quickies for now:
> 
> + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"
> numeric model forever.

OK, I hope this e-mail address reaches him: I got it off his webpage.

[Tim Peters]
> + Everyone has IEEE-754 fp hardware today; some people actually want to use
> it; Moshe doesn't, but whatever revamping we get needs to allow others their
> delusions too.

My proposal has nothing *against* 754 either. For example, "all inexact
operations are done compliant to 754" is a perfectly acceptable addition.

[Tim Peters, about rationals]
> (provided they're *displayed* in rounded decimal fp notation (restart
> "str" vs "repr" rant, and that the interactive prompt uses the wrong one,
> and ditto str(container))),

Tim, that's the other PEP <wink>

[Tim Peters]
> IEEE-754 defines exactly what to do with this, for binary floats (and your
> hardware has an "inexact result" flag set or not after every fp operation).

Cool! I didn't know about that.

[Tim Peters]
> Conversion of the string "1.0" to float must not set it; conversion of "0.1"
> must set it; and similarly for + - * / sqrt:  "inexact result" gets set
> whenever the infinitely precise result differs from the computed result.

Is there some API for it in C? If not, we might as well assume that any
floating point number is inexact.

[Tim Peters]
> So
> inexactness there is neither a property of types nor of numbers, but of
> specific computations.  Extreme example:
> 
>     x = 1./3.   # inexact
>     y = x-x     # exact result (from inexact inputs!)
> 
> I know that this version (operation-based) of inexactness can be useful.  I
> see almost no earthly use for calling every number of a given type inexact.
> Tagging individual numbers with an exact/inexact bit is an extremely crude
> form of interval arithmetic (where the intervals are single points or
> infinite).

The tagging is whether you *want* exact operations or inexact operations.

I.e.:

1.0/3 --> inexact
1/3 --> exact.

Well, this is the classical definition of "type": what do the operations
mean? Which is why I need the inexact() function in my PEP.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov  6 13:13:06 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:13:06 +0100
Subject: [Python-Dev] PEP 224 (Attribute Docstrings)
References: <Pine.WNT.4.21.0011031535350.1220-100000@cr469175-a>  
	            <3A03DBD2.7E024978@lemburg.com> <200011060125.UAA03986@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A052.1990A512@lemburg.com>

Guido van Rossum wrote:
> 
> Marc-Andre:
> > I can take over the coercion PEP: I've been working
> > on this before (see the proposal on my Python Pages).
> 
> Thanks, excellent (although I haven't seen your proposal yet).
> 
> > I would also like to know whether the PEP-0224 will be considered
> > for 2.1 if I update the patch to make it a little more robust
> > w/r to the problems mentioned in that PEP -- I'd really like
> > to see this in Python soon, since it makes documenting Python
> > programs so much easier.
> 
> I "kinda" like the idea of having attribute docstrings (meaning it's
> not of great importance to me) but there are two things I don't like
> in your current proposal:
> 
> 1. The syntax you propose is too ambiguous: as you say, stand-alone
>    string literal are used for other purposes and could suddenly
>    become attribute docstrings.

This can be fixed by introducing some extra checks in the
compiler to reset the "doc attribute" flag in the compiler
struct.
 
> 2. I don't like the access method either (__doc_<attrname>__).

Any other name will do. It will only have to match these
criteria:

* must start with two underscores (to match __doc__)
* must be extractable using some form of inspection (e.g. by using
  a naming convention which includes some fixed name part)
* must be compatible with class inheritence (i.e. should be
  stored as attribute)

> > Note that I won't get around to do much work on these before
> > January... way too busy at the moment :-/
> 
> That's a problem -- we really want to have the PEPs ready for review
> by mid December.  This will also be a problem for the coercion PEP --
> if you think you won't be able to work on it before then, I'd prefer
> to find another (co-)author.

I think I'll need a co-author for the PEPs -- I have high-priority
project running which has a deadline in mid-December too.

Much food-for-thought is already available (see the PEP 214 and
the coercion proposal on my Python Pages) and I should find
time for some review.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Mon Nov  6 13:25:26 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 13:25:26 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com>
Message-ID: <3A06A336.5FD1EBC7@lemburg.com>

Guido van Rossum wrote:
> 
> [me]
> > > - Class/type dichotomy???
> 
> [MAL]
> > One thing that would probably be implementable is a way to
> > maintain "instance" dictionaries for types (which are created
> > on-demand whenever an assignment is made).
> >
> > This would enable
> > extending types with new methods and attributes. "Subclassing"
> > could then be emulated by using new contructors which add the
> > new or changed methods to each created type instance, e.g.
> >
> > class myclose:
> >
> >     def __init__(self, object, basemethod):
> >         self.object = object
> >         self.basemethod = basemethod
> >
> >     def __call__(self):
> >         print 'Closed file %s' % self.object
> >         self.basemethod()
> >
> > def myfile(filename):
> >     f = open(filename)
> >     # add/override attributes
> >     f.newattribute = 1
> >     # add/override methods
> >     f.close = myclose(f, f.close)
> >     return f
> >
> > Types would have to be made aware of this possibility. Python
> > could provide some helping APIs to make life easier for the
> > programmer.
> 
> But this would require an extra pointer field for *all* built-in
> types.  That would seriously impact the space requirements for ints
> and floats!

True.

> As long as we're proposing hacks like this that don't allow smooth
> subclassing yet but let you get at least some of the desired effects,
> I'd rather propose to introduce some kind of metaclass that will allow
> you to use a class statement to define this.  Thinking aloud:
> 
> import types
> filemetaclass = metaclass(types.FileType)
> 
> class myfile(filemetaclass):
> 
>       def __init__(self, filename):
>           filemetaclass.__init__(filename)
>           self.newattribute = 1
> 
>       def close(self):
>           myclose(self)
>           filemetaclass.close(self)
> 
> I'm not quite sure what kind of object "filemetaclass" here should be
> or what exactly "metaclass()" should do, but it could create a new
> type that has the lay-out of an existing file object, with an instance
> dictionary (for newattribute) tacked on the end.  Later maybe (I'm
> going to brainstorm with Jim Fulton about types and classes).

I think the problem we currently have with subclassing types
is strongly related to the fact that all Py<type>_Check()
macros only work on a address compare basis.

If we could find a way to change this to some kind of (very)
fast different lookup scheme we'd open a door which could
lead to making subclassing of types a whole lot easier.

<Brainstorming>

Perhaps a simple indirection could help...
instead of obj->ob_type == PyInteger_Type we'd write
obj->ob_type->base_type == PyInteger_Type_ID.

A subclass could then identify itself as integer subclass
by setting the base_type id to PyInteger_Type_ID. It would
of course have to publish the same internal structure in
order to remain compatible to the PyInteger_*() API, but there
would be a possibility to extend the object struct and
slots could also be replaced with new ones.

</>

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From jim at interet.com  Mon Nov  6 14:11:25 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Mon, 06 Nov 2000 08:11:25 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com>
Message-ID: <3A06ADFD.5C735BBA@interet.com>

"M.-A. Lemburg" wrote:

I am happy to modify zipfile.py as desired by this group.
 
> I'm having trouble opening ZIP files created using InfoZIP's
> zip utility (which uses zlib) with zipfile.py:
> 
> >>> x = z.read('README')
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
>   File "/home/lemburg/lib/zipfile.py", line 242, in read
>     bytes = dc.decompress(bytes)
> zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> 
> Is this due to the installed zlib on my system being incompatible,
> or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> zip version 2.2.

I "borrowed" the zlib code from someone else, and it uses
undocumented features.  Since WinZip seems happy with the compression
I left it alone.  I wouldn't be surprised if other utilities have
problems, especially if they use zlib.  I am not sure what to
do about this.  My starting view is that there is only one
compression method, and if WinZip accepts it, then InfoZip
is wrong.  Is there perhaps a version difference in zlib
in Python vs InfoZip?

Could you send me an InfoZip zip file?

> Also, I wonder why zipfile forces the mode flag to be 'r',
> 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> to the mode flag instead ?!

The 'b' is not necessary, as it is always added by zipfile.py
when the file is opened.  The [rwa] each do something
different, and aren't really the same as a file open.  I
am not sure what you mean here.
 
> The ZipFile is also missing some kind of method which
> extracts files in the ZIP archive to a file-like object. This
> would be very useful for extracting large files from a ZIP
> archive without having to first read in the whole file into
> memory.

The "read(self, name)" method returns the bytes, which you can
write to a file if you want.  What new method would you like?

JimA



From gvwilson at nevex.com  Mon Nov  6 14:39:37 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 6 Nov 2000 08:39:37 -0500 (EST)
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.10.10011060823080.26810-100000@akbar.nevex.com>

> Jeremy wrote:
> I tend to agree with you, Guido.  I think we would do well to purposefully
> omit continuations from the Python language.  There seems to be little need
> for a facility to implement arbitrary control structures in Python.  If
> Python support coroutines and microthreads, I am not sure what else would be
> needed.

I just finished reading Thomas and Hunt's "Programming Ruby" (the first book in
English on the language).  It's pretty clear that their favorite language
feature in Ruby is the block, which is any group of statements inside either
braces or do...end.  Blocks are invoked using the 'yield' construct, and can
take any number of arguments (enclosed in bars):

    def fibUpTo(max)
      i1, i2 = 1, 1
      while i1 <= max
        yield i1                      # 'call' the block
        i1, i2 = i2, i1 + i2
      end
    end

    fibUpTo(1000) { |f| print f, " " }

Most built-in types have iterators that understand blocks:

    [1, 3, 5].each { |i| puts i }     # prints 1, 3, and 5 on separate lines

Programmers can use blocks and 'yield' to create new control structures, subject
to the limitation that a statement can only be given one block (which means that
a multi-way interleaving loop can't be built out of blocks).  It would be
interesting to see how many of their examples can be done (easily) with
stackless...

Thanks,
Greg





From mal at lemburg.com  Mon Nov  6 15:00:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 15:00:33 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com>
Message-ID: <3A06B981.B3D73091@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> I am happy to modify zipfile.py as desired by this group.
> 
> > I'm having trouble opening ZIP files created using InfoZIP's
> > zip utility (which uses zlib) with zipfile.py:
> >
> > >>> x = z.read('README')
> > Traceback (innermost last):
> >   File "<stdin>", line 1, in ?
> >   File "/home/lemburg/lib/zipfile.py", line 242, in read
> >     bytes = dc.decompress(bytes)
> > zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree
> >
> > Is this due to the installed zlib on my system being incompatible,
> > or is this a bug in zipfile.py ? I have libz version 1.1.3 and
> > zip version 2.2.
> 
> I "borrowed" the zlib code from someone else, and it uses
> undocumented features.  Since WinZip seems happy with the compression
> I left it alone.  I wouldn't be surprised if other utilities have
> problems, especially if they use zlib.  I am not sure what to
> do about this.  My starting view is that there is only one
> compression method, and if WinZip accepts it, then InfoZip
> is wrong.  Is there perhaps a version difference in zlib
> in Python vs InfoZip?
> 
> Could you send me an InfoZip zip file?

As it turned out it was a false alarm: Python must have picked
up an old zlibmodule.so from somewhere which caused the
problem. A clean rebuild made zipfile work with my InfoZIP
files (all my Python Pages ZIP-archives are built using InfoZIP 2.2).

Now I get this error after working in interactive Python
mode with zipfile:

Exception exceptions.AttributeError: 
"ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

I guess the __del__ method has to be a bit more careful about
what it expects to find... sometimes the builtins may have
already been garbage collected.

> > Also, I wonder why zipfile forces the mode flag to be 'r',
> > 'w' and 'a' -- wouldn't it make more sense to only add 'b', etc.
> > to the mode flag instead ?!
> 
> The 'b' is not necessary, as it is always added by zipfile.py
> when the file is opened.  The [rwa] each do something
> different, and aren't really the same as a file open.  I
> am not sure what you mean here.

Sorry, I was only reading the code and got the impression that
mode testing was done on the complete string you pass to the
constructor ... I missed the line "key = mode[0]".
 
> > The ZipFile is also missing some kind of method which
> > extracts files in the ZIP archive to a file-like object. This
> > would be very useful for extracting large files from a ZIP
> > archive without having to first read in the whole file into
> > memory.
> 
> The "read(self, name)" method returns the bytes, which you can
> write to a file if you want.  What new method would you like?

I would like a method .copy(self, name, output) which
reads the file name from the ZIP archive and writes it directly to
the file-like object output. This should copy the file in chunks
of say 64kB in order to reduce memory load.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From nas at arctrix.com  Mon Nov  6 08:44:53 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 5 Nov 2000 23:44:53 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06A336.5FD1EBC7@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 01:25:26PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com>
Message-ID: <20001105234453.A8255@glacier.fnational.com>

On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> I think the problem we currently have with subclassing types
> is strongly related to the fact that all Py<type>_Check()
> macros only work on a address compare basis.

I don't think this is the problem, although it is closely
related.  The problem is that the interpreter uses these type
checks to special case the handling of certain types.
PyInstance_Check() is a big offender.

Behavior should be based solely on the type structure.  Extension
types would then be able to behave exactly like any other builtin
type.  Your coercion proposal and David's rich comparisions both
remove some of this special casing based on type.

  Neil



From gmcm at hypernet.com  Mon Nov  6 15:54:57 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 09:54:57 -0500
Subject: [Python-Dev] Stackless pages
In-Reply-To: <14854.11173.601039.883893@bitdiddle.concentric.net>
References: <200011060348.WAA04560@cj20424-a.reston1.va.home.com>
Message-ID: <3A067FF1.10498.57C83EB@localhost>

Jeremy wrote:

> ...  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.

I'm not sure what you mean here. So far, no one has asked for 
anything to get added to the *language* (although generators 
and coroutines could be made nicer with a bit of keyword 
support, it's not necessary).
 
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?  How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?

If by "control structure" you mean for / while / do type things, 
well, Python's not a functional language, and stackless / 
continuations won't grow you any new ones.

OTOH, if threads added new "control structures", then yes, 
continuations adds new ones (although from a language 
viewpoint, they don't look like anything new).
 
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.

You mean when C code (called by Python) ends up calling 
eval_code2, and the Python code so invoked wants to use a 
continuation which "resides" in some other invocation of 
eval_code2? Yes, Christian gives you an error at that point (I 
believe; I've never stumbled on this personally).

> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation. 

Well, sys.exit breaks that notion too :-). If an API says that a 
return is required, it's a programmer error not to return 
something.

I think it's universally agreed that there should be a "coroutine" 
API. Perhaps it is best done from scratch, or perhaps it is to 
the continuation module as threading is to thread. But I think 
Greg Ewing is right - it doesn't make much difference as far as 
stackless is concerned.

> I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?

I'm not understanding. 

In practical terms, I toasted my machine more times and more 
thoroughly playing with the thread module than I have with the 
continuation module. In that respect, continuations are a good 
deal safer - they don't have any effect on the state of your OS.

If you're playing with the raw "go-to" features of the 
continuation module, it's very easy to screw up. That's simply 
because code has to "balance", somehow. Call and return are 
the universally accepted way of doing that in the procedural 
community. But they're just a protocol based on go-to. The go-
to is still there, even if it's in the chip's microcode.

Christian's continuation module exposes a bunch of primitives. 
Most of the material on the pages I put up is concerned with 
how to mix and match those to get useful results (one 
example is coded 6 different ways). No one thinks this is a 
good "end state", just like the thread module was a lousy "end 
state".  But we can't steal Java's API here, and we can't steal 
from functional languages either, 'cause Python just ain't 
functional.

Everybody wants to see safer ways of using this stuff, but 
there's still a lot of experimenting to do. And I, for one, don't 
want to slam the door after one or two safe ways have been 
found. (And personally, I've found most of the attempts at, e.g. 
writing a "coroutine" class, less comprehensible than using 
the primitives directly.)



- Gordon



From tismer at tismer.com  Mon Nov  6 15:23:48 2000
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 16:23:48 +0200
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>
Message-ID: <3A06BEF4.95B773BD@tismer.com>


Greg Ewing wrote:
> 
> Jeremy Hylton <jeremy at alum.mit.edu>:
> 
> > Are continuations necessary to implement them or are
> > there other options?
> 
> I think you'll find that any implementation of microthreads
> or coroutines or whatever you want to call them, that
> doesn't rely on playing nonportable tricks with the C
> stack, will be just as mindbending as Stackless.
> 
> > The problem is that continuations break the notion a C API call will
> > always return an answer;
> 
> So do threads or coroutines. As soon as you have multiple
> threads of control, you have the chance that one of them
> will switch to another and never get back.

This is correct!
The only "special" thing with my continuation implementation
is that frames are armed to be able to accept a return
from a callee multiple times.
This little piece on top of frames turns them into full
continuations.

Without doing this, the API doesn't change, just the implementation
gets a little simpler.
What remains is still what Greg says:
The guarantee of stack-like frame execution no longer holds,
and every stackless C extension must either provide a way
to handle calls in a tail-recursive manner, *or* it must
enforce stack-like execution, like it is done today.

The *only* thing out of (coroutines, generators, uthreads)
which can be built without breaking this assumption are
the simple ICON-style generators.
But those can be implemented without becoming stackless
at all.

If you want full coroutines, or uthreads, the non-trivial
change of execution-order which Stackless permits *is*
necessary.

The step from there to supporting full continuations
is tiny, can be done easily or left completely.
The order of complexity of the Python implementation
is not increased by continuations.
In fact, supporting uthreads and coroutines and not only
stack-based generators impose the real problem.

That's one of my reasons to support continuations:
Making Python completely coroutine aware, without
tricking the C stack, is 90 percent of the problem.
But after walking that far, there is no reason
to leave the other 10 percent alone.

...
Greg:
> I can't see any way out of this. Continuations/coroutines/
> microthreads are all basically the same thing underneath, and
> they imply an execution model that just doesn't fit well
> with C.

Yes, you understood it completely.

> Maybe we need to reimplement Python in Scheme, and then
> feed it through a good Scheme compiler. SPython, anyone?

I believe the current implementation is quite pleasant
for a lot of people, while it isn't perfectly stackless.

A lot of internal operations could be made stackless by
introduction of a couple more opcodes, turning these
operations into bytecode. This would probably not cost
much speed, since only those code branches are a problem
which cause an interpreter recursion today, anyway.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From guido at python.org  Mon Nov  6 16:41:02 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 10:41:02 -0500
Subject: [Python-Dev] Three argument slices.
In-Reply-To: Your message of "Sat, 04 Nov 2000 16:21:48 GMT."
             <3a04376a.28016074@smtp.worldonline.dk> 
References: <3a04376a.28016074@smtp.worldonline.dk> 
Message-ID: <200011061541.KAA07351@cj20424-a.reston1.va.home.com>

> While updating the difference page in the Jython documentation, I came
> across this:
> 
> - JPython sequences support three argument slices. i.e. 
>   range(3)[::-1] == [2,1,0]. 
>   CPython should be fixed.
> 
> Is this actually true? Should (and will cpython) change in this respect?

Good question.  I haven't pronounced on Michael's patch

http://sourceforge.net/patch/?func=detailpatch&patch_id=100998&group_id=5470

because I don't know whether this is really a good idea.  I don't know
that I ever *will* pronounce uness someone points out how useful it is
for some practical application.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at wooz.org  Mon Nov  6 16:50:35 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:50:35 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
Message-ID: <14854.54091.14275.140381@anthem.concentric.net>

>>>>> "MZ" == Moshe Zadka <py-dev at zadka.site.co.il> writes:

    MZ> (which was meant for Barry to assign me a PEP number
    MZ> primarily...)

Looks like Guido beat me to it this morning and assigned 228 to your
PEP.  It really pisses me off though, him stepping on my toes there,
because clearly the PEP should have been assigned PEP .3/1.e-3

:)

-Barry



From barry at wooz.org  Mon Nov  6 16:59:40 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 10:59:40 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011042115.WAA01162@loewis.home.cs.tu-berlin.de>
	<E13sL2k-0004Rz-00@darjeeling.zadka.site.co.il>
	<200011050828.JAA00672@loewis.home.cs.tu-berlin.de>
	<E13sTXy-0004av-00@darjeeling.zadka.site.co.il>
	<200011051024.LAA01015@loewis.home.cs.tu-berlin.de>
	<E13sV5U-0004cp-00@darjeeling.zadka.site.co.il>
	<200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<20001105034044.M10135@lyra.org>
Message-ID: <14854.54636.49888.781754@anthem.concentric.net>

>>>>> "GS" == Greg Stein <gstein at lyra.org> writes:

    GS> Allocate yourself a PEP number and publish the darn thing. If
    GS> you don't feel comfortable grabbing a PEP number, then just
    GS> post it to the list or something.

A general rule is that meta-peps usually get assigned numbers < 100
and standards track peps just get the next free number above 200.
Pretty simple really.

However, I'm trying to encourage two things: only serious proposals
get pep'd, and peps ought to have a consistent style.  I want to avoid
approving peps for incomplete proposals (such as 207, not to purposely
pick on David).  There are enough good peps that you should largely be
able to figure out the style to use.  I can clean up any minor issues
after the fact.

I think the way Moshe did this one was fine.  He wrote it up and
posted it, and got assigned a pep number fairly quickly.

-Barry



From rrdn60 at email.sps.mot.com  Mon Nov  6 17:04:41 2000
From: rrdn60 at email.sps.mot.com (Norman Shelley (rrdn60))
Date: Mon, 06 Nov 2000 09:04:41 -0700
Subject: [Stackless] Re: [Python-Dev] Stackless pages
References: <14854.11173.601039.883893@bitdiddle.concentric.net>
Message-ID: <3A06D699.D3AFAC0@email.sps.mot.com>

Jeremy Hylton wrote:

> [Changed discussion list from general python-list to specific
> stackless.]
>
> >>>>> "GvR" == Guido van Rossum <guido at python.org> writes:
>
>   >> I have put up 6 pages of information about stackless at
>   >> http://www.mcmillan-inc.com/stackless.html
>
>   GvR> Gordon, thanks for doing this.  I still have a review of
>   GvR> Stackless on my TODO list.  It takes a serious chunk of my time
>   GvR> to do it justice, and this continues to be a problem, but the
>   GvR> existience of your overview certainly helps.  I still think
>   GvR> that the current Stackless implementation is too complex, and
>   GvR> that continuations aren't worth the insanity they seem to
>   GvR> require (or cause :-), but that microthreads and coroutines
>   GvR> *are* worth having and that something not completely unlike
>   GvR> Stackless will be one day the way to get there...
>
> I tend to agree with you, Guido.  I think we would do well to
> purposefully omit continuations from the Python language.  There seems
> to be little need for a facility to implement arbitrary control
> structures in Python.  If Python support coroutines and microthreads,
> I am not sure what else would be needed.
>
> It would be very helpful if the PEPs on Stackless could address this
> issue.  One way to address it is to ask these questions: What new
> control structures do users want in Python?

This kind of question/thought disturbs me.  It presumes we can determine
apriori all the ways one might wish to use the features that Stackless
provides.  Closing off or bounding innovation just because we can't
answer the question as to how it will be used will just cause future
forks in Python or promote or  non-Python choices.

> How best can they be
> implemented?  Are continuations necessary to implement them or are
> there other options?
>
> The sort of implementation complexity that I worry about with
> Stackless is, e.g. clean interaction with the C stack.  If a Python C
> API call is made that pushes a C stack frame, e.g. PyObject_Compare,
> then a continuation stored before that call can no longer be invokved.
> The problem is that continuations break the notion a C API call will
> always return an answer; they create a situation in which the C call
> that is made should never return, because control is transferred to
> the continuation.  I assume Stackless raises an error in this case,
> but this seems pretty messy: How do we right a language spec that
> explains when an error will occur without appealing to the
> language implementation?
>
> Jeremy
>
> _______________________________________________
> Stackless mailing list
> Stackless at starship.python.net
> http://starship.python.net/mailman/listinfo/stackless




From barry at wooz.org  Mon Nov  6 17:05:51 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:05:51 -0500 (EST)
Subject: [Python-Dev] Re: Revamping Python's Numeric Model
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de>
	<Pine.GSO.4.10.10011051411170.28025-100000@sundial>
	<14853.36547.748584.450976@cj42289-a.reston1.va.home.com>
	<14853.52668.117844.28459@bitdiddle.concentric.net>
	<14854.7221.546916.848838@cj42289-a.reston1.va.home.com>
Message-ID: <14854.55007.188953.545215@anthem.concentric.net>

BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
unreasonable that he'd have to wait until Monday to get a PEP number.
I reserve the right to apply Warsaw's 2nd Law to these cases. :)

-Barry

Warsaw's Second Law: Unbending Law of Commit Scheduling 
    Never change anything after 3pm on a Friday. 



From hinsen at cnrs-orleans.fr  Mon Nov  6 17:12:39 2000
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 6 Nov 2000 17:12:39 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <Pine.GSO.4.10.10011061235400.4175-100000@sundial> (message from
	Moshe Zadka on Mon, 6 Nov 2000 12:42:34 +0200 (IST))
References: <Pine.GSO.4.10.10011061235400.4175-100000@sundial>
Message-ID: <200011061612.RAA05582@chinon.cnrs-orleans.fr>

> > + Konrad Hinsen needs to be sucked in.  He's been arguing for a "unified"

I'd like to know what I am about to be sucked into here ;-)

> OK, I hope this e-mail address reaches him: I got it off his webpage.

That's fine, my web page always know how to reach me.


From barry at wooz.org  Mon Nov  6 17:23:43 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 11:23:43 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0228.txt,NONE,1.1 pep-0000.txt,1.44,1.45
References: <200011051655.IAA13536@slayer.i.sourceforge.net>
	<20001105201851.C27208@xs4all.nl>
	<20001105135439.V10135@lyra.org>
	<20001105231235.X12776@xs4all.nl>
Message-ID: <14854.56079.737563.729305@anthem.concentric.net>

>>>>> "TW" == Thomas Wouters <thomas at xs4all.net> writes:

    TW> Well, syncmail was written to manage the Python CVS tree on a
    TW> slow Sun (I believe)

Correct.
    
    TW> and did an rsync-over-ssh to another machine as well. That can
    TW> definately take long ;) If we just remove the fork, the rest
    TW> of syncmail might just work, even with new files. In the mean
    TW> time, I'll check in my change. It might be the best thing to
    TW> do anyway, since it shouldn't interfere unless the file isn't
    TW> there.

Your patch looks good Thomas, thanks for tracking this down.  IIRC,
without the fork, the checkin would actually deadlock trying to get
the diff.

-Barry



From guido at python.org  Mon Nov  6 17:28:50 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 11:28:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 15:52:55 +0200."
             <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com>  
            <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
Algol-60 did this.  Fortran does this.  And rational numbers are less
innovative?

And what's dangerous about 1/2 yielding 0.5?

> My PEP does not yet deal with either written or inputted representation.

These affect the numeric model though (see Tim's posts) so should be
considered.

> [GvR]
> > (2) Rational numbers have the unpleasant property of growing
> >     unboundedly during many innocent calculations, thereby using up
> >     exorbitant amounts of memory and slowing down the calculation --
> >     often mysteriously, because what is displayed is truncated.
> 
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
is clearly an issue open for debate, but note that I speak from
experience: ABC used rationals unless you forced it to use reals, and
the rationals *did* cause real users to complain about how slow ABC
was.

> [GvR]
> > Another issue that I might bring up is that there are no inexact
> > numbers (each floating point number is perfectly exact and rational)
> 
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

Without a clear implementation plan your PEP is incomplete, PEP
guidelines notwithstanding.  Quality of implementation is important
for such a basic feature!

> [GvR]
> > If we take its meaning literally, the isreal() function should only
> > return true for numbers for which isrational() is also true:
> 
> Correct -- in my current model. If you later add things like constructive
> reals, that is no longer true: if I have a constructive Pi, it's not 
> rational. 
> 
> [GvR]
> > mathematically speaking, real numbers that aren't also rational don't
> > have an easy finite representation, since they are numbers like
> > sqrt(2) or pi.
> 
> But numbers don't have to have a finite (periodic really) representation
> to be representable in the computer: what about infinite continued fractions,
> for example?

So the question is, what implementation do you have in mind?  You
can't just go prescribe idealistic semantics and hope it gets
implemented by magic (even Tim can't do that :-).

> [GvR]
> > Finally, the PEP doesn't talk about how the numeric model can be
> > extended
> 
> That's because its rich enough not to need it. 
> Let me explain exactly what I mean: as long as all field operations between
> Python numbers give honest to god Python numbers, then everything else
> can be solved with Python's current model of coercions, and can be solved
> well when the coercion PEP will be written.

I think this deserves very explicit mention in your PEP.  An example
of how I would go about implementing my own Rational class or
extension type would help.

Also, the coercions PEP is still in need of an author.  Maybe you want
to take this on too?  It will help your numeric proposal if you can
write down how you think coercions should work.

> [GvR]
> > I've
> > heard of wild floating point representations that make multiplication
> > and division really cheap but addition a pain, rather than the other
> > way around;
> 
> Well, no problems: write wild.inexact()  and wildmath.{sqrt,...} and use
> that instead of inexact() and math.{...}. How these numbers interact
> with builtin Python numbers is your responsibility -- and that's what
> the coercions are for.

So explain how to do the coercions.  This will force you to be
explicit about implementation details.  (Both from Python and from C.)

> Same goes for gmp: as long as you're not expecting to be able to change
> 10000000000+10000000000 to be a gmp long rather then a Python long, then
> there shouldn't be a problem.

Fair enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 18:02:51 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:02:51 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com>
Message-ID: <3A06E43B.D95EC267@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 01:25:26PM +0100, M.-A. Lemburg wrote:
> > I think the problem we currently have with subclassing types
> > is strongly related to the fact that all Py<type>_Check()
> > macros only work on a address compare basis.
> 
> I don't think this is the problem, although it is closely
> related.  The problem is that the interpreter uses these type
> checks to special case the handling of certain types.
> PyInstance_Check() is a big offender.
> 
> Behavior should be based solely on the type structure.  Extension
> types would then be able to behave exactly like any other builtin
> type.  Your coercion proposal and David's rich comparisions both
> remove some of this special casing based on type.

Even though this could remove some of the problems, it doesn't
help much with a common use of Py<Type>_Check(): that of
using fast access macros and native Py<Type>_*() APIs once an
object has been classified as being of a certain type.
This usually improves performance.

By changing the simple address compare to a type handle
system, we might be able to add some more flexibility to
the system while keeping b/w compatibility.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 18:17:22 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:17:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."
             <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>

> [Guido]:
> > David Scherer proposed to spell this pragma as a "magical import"
> > (http://www.python.org/pipermail/idle-dev/2000-April/000138.html). 

[Peter Funk]
> Huh?  AFAIR David Scherer and Bruce Sherwood used the 'global'-statement 
> at module level as a backward compatible method to introduce module level
> pragmas.  (http://www.python.org/pipermail/idle-dev/2000-April/000140.html)
> I still like David Scherers proposal very much.

Oops, you're right.  That URL mentions "global olddivision".  The
"import floatdivision" proposal came from a more recent private mail
from Bruce Sherwood.  I maintain that neither seems the right way to
spell "directive".  I think "import" is slightly better if the import
is supposed to enable a feature that is not supported by previous
versions, because the import will cause a clear failure on systems
that don't have the new feature (rather than silently giving wrong
results sometimes).

> BTW:  I think the "symbol" '//' is incredible ugly and starting with
> IBMs JCL years ago all languages I encountered, that used this symbol
> for something, did suck in some way or another.  I would appreaciate
> very much, if it could be avoided alltogether to add a symbol '//'
> to Python.  '//' will look like a comment delimiter to most people today.

Good point.  I've just used // as a placeholder for a new way to spell
integer division.

> Using a new keyword like 'div' in the tradition of languages like 
> Modula-2 looks far more attractive to me.

That's a possibility too.  It's a new keyword though, which has a much
higher threshold for acceptance than a new two-character operator
symbol.  We could spell it as a built-in function: div(a, b),
(analogous to divmod(a, b)) but that's not very user-friendly either.

Keep looking...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 18:20:03 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:20:03 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 10:14:12 +0100."
             <3A067664.22D09D03@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>  
            <3A067664.22D09D03@lemburg.com> 
Message-ID: <200011061720.MAA07862@cj20424-a.reston1.va.home.com>

[GvR]
> > Hm...  There's also the problem that there's no easy way to do Unicode
> > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > local encoding), which should mean that writing Unicode objects to the
> > file should "do the right thing" (in particular should not try to
> > coerce it to an 8-bit string using the default encoding first, like
> > print and str() currently do) and that writing 8-bit string objects to
> > it should first convert them to Unicode using the default encoding
> > (meaning that at least ASCII strings can be written to a Unicode file
> > without having to specify a conversion).  I support that reading from
> > a "Unicode file" should always return a Unicode string object (even if
> > the actual characters read all happen to fall in the ASCII range).
> > 
> > This requires some serious changes to the current I/O mechanisms; in
> > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > added that it used in certain cases.  Tricky, tricky!

[MAL]
> It's not all that tricky since you can write a StreamRecoder
> subclass which implements this. AFAIR, I posted such an implementation
> on i18n-sig.
> 
> BTW, one of my patches on SF adds unistr(). Could be that it's
> time to apply it :-)

Adding unistr() and StreamRecoder isn't enough.  The problem is that
when you set sys.stdout to a StreamRecoder, the print statement
doesn't do the right thing!  Try it.  print u"foo" will work, but
print u"\u1234" will fail because print always applies the default
encoding.

The required changes to print are what's tricky.  Whether we even need
unistr() depends on the solution we find there.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tismer at tismer.com  Mon Nov  6 17:19:21 2000
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 06 Nov 2000 18:19:21 +0200
Subject: [Python-Dev] cgi.py and huge uploads problem
Message-ID: <3A06DA09.143D2D9@tismer.com>

Howdy,

there is a problem with the cgi.py implementation of Python 1.5.2
and uploading of huge files.
(found out by Joachim Rudolph, Virtual Photonics)

Class FieldStorage of cgi.py has a couple of methods which
add accumulated lines to a self.lines array. This array fills and
fills until the whole upload is done, with the side effect of
loading the whole file into memory. The memory is freed
after the whole upload is done.

This is no problem, until a company like VPI uses cgi.py to
upload whole distributions of 100 MB and more, via Zope. :-)

Looking into cgi.py, I can't find a reason why this happens.
Is this possibly just a debugging feature which is no longer
needed?
While cgi.py was modified for cosmetic resons, I didn't find
changes for Python 2.0 on this topic.

Does it make sense to use a debug flag for this, or should
the feature vanish completely?
Do you want a patch?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at zadka.site.co.il  Tue Nov  7 02:30:05 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:30:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Mon, 06 Nov 2000 11:28:50 EST." <200011061628.LAA07574@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il>  <200011061628.LAA07574@cj20424-a.reston1.va.home.com> 
Message-ID: <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>

[Moshe]
> The reason the PEP was written now was because you started making sounds
> of causing 1/2 to be 0.5, which is dangerous. I tried to outline the PEP
> to show a less dangerous (and less innovative) way of getting similar
> usability.

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

OK, I take back the innovation claim.

[Guido]
> And what's dangerous about 1/2 yielding 0.5?

Nothing, but then people would expect 1/3 to yield 0.333333333....,
while simultaneusly expecting (1/3)*3 == 1.
IOW, shoving floating points in people's faces is not user-friendly.

[Moshe]
> My PEP does not yet deal with either written or inputted representation.

[Guido]
> These affect the numeric model though (see Tim's posts) so should be
> considered.

I agree. That's why they are in the "unresolved issues" of the PEP.

[Guido]
> (2) Rational numbers have the unpleasant property of growing
>     unboundedly during many innocent calculations, thereby using up
>     exorbitant amounts of memory and slowing down the calculation --
>     often mysteriously, because what is displayed is truncated.

[Moshe]
> Those calculations, if performed with floating points, would often build
> up inaccuracy. I prefer to go slow+memory hog (which the user feels), then
> to be wildly inaccurate.

[Guido]
> That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> is clearly an issue open for debate, but note that I speak from
> experience: ABC used rationals unless you forced it to use reals, and
> the rationals *did* cause real users to complain about how slow ABC
> was.

Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
list, and nobody ever complained. Besides, if "forcing" is simply saying
either inexact() (and note that inexact-exact operations are performed
inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
be), it might be easier. I don't know ABC well enough to say how the
forcing mechanism worked.

[Guido]
> Another issue that I might bring up is that there are no inexact
> numbers (each floating point number is perfectly exact and rational)

[Moshe]
> You'll note that my PEP does not mention floating point explicitly --
> and once again I mention that my PEP does not yet deal with number
> literals. All it allows (not requires) is for things like math.sqrt()
> to return inexact results. Naive implementations (which we might use)
> would make math.sqrt(1) an *inexact* 1, tentatively called 1.0. Of 
> course, good implementations of math.sqrt() would realize that 1 has
> an exact root, but that might be too hard to do for not enough gain.

[Guido]
> Without a clear implementation plan your PEP is incomplete, PEP
> guidelines notwithstanding.  Quality of implementation is important
> for such a basic feature!

Well, I agree. When I set down to complete the PEP, I'll go over all
the math/cmath functions and remark how the implementation should change.
I also plan to draft an implementation design. I just wanted to throw
the idea out into the open, to get some feedback -- in no way can the
PEP be considered complete.

[Guido]
> So the question is, what implementation do you have in mind?  You
> can't just go prescribe idealistic semantics and hope it gets
> implemented by magic (even Tim can't do that :-).

Well, for constructive reals, none. It's too much of pain to implement,
and too little to gain. That wouldn't preclude a later day implementation,
in case it turns out not to be the case. Besides, the tower wouldn't
look clean without it.

[Guido]
> I think this deserves very explicit mention in your PEP.  An example
> of how I would go about implementing my own Rational class or
> extension type would help.

Well, I don't see why there should be a difference from what happens
currently. The thing is, the model will not support you telling it
at runtime what the results for operations on types it already knows
should be: that would make us Scheme, not Python. 

[Guido]
> Also, the coercions PEP is still in need of an author.  Maybe you want
> to take this on too?  It will help your numeric proposal if you can
> write down how you think coercions should work.

Smooth, real smooth. OK, sold to the highest bidder -- I take it.
I'll update it and 0000, and start reading MAL's pages.

[Guido]
> So explain how to do the coercions.  This will force you to be
> explicit about implementation details.  (Both from Python and from C.)

Again this is no different from current day Python, modulu other PEPs. <wink> 
-- 
Moshe Zadka <sig at zadka.site.co.il>



From gmcm at hypernet.com  Mon Nov  6 18:27:22 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Mon, 6 Nov 2000 12:27:22 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 10:05:12 +0100."             <m13siDI-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <3A06A3AA.1401.6080F51@localhost>

[Peter Funk]
> > Using a new keyword like 'div' in the tradition of languages like 
> > Modula-2 looks far more attractive to me.
[Guido]:
> That's a possibility too.  It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.  We could spell it as a built-in function: div(a, b),
> (analogous to divmod(a, b)) but that's not very user-friendly either.
> 
> Keep looking...

FWIW, I remember as a newbie being sure that integer 
division was spelled "a div b". In fact when it didn't work, I went 
digging through PP, IPWP and the docs in dumbfounded 
disbelief.

this-is-your-brain-on-Modula-2-ly y'rs

- Gordon



From nas at arctrix.com  Mon Nov  6 11:35:29 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 6 Nov 2000 02:35:29 -0800
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A06E43B.D95EC267@lemburg.com>; from mal@lemburg.com on Mon, Nov 06, 2000 at 06:02:51PM +0100
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com>
Message-ID: <20001106023529.B8639@glacier.fnational.com>

On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> Neil Schemenauer wrote:
> > Behavior should be based solely on the type structure.  Extension
> > types would then be able to behave exactly like any other builtin
> > type.  Your coercion proposal and David's rich comparisions both
> > remove some of this special casing based on type.
> 
> Even though this could remove some of the problems, it doesn't
> help much with a common use of Py<Type>_Check(): that of
> using fast access macros and native Py<Type>_*() APIs once an
> object has been classified as being of a certain type.
> This usually improves performance.

Can you clarify what you mean by "it doesn't help much"?  Do you
mean that extension types will not be able to perform as well as
types that get special treatment by the interpreter?  I think the
major problem that extension types _cannot_ behave the same as
the builtin types.

> By changing the simple address compare to a type handle
> system, we might be able to add some more flexibility to
> the system while keeping b/w compatibility.

I don't see what this buys us.  The Python interpreter shouldn't
care about which type object it is dealing with.  Can you give an
example of where you think this would be useful?

  Neil



From moshez at zadka.site.co.il  Tue Nov  7 02:40:30 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 03:40:30 +0200
Subject: [Python-Dev] Re: Revamping Python's Numeric Model 
In-Reply-To: Message from barry@wooz.org (Barry A. Warsaw) 
   of "Mon, 06 Nov 2000 11:05:51 EST." <14854.55007.188953.545215@anthem.concentric.net> 
References: <200011051121.MAA01234@loewis.home.cs.tu-berlin.de> <Pine.GSO.4.10.10011051411170.28025-100000@sundial> <14853.36547.748584.450976@cj42289-a.reston1.va.home.com> <14853.52668.117844.28459@bitdiddle.concentric.net> <14854.7221.546916.848838@cj42289-a.reston1.va.home.com>  <14854.55007.188953.545215@anthem.concentric.net> 
Message-ID: <E13sxkU-0005PL-00@darjeeling.zadka.site.co.il>

[Barry]
> BTW, Moshe posted his PEP on Saturday, and I don't think it's at all
> unreasonable that he'd have to wait until Monday to get a PEP number.

It is to me. If I work on Sunday, I might as well do some Python work too...
Problems of truly international open source projects <wink>
-- 
Moshe Zadka <sig at zadka.site.co.il>



From guido at python.org  Mon Nov  6 18:43:07 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:43:07 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:19:21 +0200."
             <3A06DA09.143D2D9@tismer.com> 
References: <3A06DA09.143D2D9@tismer.com> 
Message-ID: <200011061743.MAA08031@cj20424-a.reston1.va.home.com>

> there is a problem with the cgi.py implementation of Python 1.5.2
> and uploading of huge files.
> (found out by Joachim Rudolph, Virtual Photonics)
> 
> Class FieldStorage of cgi.py has a couple of methods which
> add accumulated lines to a self.lines array. This array fills and
> fills until the whole upload is done, with the side effect of
> loading the whole file into memory. The memory is freed
> after the whole upload is done.
> 
> This is no problem, until a company like VPI uses cgi.py to
> upload whole distributions of 100 MB and more, via Zope. :-)
> 
> Looking into cgi.py, I can't find a reason why this happens.
> Is this possibly just a debugging feature which is no longer
> needed?
> While cgi.py was modified for cosmetic resons, I didn't find
> changes for Python 2.0 on this topic.
> 
> Does it make sense to use a debug flag for this, or should
> the feature vanish completely?
> Do you want a patch?

You know, I have *no* idea why this is.  I have looked through various
revisions (this feature is as old as cgi.py:1.8) and cannot find any
use of or need for self.lines!  It just gets appended to.

So I think it's safe to toss all the references to self.lines and see
who complains.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 18:36:54 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 18:36:54 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com>  
	            <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>
Message-ID: <3A06EC36.61B32B4D@lemburg.com>

Guido van Rossum wrote:
> 
> [GvR]
> > > Hm...  There's also the problem that there's no easy way to do Unicode
> > > I/O.  I'd like to have a way to turn a particular file into a Unicode
> > > output device (where the actual encoding might be UTF-8 or UTF-16 or a
> > > local encoding), which should mean that writing Unicode objects to the
> > > file should "do the right thing" (in particular should not try to
> > > coerce it to an 8-bit string using the default encoding first, like
> > > print and str() currently do) and that writing 8-bit string objects to
> > > it should first convert them to Unicode using the default encoding
> > > (meaning that at least ASCII strings can be written to a Unicode file
> > > without having to specify a conversion).  I support that reading from
> > > a "Unicode file" should always return a Unicode string object (even if
> > > the actual characters read all happen to fall in the ASCII range).
> > >
> > > This requires some serious changes to the current I/O mechanisms; in
> > > particular str() needs to be fixed, or perhaps a ustr() needs to be
> > > added that it used in certain cases.  Tricky, tricky!
> 
> [MAL]
> > It's not all that tricky since you can write a StreamRecoder
> > subclass which implements this. AFAIR, I posted such an implementation
> > on i18n-sig.
> >
> > BTW, one of my patches on SF adds unistr(). Could be that it's
> > time to apply it :-)
> 
> Adding unistr() and StreamRecoder isn't enough.  The problem is that
> when you set sys.stdout to a StreamRecoder, the print statement
> doesn't do the right thing!  Try it.  print u"foo" will work, but
> print u"\u1234" will fail because print always applies the default
> encoding.

Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
Perhaps we ought to let it call PyObject_Unicode() (which you
find in the patch on SF) instead for Unicode objects. That way
the file-like .write() method will be given a Unicode object
and StreamRecoder could then do the trick.

Haven't tried this, but it could work (the paths objects take
through Python to get printed are somewhat strange at times
-- there are just so many different possiblities and special
cases that it becomes hard telling from just looking at the
code).

> The required changes to print are what's tricky.  Whether we even need
> unistr() depends on the solution we find there.

I think we'll need PyObject_Unicode() and unistr() one way
or another. Those two APIs simply complement PyObject_Str()
and str() in that they always return Unicode objects and
do the necessary conversion based on the input object type.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 18:49:35 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:49:35 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Tue, 07 Nov 2000 03:30:05 +0200."
             <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com>  
            <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>

> [Guido]
> > That's your choice.  Wildly inaccurate is sometimes unavoidable.  This
> > is clearly an issue open for debate, but note that I speak from
> > experience: ABC used rationals unless you forced it to use reals, and
> > the rationals *did* cause real users to complain about how slow ABC
> > was.

[Moshe]
> Well, experiences differ. I'm on the Scheme48 (which uses rationals) mailing
> list, and nobody ever complained. Besides, if "forcing" is simply saying
> either inexact() (and note that inexact-exact operations are performed
> inexactly) or mixing in an inexact literal (assuming that's what 1.0 will
> be), it might be easier. I don't know ABC well enough to say how the
> forcing mechanism worked.

Scheme has no real users -- only CS types. :-)

ABC's forcing was as simple as writing ~x or mixing inexact numbers.
We did have the notion that 1.0 was an exact literal (to me it looks
exact!) so maybe that was an additional problem.

> [Guido]
> > I think this deserves very explicit mention in your PEP.  An example
> > of how I would go about implementing my own Rational class or
> > extension type would help.
> 
> Well, I don't see why there should be a difference from what happens
> currently. The thing is, the model will not support you telling it
> at runtime what the results for operations on types it already knows
> should be: that would make us Scheme, not Python. 

Agreed.  I think what makes me feel uneasy is that your proposal
assumes that there is One True Numeric Type, and all the rest are
second-class numbers.  Python currently makes no such claim: there are
many numeric types built in and you can add your own types that play
along.  The only thing that makes a difference is that there are
literals for the built-in types and not for the extension types; but
apart from that they have all the same privileges, and the coercion
rules work in everybody's favor (except when they don't :-). Placing
more emphasis on One True Numeric Type runs a risk of discriminating
against the others.

> [Guido]
> > Also, the coercions PEP is still in need of an author.  Maybe you want
> > to take this on too?  It will help your numeric proposal if you can
> > write down how you think coercions should work.
> 
> Smooth, real smooth. OK, sold to the highest bidder -- I take it.
> I'll update it and 0000, and start reading MAL's pages.

OK, it's a deal.  But make yourself a co-author with MAL.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov  6 18:53:36 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 12:53:36 -0500
Subject: [Python-Dev] More Unicode support
In-Reply-To: Your message of "Mon, 06 Nov 2000 18:36:54 +0100."
             <3A06EC36.61B32B4D@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>  
            <3A06EC36.61B32B4D@lemburg.com> 
Message-ID: <200011061753.MAA08112@cj20424-a.reston1.va.home.com>

[Guido]
> > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > when you set sys.stdout to a StreamRecoder, the print statement
> > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > print u"\u1234" will fail because print always applies the default
> > encoding.

[MAL]
> Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> Perhaps we ought to let it call PyObject_Unicode() (which you
> find in the patch on SF) instead for Unicode objects. That way
> the file-like .write() method will be given a Unicode object
> and StreamRecoder could then do the trick.

That's still not enough. Classes and types should be able to have a
__str__ (or tp_str) that yields Unicode too.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at zadka.site.co.il  Tue Nov  7 03:08:05 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 04:08:05 +0200
Subject: [Python-Dev] Revamping Python's Numeric Model 
In-Reply-To: Message from Guido van Rossum <guido@python.org> 
   of "Mon, 06 Nov 2000 12:49:35 EST." <200011061749.MAA08063@cj20424-a.reston1.va.home.com> 
References: <E13s7cZ-0002mA-00@darjeeling.zadka.site.co.il> <200011060119.UAA03952@cj20424-a.reston1.va.home.com> <E13smhk-0005BM-00@darjeeling.zadka.site.co.il> <200011061628.LAA07574@cj20424-a.reston1.va.home.com> <E13sxaQ-0005Lm-00@darjeeling.zadka.site.co.il>  <200011061749.MAA08063@cj20424-a.reston1.va.home.com> 
Message-ID: <E13syBB-0005Zr-00@darjeeling.zadka.site.co.il>

[Guido]
> Scheme has no real users -- only CS types. :-)

I resent that! I'm not a CS type: I never studied CS, and I refuse
to stand for that kind of mud slinging.

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

Let me clarify: exactness *is* in the operations. Since Python is
an OO language, an object decides semantics of operations on it. "1.0"
decides to have inexact operations on it, no matter how exact it
looks to you.

[Guido]
> Agreed.  I think what makes me feel uneasy is that your proposal
> assumes that there is One True Numeric Type, and all the rest are
> second-class numbers.  Python currently makes no such claim: there are
> many numeric types built in and you can add your own types that play
> along.

My proposal doesn't change that, as far as it's true. (And it isn't
really true: I cant have my new types as literals, or as results of
existing operations). It changes the number of built-in numeric types
to 1.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Mon Nov  6 19:15:27 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 19:15:27 +0100
Subject: [Python-Dev] More Unicode support
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060340.WAA04479@cj20424-a.reston1.va.home.com> <3A067664.22D09D03@lemburg.com> <200011061720.MAA07862@cj20424-a.reston1.va.home.com>  
	            <3A06EC36.61B32B4D@lemburg.com> <200011061753.MAA08112@cj20424-a.reston1.va.home.com>
Message-ID: <3A06F53F.2CF6760@lemburg.com>

Guido van Rossum wrote:
> 
> [Guido]
> > > Adding unistr() and StreamRecoder isn't enough.  The problem is that
> > > when you set sys.stdout to a StreamRecoder, the print statement
> > > doesn't do the right thing!  Try it.  print u"foo" will work, but
> > > print u"\u1234" will fail because print always applies the default
> > > encoding.
> 
> [MAL]
> > Hmm, that's due to PyFile_WriteObject() calling PyObject_Str().
> > Perhaps we ought to let it call PyObject_Unicode() (which you
> > find in the patch on SF) instead for Unicode objects. That way
> > the file-like .write() method will be given a Unicode object
> > and StreamRecoder could then do the trick.
> 
> That's still not enough. Classes and types should be able to have a
> __str__ (or tp_str) that yields Unicode too.

Instances are allowed to return Unicode through their __str__
method and PyObject_Unicode() will pass it along. PyObject_Str()
will still convert it to an 8-bit string though because there's
too much code out there which expects a string object (without
checking !) ... even the Python core.

So if you print an instance which returns Unicode through __str__,
the wrapper should see a real Unicode object at its end... at least
I think we're getting closer ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From barry at wooz.org  Mon Nov  6 19:32:44 2000
From: barry at wooz.org (Barry A. Warsaw)
Date: Mon, 6 Nov 2000 13:32:44 -0500 (EST)
Subject: [Python-Dev] cgi.py and huge uploads problem
References: <3A06DA09.143D2D9@tismer.com>
	<200011061743.MAA08031@cj20424-a.reston1.va.home.com>
Message-ID: <14854.63820.201069.434450@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> You know, I have *no* idea why this is.  I have looked
    GvR> through various revisions (this feature is as old as
    GvR> cgi.py:1.8) and cannot find any use of or need for
    GvR> self.lines!  It just gets appended to.

    GvR> So I think it's safe to toss all the references to self.lines
    GvR> and see who complains.

There are two bug reports related to this in the SF database.  The
first one was 110674, which we closed after adding the feature request
to PEP 42.  The second is 119806, which looks fairly new, but wasn't
submitted by Chris or Joachim.

I came to the same conclusion Guido does above when I looked at 110674
in the Python 2.0 time frame, but didn't feel comfortable making that
change for 2.0.  I think it's correct to make the change now.

I will do the following:

- remove self.lines from cgi.py
- close bug #119806
- update pep 42

Attached below is the patch.

-Barry

-------------------- snip snip --------------------
Index: cgi.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/cgi.py,v
retrieving revision 1.55
diff -u -r1.55 cgi.py
--- cgi.py	2000/10/03 13:51:09	1.55
+++ cgi.py	2000/11/06 18:32:18
@@ -497,7 +497,6 @@
 
         self.list = self.file = None
         self.done = 0
-        self.lines = []
         if ctype == 'application/x-www-form-urlencoded':
             self.read_urlencoded()
         elif ctype[:10] == 'multipart/':
@@ -633,7 +632,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             self.file.write(line)
 
     def read_lines_to_outerboundary(self):
@@ -646,7 +644,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:
@@ -676,7 +673,6 @@
             if not line:
                 self.done = -1
                 break
-            self.lines.append(line)
             if line[:2] == "--":
                 strippedline = string.strip(line)
                 if strippedline == next:



From guido at python.org  Mon Nov  6 20:09:26 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 14:09:26 -0500
Subject: [Python-Dev] cgi.py and huge uploads problem
In-Reply-To: Your message of "Mon, 06 Nov 2000 13:32:44 EST."
             <14854.63820.201069.434450@anthem.concentric.net> 
References: <3A06DA09.143D2D9@tismer.com> <200011061743.MAA08031@cj20424-a.reston1.va.home.com>  
            <14854.63820.201069.434450@anthem.concentric.net> 
Message-ID: <200011061909.OAA08466@cj20424-a.reston1.va.home.com>

>     GvR> You know, I have *no* idea why this is.  I have looked
>     GvR> through various revisions (this feature is as old as
>     GvR> cgi.py:1.8) and cannot find any use of or need for
>     GvR> self.lines!  It just gets appended to.
> 
>     GvR> So I think it's safe to toss all the references to self.lines
>     GvR> and see who complains.

[Barry]
> There are two bug reports related to this in the SF database.  The
> first one was 110674, which we closed after adding the feature request
> to PEP 42.  The second is 119806, which looks fairly new, but wasn't
> submitted by Chris or Joachim.
> 
> I came to the same conclusion Guido does above when I looked at 110674
> in the Python 2.0 time frame, but didn't feel comfortable making that
> change for 2.0.  I think it's correct to make the change now.
> 
> I will do the following:
> 
> - remove self.lines from cgi.py
> - close bug #119806
> - update pep 42
> 
> Attached below is the patch.

Patch looks good.  Go for it.

Somehow when I skimmed the original bug reports I never really
understood what was going on -- thanks to Christian for pointing out
*exactly* what was the problem in words I could understand. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov  6 21:01:40 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 06 Nov 2000 21:01:40 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>
Message-ID: <3A070E24.B07FA7D7@lemburg.com>

Neil Schemenauer wrote:
> 
> On Mon, Nov 06, 2000 at 06:02:51PM +0100, M.-A. Lemburg wrote:
> > Neil Schemenauer wrote:
> > > Behavior should be based solely on the type structure.  Extension
> > > types would then be able to behave exactly like any other builtin
> > > type.  Your coercion proposal and David's rich comparisions both
> > > remove some of this special casing based on type.
> >
> > Even though this could remove some of the problems, it doesn't
> > help much with a common use of Py<Type>_Check(): that of
> > using fast access macros and native Py<Type>_*() APIs once an
> > object has been classified as being of a certain type.
> > This usually improves performance.
> 
> Can you clarify what you mean by "it doesn't help much"?  Do you
> mean that extension types will not be able to perform as well as
> types that get special treatment by the interpreter?  I think the
> major problem that extension types _cannot_ behave the same as
> the builtin types.

If you only define types by interface, the interpreter will have
to apply the interface availability checks every time it calls
a slot. This would cause a major performance hit which would not
be acceptable.

The "doesn't help much" refers to the fact that once an object
has been identified as say Integer you are free to use whatever
access macro or function you choose.

> > By changing the simple address compare to a type handle
> > system, we might be able to add some more flexibility to
> > the system while keeping b/w compatibility.
> 
> I don't see what this buys us.  The Python interpreter shouldn't
> care about which type object it is dealing with.  Can you give an
> example of where you think this would be useful?

Take e.g. dictionaries: you could easily add a new dictionary
type which uses case-insensitive string keys by extending the
existing dictionary type.

The new type would reuse most of the slots of the original
type and only replace the ones needed for lookup with the
new logic for case-insensitivity.

Then it sets the type id to PyDict_TypeID and Python will
use it as if it were an original dictionary object. The
underlying type objects would be different though (and also
the type object address which is currently used to identify
a builtin type).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov  6 22:02:27 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 16:02:27 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Sun, 05 Nov 2000 22:24:27 PST."
             <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> 
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> 
Message-ID: <200011062102.QAA09294@cj20424-a.reston1.va.home.com>

Paul, thanks for submitting a warnings framework.  I'd like to give
some feedback, comparing it to my own proposal.  Please also give
explicit feedback on mine!

[Paul Prescod]
> Abstract
> 
>     This PEP describes a generalized warning mechanism for Python 2.1. The
>     primary purpose of this mechanism is to alert the programmer or user
>     of a program to potential or likely errors which, for whatever reason,
>     are not considered exception-worthy. For example, this might be done
>     to keep old code working during a transitional period or to alert the
>     programmer or user of a recoverable error.

What's missing here is a set of requirements.  Comparing your proposal
to my requirements, I find that you are not addressing two
requirements that are important in my mind: a convenient and efficient
C API, and a mechanism that prints a warning message only the first
time that the warning is issued.  These are important to me, because I
expect that most warnings will probably be generated by C code, and in
order to be useful we must avoid mindless repetition.  If a function
divides two integers using the / operator, this is being detected by C
code (the int or long division implementation) and we only want to
print the warning once per program execution and per source location.

My expectation is that if a single occurrence (in the program) of a
warning condition caused an endless sequence of warnings to be spit
out, people would quickly grow a habit of disabling all warnings, thus
defeating the purposes.

Warnings are more a human engineering issue than a technical issue!
That's also why I am emphasizing a C API -- I want to make it real
easy to ussue quality warnings in the runtime.  It's also why I
specify a rich (probably too rich!) filtering mechanism.

> Syntax
> 
>     assert >> cls, test[[[, arg], arg]...]

I have several problems with this.  First of all, using "assert" means
that in "optimizing" mode (python -O) you won't get *any* warnings.  I
think that the decision to disable all warnings should be independent
from the decision to "optimize".  Second, you're hypergeneralizing the
extended print syntax.  Just because I think it's okay to add >>file
to the print syntax doesn't mean that it's now okay to add >>object
syntax to all statements!

I also don't see what warnings have to do with assertions.  Assertions
are a mechanism to check for error conditions.  What happens if the
error is detected is of less importance -- it could raise an exception
(Python), issue a fatal error (C), or do nothing (in -O mode).

With warnings I believe the issue is not so much the detection of the
condition (for which a regular 'if' statement does just fine) but the
reporting.  Again, this is motivated by the fact that I expect that
flexible filtering is essential for a successful warning mechanism.

>     "cls" may be any callable object that takes a list as a single
>     argument argument list and returns an object with the required
>     attributes "get_action" and "format_message"
>     
>        * get_action() -> "warn"|"error"|"suppress"
>        * format_message() -> string
> 
>     A provided base class implements these methods in a reusable
>     fashion. Warning creators are encouraged to merely subclass.

This is just a matter of exposition, but when I first read your PEP I
had a hard time figuring out the purpose of the cls object.  It wasn't
until I got to the very end where I saw your example classes that I
realized what it is: it represents a specific warning condition or a
group of related warning conditions.

>     This extended form of the assertion statement calls the assertion
>     handler code in the new "assertions" module. 

I won't harp on this each time, but I'd like to point out once more
that "assertion" is the wrong name for a warning feature.  Although it
isn't part of the Zen of Python (by Tim Peters), it should be: a
suggestive name for a feature is worth half a spec!

>     The semantics of the built-in assertion handler are defined by the
>     following code. It should be exposed in a new "assertions" module.
> 
>     def handle_assertion(cls, message = ""):
>         "This code is called when an assertion fails and cls is not None"
> 
>         obj = cls(message)
>         action = obj.get_action()
>             
>         if action=="error":
>             *** existing assertion code ***

That's just

              raise AssertionError, message

Right?

>         elif action=="warn":
>             sys.stderr.write(obj.format_message())
>         elif action=="suppress":
>             pass
>         else:
>             assert action in ["warn","error","suppress"]
> 
>     Even if handle_assertion is implemented in C, it should be exposed as
>     assertions.handle_assertion so that it may be overriden. 

Suggestion: have separate warning and error handlers, so that if I
want to override these branches of the if statement I don't have to
repeat the entire handler.

>     The generic warning base class is defined below:
> 
>     class Assertion:
>       def __init__(self, *args):
>           if len(args) == 1:
>               self.args = args[0]
>           else:
>               self.args = args
> 
>       def format_message(self):
>           sys.stderr.write("%s: %s" %(obj.__name__, self.args))
> 
>       def get_action(self):
>           return (self.get_user_request(self.__class__) 
>                   or sys.default_warning_action)
> 
>       def get_user_request(self, cls):
>           if cls.__name__ in sys.errors:
>               return "error"
>           elif cls.__name__ in sys.warnings:
>               return "warn"
>           elif cls.__name__ in sys.disabled_warnings:

I see no definition of sys.disabled_warnings.  Instead of
sys.disabled_warnings you meant sys.suppress, right?

>               return "suppress"
>           for base in cls.__bases__:
>               rc = self.get_user_request(base)
>               if rc:
>                   return rc
>           else:
>               return None

This approach (searching for the class name or the name of one of its
base classes in a list) doesn't look very object-oriented.  It would
make more sense to store the desired return value as a class or
instance attribute.  The default warning action could be stored on the
base class.

>     The Assertion class should be implemented in Python so that it can be 
>     used as a base class.
> 
>     Because this code inherently implements "warning state inheritance",
>     it would be rare to override any of the methods, but this is possible
>     in exceptional circumstances.
> 
> Command line
> 
>     By default the special variables have the following contents:
> 
>     sys.warnings = []
>     sys.errors = []
>     sys.suppress = []
>     sys.default_warning_action = "warn"
>     
>     These variables may be changed from the command line. The command line
>     arguments are interpreted as described below:
> 
>     -w XXX => sys.warnings.append("XXX")
>     -e XXX => sys.errors.append("XXX")
>     -no-w XXX => sys.suppress.append("XXX")
>     -wall => sys.default_warning_action => "warn"
>     -eall => sys.default_warning_action => "error"
>     -no-wall => sys.default_warning_action => "suppress"

Python doesn't support long options (I don't *like* long options so I
doubt that this is a good occasion to start lobbying for them :-).  We
can come up with different options though.

>     As per the code above, errors take precedence over warnings and
>     warnings over suppressions unless a particular assertion class
>     specifies otherwise.

I would use a different precedence scheme: a more specific filter
takes precedence over a more general filter.  So -eall -wdubious would
mean that "dubious" class warnings are warnings but all others are
errors, and -wall -edubious would mean the opposite.

> Built-in warning objects:
> 
>     class exception_in_del(Assertion):
>         "An exception was ignored in an __del__ method"
> 
>     class deprecation(Assertion):
>         "This feature may be removed in a future version of Python."
> 
>     class dubious(Assertion):
>         "There is a common error associated with this feature."
> 
>     These class definitions are part of the "Assertion" module. They
>     should only ever be used when there exists a way for the programmer to 
>     accomplish the same thing without triggering the warning. For instance
>     the way to suppress the deletion exception is to trap exceptions in
>     __del__ methods with a try/except block.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Mon Nov  6 22:36:49 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:49 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061628.LAA07574@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com>

[Guido]
> 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> Algol-60 did this.  Fortran does this.  And rational numbers are less
> innovative?

Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
new rules, int div always truncates).

So far as innovation goes, no choice on the table so far is innovative
(neither mathematically nor in programming languages), so there's no basis
for choosing there.

Guido, *why* did ABC use rationals by default?  Was that driven by usability
studies?





From tim_one at email.msn.com  Mon Nov  6 22:36:50 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Mon, 6 Nov 2000 16:36:50 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: <200011061749.MAA08063@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPHHOAA.tim_one@email.msn.com>

[Guido]
> ABC's forcing was as simple as writing ~x or mixing inexact numbers.
> We did have the notion that 1.0 was an exact literal (to me it looks
> exact!) so maybe that was an additional problem.

I seriously wonder whether that was *the* problem with ABC:  not only was
1.0 treated as an exact rational in ABC, so was 6.02e23 and 3.14159e-314
etc.  At least for me, this caused rationals to get used in many places I
*intended* to use floats.  I assume many others got burned by this too, as
I'd say it's impossible for users coming from other languages not to see
6.02e23 etc as float literals.





From paulp at ActiveState.com  Mon Nov  6 23:01:27 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Mon, 06 Nov 2000 14:01:27 -0800
Subject: [Python-Dev] Warnings PEP
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>
Message-ID: <3A072A37.C2CAFED1@activestate.com>

Guido van Rossum wrote:
> 
> Paul, thanks for submitting a warnings framework.  I'd like to give
> some feedback, comparing it to my own proposal.  Please also give
> explicit feedback on mine!

Okay, but I'm a little bit worried about getting into a science project
-- in terms of my time to work on it. I'm perfectly happy with the two
requirements you mentioned but I haven't thought enough about filters to
be able to whip off ideas about them quickly.

> What's missing here is a set of requirements.  Comparing your proposal
> to my requirements, I find that you are not addressing two
> requirements that are important in my mind: a convenient and efficient
> C API, and a mechanism that prints a warning message only the first
> time that the warning is issued.  

I agree that these are both important things and I will add them to my
proposal. Also, I like your requirement that the target for warnings
should be easily customizable.

> Warnings are more a human engineering issue than a technical issue!
> That's also why I am emphasizing a C API -- I want to make it real
> easy to ussue quality warnings in the runtime.  It's also why I
> specify a rich (probably too rich!) filtering mechanism.

Let me put out the strawman proposal that "grep" is a nicely orthogonal
filtering mechanism. <duck!>

I guess my thinking is that you turn off warnings at the source. If you
get a warning about __del__ exceptions then you put in a try/except. If
you get a warning about an unused variable then you assign it to itself.
If you get a warning about integer division then you should pass in a
float and so forth.

> > Syntax
> >
> >     assert >> cls, test[[[, arg], arg]...]
> 
> I have several problems with this.  First of all, using "assert" means
> that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> think that the decision to disable all warnings should be independent
> from the decision to "optimize".  

Arguable. I see it as "release" and "debug" configurations. python and
python_d.

> Second, you're hypergeneralizing the
> extended print syntax.  Just because I think it's okay to add >>file
> to the print syntax doesn't mean that it's now okay to add >>object
> syntax to all statements!

Well getting new syntax into Python is really, really hard so we've got
to squeeze out as much value out of what we have as possible. But
anyhow, assertions are not allowed to 

You and I agree that there is an important sociological dimension to
this. We can't require:

  import warnings

  warnings.warn("foo", "bar")

I prefer:

  warn Foo, "Bar"

just like:

  raise Foo, "Bar"

> I also don't see what warnings have to do with assertions.  Assertions
> are a mechanism to check for error conditions.  What happens if the
> error is detected is of less importance -- it could raise an exception
> (Python), issue a fatal error (C), or do nothing (in -O mode).

Warnings are issued when an error or dubious construct is detected.
Assertions are "fatal warnings". You agree that it is appropriate for
some "warnings" to kill the app in some circumstances. Isn't it just a
hop-skip-and-a-jump to say that warnings and errors are just points on
the spectrum:

Report Once
Report Always
Report and Die

Python trained me to think of function calls and object constructions as
being basically the same thing -- and keyword arguments and variable
names being basically the same thing. etc.

> With warnings I believe the issue is not so much the detection of the
> condition (for which a regular 'if' statement does just fine) but the
> reporting.  Again, this is motivated by the fact that I expect that
> flexible filtering is essential for a successful warning mechanism.

I don't see why assertions need special syntax but warnings do not! I
would have been happy for a "warn" keyword with assert-like syntax but I
don't think I'll see that in my lifetime.

> This is just a matter of exposition, but when I first read your PEP I
> had a hard time figuring out the purpose of the cls object.  It wasn't
> until I got to the very end where I saw your example classes that I
> realized what it is: it represents a specific warning condition or a
> group of related warning conditions.

It sort of evolved into an exception-like mechanism in that the class is
instantiated with arguments just as exception classes are.

> >         if action=="error":
> >             *** existing assertion code ***
> 
> That's just
> 
>               raise AssertionError, message
> 
> Right?

Well I'd like the traceback to emanate from the caller's position not
the warning handler's position. Python doesn't really have a way to say
that simply. This may well be implemented in C so it might not matter.

> Suggestion: have separate warning and error handlers, so that if I
> want to override these branches of the if statement I don't have to
> repeat the entire handler.

Good idea.

> I see no definition of sys.disabled_warnings.  Instead of
> sys.disabled_warnings you meant sys.suppress, right?

Right.

> >               return "suppress"
> >           for base in cls.__bases__:
> >               rc = self.get_user_request(base)
> >               if rc:
> >                   return rc
> >           else:
> >               return None
> 
> This approach (searching for the class name or the name of one of its
> base classes in a list) doesn't look very object-oriented.  It would
> make more sense to store the desired return value as a class or
> instance attribute.  The default warning action could be stored on the
> base class.

The way I came to this odd structure is I wanted most subclasses to be
just no-ops as many exceptions are:

class ActiveStateLintWarning(Warning): pass

class CodeLooksLikePerl(ActiveStateLintWarning): pass

class TooMuchOnOneLine(CodeLooksLikePerl): pass

So what __init__ could I write that would look up -wTooMuchOnOneLine and
then if it failed that, look up -wCodeLooksLikePerl and so forth?

It gets pretty mind-bending because you sort of want one method to call
another and yet you want it to be the *same* inherited method (i.e.
don't have to code it each time). E.g. you want to run the *same method*
at each level of the hierarchy. So I just do that directly.

> >     -w XXX => sys.warnings.append("XXX")
> >     -e XXX => sys.errors.append("XXX")
> >     -no-w XXX => sys.suppress.append("XXX")
> >     -wall => sys.default_warning_action => "warn"
> >     -eall => sys.default_warning_action => "error"
> >     -no-wall => sys.default_warning_action => "suppress"
> 
> Python doesn't support long options (I don't *like* long options so I
> doubt that this is a good occasion to start lobbying for them :-).  We
> can come up with different options though.

Are these long options? Or just overloaded behavior on -w, -e, -n .
Think of "all" as the base class for warnings or something.

> >     As per the code above, errors take precedence over warnings and
> >     warnings over suppressions unless a particular assertion class
> >     specifies otherwise.
> 
> I would use a different precedence scheme: a more specific filter
> takes precedence over a more general filter.  So -eall -wdubious would
> mean that "dubious" class warnings are warnings but all others are
> errors, and -wall -edubious would mean the opposite.

Good idea. I think the code may work that way already. :)

 Paul Prescod



From guido at python.org  Mon Nov  6 23:16:29 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 17:16:29 -0500
Subject: [Python-Dev] Revamping Python's Numeric Model
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:36:49 EST."
             <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCOEPGHOAA.tim_one@email.msn.com> 
Message-ID: <200011062216.RAA09669@cj20424-a.reston1.va.home.com>

> [Guido]
> > 1/2 yielding 0.5 is innovative?  Give me a break.  Pascal did this.
> > Algol-60 did this.  Fortran does this.  And rational numbers are less
> > innovative?

[Tim]
> Small correction:  Fortran does not -- 1/2 is 0 in Fortran (same as C99's
> new rules, int div always truncates).

I stand corrected -- the idea is only 40 years old, not 44. :-)

> So far as innovation goes, no choice on the table so far is innovative
> (neither mathematically nor in programming languages), so there's no basis
> for choosing there.
> 
> Guido, *why* did ABC use rationals by default?  Was that driven by usability
> studies?

I assume that the use of rationals for exact numbers was driven by
usability studies -- like us, the ABC designers were tired of
explaining the vagaries of floating point to novices.

I remember that I pushed for using rationals for 1E1000 and 1E-1000,
probably out of a mistaken sense of consistency.  I don't think I'm
responsible for 1.0 being exact -- in "The B Programmer's Handbook"
(CWI, 1985) 1.0 is exact and 1E10 is approximate.  In "The ABC
Progammer's Handbook (Prentice Hall, 1990) these are all exact.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From martin at loewis.home.cs.tu-berlin.de  Tue Nov  7 01:49:34 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 01:49:34 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070049.BAA01485@loewis.home.cs.tu-berlin.de>

> Is there some API for it in C?

In C99, you have access to the floating-point environment:

               #include <fenv.h>
               /* ... */
               {
                       #pragma STDC FENV_ACCESS ON
                       int set_excepts;
                       feclearexcept(FE_INEXACT | FE_OVERFLOW);
                       // maybe raise exceptions
                       set_excepts = fetestexcept(FE_INEXACT | FE_OVERFLOW);
                       if (set_excepts & FE_INEXACT) f();
                       if (set_excepts & FE_OVERFLOW) g();
                       /* ... */
               }

It defines the following symbolic exception constants:

               FE_DIVBYZERO
               FE_INEXACT
               FE_INVALID
               FE_OVERFLOW
               FE_UNDERFLOW

Few compilers support that, though.

Regards,
Martin



From greg at cosc.canterbury.ac.nz  Tue Nov  7 02:06:04 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 14:06:04 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
Message-ID: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>

> It's a new keyword though, which has a much
> higher threshold for acceptance than a new two-character operator
> symbol.

It could be non-reserved, since a div b is currently
a syntax error.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From martin at loewis.home.cs.tu-berlin.de  Tue Nov  7 02:13:04 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 7 Nov 2000 02:13:04 +0100
Subject: [Python-Dev] Revamping Python's Numeric Model
Message-ID: <200011070113.CAA01767@loewis.home.cs.tu-berlin.de>

> I seriously wonder whether that was *the* problem with ABC: not only
> was 1.0 treated as an exact rational in ABC, so was 6.02e23 and
> 3.14159e-314 etc.  At least for me, this caused rationals to get
> used in many places I *intended* to use floats.  I assume many
> others got burned by this too, as I'd say it's impossible for users
> coming from other languages not to see 6.02e23 etc as float
> literals.

There seems to be a long tradition in Python of annotating literals to
get them interpreted in a different way; I think it would be
reasonable to tell apart floating point literals and rational literals
(with a power-of-ten denominator).

Specifically, the "scientific notation" could be used: 1.1 would be
exactly the same as 11/10, 1.1e0 would be binary floating point, and
only approximately equal to 11/10.

Regards,
Martin



From guido at python.org  Tue Nov  7 04:07:49 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 06 Nov 2000 22:07:49 -0500
Subject: [Python-Dev] Warnings PEP
In-Reply-To: Your message of "Mon, 06 Nov 2000 14:01:27 PST."
             <3A072A37.C2CAFED1@activestate.com> 
References: <Pine.LNX.4.21.0011051522460.29249-100000@latte.ActiveState.com> <200011062102.QAA09294@cj20424-a.reston1.va.home.com>  
            <3A072A37.C2CAFED1@activestate.com> 
Message-ID: <200011070307.WAA10555@cj20424-a.reston1.va.home.com>

[Guido]
> > Warnings are more a human engineering issue than a technical issue!
> > That's also why I am emphasizing a C API -- I want to make it real
> > easy to ussue quality warnings in the runtime.  It's also why I
> > specify a rich (probably too rich!) filtering mechanism.

[Paul]
> Let me put out the strawman proposal that "grep" is a nicely orthogonal
> filtering mechanism. <duck!>
> 
> I guess my thinking is that you turn off warnings at the source. If you
> get a warning about __del__ exceptions then you put in a try/except. If
> you get a warning about an unused variable then you assign it to itself.
> If you get a warning about integer division then you should pass in a
> float and so forth.

I'm thinking of a different situation, where the user isn't very Unix
or Python sophisticated or the code generating warnings isn't easily
accessible for editing, or the fix isn't obvious (which I expect to be
often the case for __del__ and int division warnings).

Just as with exception handlers, where unqualified except clauses are
bad because of the risk that they mask real errors, I want to avoid
the likelihood that end users (those who can't or shouldn't fix the
errors) have to turn off all warnings.

> > > Syntax
> > >
> > >     assert >> cls, test[[[, arg], arg]...]
> > 
> > I have several problems with this.  First of all, using "assert" means
> > that in "optimizing" mode (python -O) you won't get *any* warnings.  I
> > think that the decision to disable all warnings should be independent
> > from the decision to "optimize".  
> 
> Arguable. I see it as "release" and "debug" configurations. python and
> python_d.
> 
> > Second, you're hypergeneralizing the
> > extended print syntax.  Just because I think it's okay to add >>file
> > to the print syntax doesn't mean that it's now okay to add >>object
> > syntax to all statements!
> 
> Well getting new syntax into Python is really, really hard so we've got
> to squeeze out as much value out of what we have as possible. But
> anyhow, assertions are not allowed to 

(Did you forget to complete a sentence there?)

I say there's no need for new syntax, this can just be a function.

> You and I agree that there is an important sociological dimension to
> this. We can't require:
> 
>   import warnings
> 
>   warnings.warn("foo", "bar")
> 
> I prefer:
> 
>   warn Foo, "Bar"
> 
> just like:
> 
>   raise Foo, "Bar"

Hm.  Raise was made a statement because (1) Modula-3 did so, and (2)
it is a flow control modifier.  You haven't made a good case for why
warn() can't be a function.

> > I also don't see what warnings have to do with assertions.  Assertions
> > are a mechanism to check for error conditions.  What happens if the
> > error is detected is of less importance -- it could raise an exception
> > (Python), issue a fatal error (C), or do nothing (in -O mode).
> 
> Warnings are issued when an error or dubious construct is detected.
> Assertions are "fatal warnings". You agree that it is appropriate for
> some "warnings" to kill the app in some circumstances. Isn't it just a
> hop-skip-and-a-jump to say that warnings and errors are just points on
> the spectrum:
> 
> Report Once
> Report Always
> Report and Die

Possibly -- and then 'assert' is a poor choice of name for the
feature.  Objection denied.

> Python trained me to think of function calls and object constructions as
> being basically the same thing -- and keyword arguments and variable
> names being basically the same thing. etc.
> 
> > With warnings I believe the issue is not so much the detection of the
> > condition (for which a regular 'if' statement does just fine) but the
> > reporting.  Again, this is motivated by the fact that I expect that
> > flexible filtering is essential for a successful warning mechanism.
> 
> I don't see why assertions need special syntax but warnings do not! I
> would have been happy for a "warn" keyword with assert-like syntax but I
> don't think I'll see that in my lifetime.

Indeed.  But binding arbitrary unrelated semantics to an existing
statement with a carefully chosen name is poor design too.  You might
as well propose print<<file as a new input statement: they're both I/O
related. :-)

> > This is just a matter of exposition, but when I first read your PEP I
> > had a hard time figuring out the purpose of the cls object.  It wasn't
> > until I got to the very end where I saw your example classes that I
> > realized what it is: it represents a specific warning condition or a
> > group of related warning conditions.
> 
> It sort of evolved into an exception-like mechanism in that the class is
> instantiated with arguments just as exception classes are.
> 
> > >         if action=="error":
> > >             *** existing assertion code ***
> > 
> > That's just
> > 
> >               raise AssertionError, message
> > 
> > Right?
> 
> Well I'd like the traceback to emanate from the caller's position not
> the warning handler's position. Python doesn't really have a way to say
> that simply. This may well be implemented in C so it might not matter.

OK.

> > Suggestion: have separate warning and error handlers, so that if I
> > want to override these branches of the if statement I don't have to
> > repeat the entire handler.
> 
> Good idea.
> 
> > I see no definition of sys.disabled_warnings.  Instead of
> > sys.disabled_warnings you meant sys.suppress, right?
> 
> Right.
> 
> > >               return "suppress"
> > >           for base in cls.__bases__:
> > >               rc = self.get_user_request(base)
> > >               if rc:
> > >                   return rc
> > >           else:
> > >               return None
> > 
> > This approach (searching for the class name or the name of one of its
> > base classes in a list) doesn't look very object-oriented.  It would
> > make more sense to store the desired return value as a class or
> > instance attribute.  The default warning action could be stored on the
> > base class.
> 
> The way I came to this odd structure is I wanted most subclasses to be
> just no-ops as many exceptions are:
> 
> class ActiveStateLintWarning(Warning): pass
> 
> class CodeLooksLikePerl(ActiveStateLintWarning): pass
> 
> class TooMuchOnOneLine(CodeLooksLikePerl): pass

The idea of using a class hierarchy to classify warnings is definitely
a good one.

> So what __init__ could I write that would look up -wTooMuchOnOneLine and
> then if it failed that, look up -wCodeLooksLikePerl and so forth?

Yeah, it's not clear what to do.  You would like the -w option to poke
a value in a class variable, but the problem there is that it doesn't
know in which module the class is defined.  (On the other hand, if
that problem could be solved, it would be the preferred solution --
since it would solve the problem of typos in the -w argument neatly.)

> It gets pretty mind-bending because you sort of want one method to call
> another and yet you want it to be the *same* inherited method (i.e.
> don't have to code it each time). E.g. you want to run the *same method*
> at each level of the hierarchy. So I just do that directly.
> 
> > >     -w XXX => sys.warnings.append("XXX")
> > >     -e XXX => sys.errors.append("XXX")
> > >     -no-w XXX => sys.suppress.append("XXX")
> > >     -wall => sys.default_warning_action => "warn"
> > >     -eall => sys.default_warning_action => "error"
> > >     -no-wall => sys.default_warning_action => "suppress"
> > 
> > Python doesn't support long options (I don't *like* long options so I
> > doubt that this is a good occasion to start lobbying for them :-).  We
> > can come up with different options though.
> 
> Are these long options? Or just overloaded behavior on -w, -e, -n .
> Think of "all" as the base class for warnings or something.

Yes: if you change -no-w into -n, they can all be short options.  Note
that no matter what syntax we choose, we'll always be deviants
compared to Perl and GCC: those require a -w or -W option to *enable*
warnings.  (I found some great quotes in the Perl man page:

    Whenever you get mysterious behavior, try the -w switch!!!  Whenever
    you don't get mysterious behavior, try using -w anyway.

    The -w switch produces some lovely diagnostics.

    Did we mention that you should definitely consider using the -w
    switch?

    Bugs
        The -w switch is not mandatory.
)

> > >     As per the code above, errors take precedence over warnings and
> > >     warnings over suppressions unless a particular assertion class
> > >     specifies otherwise.
> > 
> > I would use a different precedence scheme: a more specific filter
> > takes precedence over a more general filter.  So -eall -wdubious would
> > mean that "dubious" class warnings are warnings but all others are
> > errors, and -wall -edubious would mean the opposite.
> 
> Good idea. I think the code may work that way already. :)

Indeed it does. :-)  Then what did you mean by your remark about the
precedence?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Tue Nov  7 05:03:59 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:03:59 +1300 (NZDT)
Subject: [Python-Dev] Warnings PEP
In-Reply-To: <3A072A37.C2CAFED1@activestate.com>
Message-ID: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>

Paul Prescod <paulp at ActiveState.com>:

> Assertions are "fatal warnings".

No, the failure of an assertion *causes* a fatal warning.
Assertions themselves are tests, not warnings.

I agree with Guido -- "assert" is the wrong verb, "warn"
is the right verb.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fdrake at users.sourceforge.net  Tue Nov  7 05:05:34 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Mon, 6 Nov 2000 20:05:34 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011070405.UAA32052@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From py-dev at zadka.site.co.il  Tue Nov  7 13:24:22 2000
From: py-dev at zadka.site.co.il (Moshe Zadka)
Date: Tue, 07 Nov 2000 14:24:22 +0200
Subject: [Python-Dev] Re: Class/type dichotomy thoughts 
In-Reply-To: Message from "M.-A. Lemburg" <mal@lemburg.com> 
   of "Mon, 06 Nov 2000 21:01:40 +0100." <3A070E24.B07FA7D7@lemburg.com> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060244.VAA04289@cj20424-a.reston1.va.home.com> <3A06A336.5FD1EBC7@lemburg.com> <20001105234453.A8255@glacier.fnational.com> <3A06E43B.D95EC267@lemburg.com> <20001106023529.B8639@glacier.fnational.com>  <3A070E24.B07FA7D7@lemburg.com> 
Message-ID: <E13t7ne-0005wj-00@darjeeling.zadka.site.co.il>

[MAL]
> Take e.g. dictionaries: you could easily add a new dictionary
> type which uses case-insensitive string keys by extending the
> existing dictionary type.
> 
> The new type would reuse most of the slots of the original
> type and only replace the ones needed for lookup with the
> new logic for case-insensitivity.
> 
> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object. The
> underlying type objects would be different though (and also
> the type object address which is currently used to identify
> a builtin type).

This kind of thing will also help a numeric system like PEP-0228
be implemented more easily.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From greg at cosc.canterbury.ac.nz  Tue Nov  7 05:19:46 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Nov 2000 17:19:46 +1300 (NZDT)
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
In-Reply-To: <3A070E24.B07FA7D7@lemburg.com>
Message-ID: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>

"M.-A. Lemburg" <mal at lemburg.com>:

> Then it sets the type id to PyDict_TypeID and Python will
> use it as if it were an original dictionary object.

Hang on a minute. What sort of assumptions is the
interpreter going to be making based on the fact that
the type id is PyDict_TypeID? Can it be sure that this
new case-insensitive dictionary doesn't break them
somehow?

In other words, what does this new type_id thing
actually *mean*?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From paul at prescod.net  Tue Nov  7 06:57:40 2000
From: paul at prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 21:57:40 -0800
Subject: [Python-Dev] Warnings PEP
References: <200011070403.RAA00158@s454.cosc.canterbury.ac.nz>
Message-ID: <3A0799D4.86FC8148@prescod.net>

Greg Ewing wrote:
> 
> Paul Prescod <paulp at ActiveState.com>:
> 
> > Assertions are "fatal warnings".
> 
> No, the failure of an assertion *causes* a fatal warning.
> Assertions themselves are tests, not warnings.

Fine, some assertions are tests for fatal errors and some assertions are
tests for non-fatal mistakes.
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From moshez at math.huji.ac.il  Tue Nov  7 07:37:01 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:37:01 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <87ofzs39lo.fsf@indy.progeny.com>
Message-ID: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>

On 6 Nov 2000, Eric Gillespie, Jr. wrote:

> There has been a problem in Debian's python-gdk-imlib package for
> quite some time: Transparent PNGs do not display properly (see
> attached example script). According to upstream
> (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> the proper solution is either to have Python use RTLD_GLOBAL in
> dlopen calls when loading extension modules,

Or, possible, using dlmodule to dlopen things ourselves rather then
using Python's import facilities for that.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Tue Nov  7 07:41:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:41:56 +0200 (IST)
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011070841110.13407-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> 
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

Oooops....I didn't mean to send it here (well, I did, then I change
my mind but forgot to tell my e-mail program about that)

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From paul at prescod.net  Tue Nov  7 07:54:14 2000
From: paul at prescod.net (Paul Prescod)
Date: Mon, 06 Nov 2000 22:54:14 -0800
Subject: [Python-Dev] Warning framework
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com>  
	            <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>
Message-ID: <3A07A716.BDF9658B@prescod.net>

Guido van Rossum wrote:
> 
> Before I fall asleep let me write up my ideas about the warning
> framework.
> 
> Requirements:
> 
> - A C-level API that lets C code issue a warning with a single call
>   taking one or two arguments, e.g. Py_Warning(level, message).  (The
>   'level' argument is an example only; I'm not sure what if any we
>   need.)

I like the C-level API.

I think the "level" argument should be some sort of characterization,
not a number or enumeration. I think of it as being like an exception --
just another Python object.

Or it could be just a string. The main point is that filtering on a
number or enum is not flexible enough.

> - An equivalent Python level API, e.g. sys.warning(level, message).

I would prefer something easier to spell and with more of a central "you
should use this alot" feeling.

> Possible implementation:
> 
> - Each module can has a dictionary __warnings__ in its global
>   __dict__, which records the state of warnings.  It is created as an
>   emprt dict if it doesn't exist when it is needed.  The keys are
>   (message, linenumber) tuples (the module or file is implicit through
>   the use of the module's __dict__).  The value is None if no more
>   action is needed for this particular warning and location.  Some
>   other values may indicate the options "always print warning" (1?)
>   and "raise an exception" (-1?).

The problem with filtering based on line number is that it can't really
be done in a static manner because it is too fragile to code changes. In
my proposal every warning was assigned a "type" which would be the key
for filtering. A string would also be fine.

In general, I'm uncomfortable that I don't understand the requirements
enough. Are warnings something that the average Python programmer sees
rarely and then immediately goes in to fix the code so they don't see it
anymore (as compiler warnings are often handled)? Or do we expect most
Python programs to issue hundreds of warnings unless they are filtered.
Is filtering something you do constantly or as a partial workaround for
half-broken code that you can't fix right now?
-- 
 Paul Prescod
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From moshez at math.huji.ac.il  Tue Nov  7 07:58:40 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 08:58:40 +0200 (IST)
Subject: [Python-Dev] Warning framework
In-Reply-To: <3A07A716.BDF9658B@prescod.net>
Message-ID: <Pine.GSO.4.10.10011070849580.13407-100000@sundial>

On Mon, 6 Nov 2000, Paul Prescod wrote:

> I like the C-level API.
> 
> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.

These are completely different things -- both should be there. An
exception doesn't need a level -- it's the highest level possible, saying
"get out of here, fast!". Warnings need both a level and characterization.
Having it a Python class is a nice touch, and keeps it consistent with
the way exceptions use classes for characterization.

> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

There are two main sources to copy from here:

gcc: You have -Wall, -Wadd-something, etc. Any warning you do see you
     either fix, or surround with a pragma so you don't see this.
     You also have -Werror to turn all warnings into errors.
Perl: -w gives runtime warnings for things any saner language would
      raise exceptions for. "dereferencing NULL", accessing non-existing
      elements in an array, etc. Warnings are serious bugs, and you must
      always use them.
Perl: "use strict" and friends: die because of some otherwise perfectly
      legal Perl if it's not declared properly.

I'd go for a more gcc-like approach: if you see a warning, you should
either 1. Silence it or 2. Fix it.
Silencing warnings is a serious issue: sometimes the programmer does
no better then the interpreter, and should have the ability to silence
any warning permanently -- otherwise he'll work with -silence_all, and
the benefit is lost.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From tim_one at email.msn.com  Tue Nov  7 09:38:35 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Tue, 7 Nov 2000 03:38:35 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011060234.VAA04271@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBCHPAA.tim_one@email.msn.com>

[Guido]
> ...
> Note: it is open for debate whether the result of x/y for integer (or
> long integer) arguments should yield an integer (or long integer) in
> those cases where the result *is* representable as such (e.g. 4/2).
> It is possible that the numeric tower will render this problem moot --
> but that depends on what happens to Moshe's PEP 228, and that's a much
> longer story.

Note that for long ints i and j, i/j may (& easily so) be too large to
represent as a native double.  Rationals (of course) have no problem with
that.  It would certainly be curious if i*j didn't overflow but i/j did.

Just an observation -- I'm a fan of unintended consequences <wink>.





From mal at lemburg.com  Tue Nov  7 10:17:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 10:17:56 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <200011070419.RAA00162@s454.cosc.canterbury.ac.nz>
Message-ID: <3A07C8C4.1F14985F@lemburg.com>

Greg Ewing wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com>:
> 
> > Then it sets the type id to PyDict_TypeID and Python will
> > use it as if it were an original dictionary object.
> 
> Hang on a minute. What sort of assumptions is the
> interpreter going to be making based on the fact that
> the type id is PyDict_TypeID? Can it be sure that this
> new case-insensitive dictionary doesn't break them
> somehow?
> 
> In other words, what does this new type_id thing
> actually *mean*?

For the interpreter it means that it can assume the type
interface to be binary compatible to the "original"
type, e.g. by setting the flag to say PyDict_TypeID
the type assures that all PyDict_*() APIs will work
on the type -- basically the same thing as PyDict_Check()
does now except that the type object needn't be the same
anymore.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Tue Nov  7 10:36:42 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 10:36:42 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
References: <200011070911.BAA13249@slayer.i.sourceforge.net>
Message-ID: <01e301c0489e$3e349540$0900a8c0@SPIFF>

Moshe Zadka wrote:
> Modified Files:
> pep-0216.txt 
> Log Message:
> Added structured text consensus.

when/where was this discussed?

</F>




From moshez at math.huji.ac.il  Tue Nov  7 10:49:54 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:49:54 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0216.txt,1.3,1.4
In-Reply-To: <01e301c0489e$3e349540$0900a8c0@SPIFF>
Message-ID: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>

On Tue, 7 Nov 2000, Fredrik Lundh wrote:

> Moshe Zadka wrote:
> > Modified Files:
> > pep-0216.txt 
> > Log Message:
> > Added structured text consensus.
> 
> when/where was this discussed?

ummmm....doc-sig, where all things documentation-related are discussed?

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From moshez at math.huji.ac.il  Tue Nov  7 10:58:41 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 7 Nov 2000 11:58:41 +0200 (IST)
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:
 python/nondist/peps pep-0216.txt,1.3,1.4)
In-Reply-To: <Pine.GSO.4.10.10011071149300.13407-100000@sundial>
Message-ID: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>

On Tue, 7 Nov 2000, Moshe Zadka wrote:

> ummmm....doc-sig, where all things documentation-related are discussed?

Just to clarify my answer: I'm happy to receive comments/complaints
by private e-mail, or you can share thm with all of doc-sig. I prefer
to be CCed on doc-sig e-mail, but if not, I'll read it in the doc-sig.

Python-Dev does not seem the place for documentation-related discussion,
since that's exactly what doc-sig is for. It's quite low-volume, so
please join, or browse the archives.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fredrik at pythonware.com  Tue Nov  7 11:59:38 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 7 Nov 2000 11:59:38 +0100
Subject: PEP-0216 (was Re: [Python-Dev] Re: [Python-checkins] CVS:python/nondist/peps pep-0216.txt,1.3,1.4)
References: <Pine.GSO.4.10.10011071155060.22569-100000@sundial>
Message-ID: <027f01c048a9$d324efa0$0900a8c0@SPIFF>

moshe wrote:
> Python-Dev does not seem the place for documentation-related discussion,
> since that's exactly what doc-sig is for. It's quite low-volume, so
> please join, or browse the archives.

oh, I thought I was subscribed.  guess I wasn't.

I'll check the archives.

</F>




From guido at python.org  Tue Nov  7 13:55:26 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 07:55:26 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: Your message of "Mon, 06 Nov 2000 22:54:14 PST."
             <3A07A716.BDF9658B@prescod.net> 
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com>  
            <3A07A716.BDF9658B@prescod.net> 
Message-ID: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>

> I think the "level" argument should be some sort of characterization,
> not a number or enumeration. I think of it as being like an exception --
> just another Python object.
> 
> Or it could be just a string. The main point is that filtering on a
> number or enum is not flexible enough.

OK, let's make this a class then.  Just keep exceptions out of it
-- this is a separate, disjoint set of classes.  Let's call this
"warning category".  There will be standard categories and user code
can add categories.

> > - An equivalent Python level API, e.g. sys.warning(level, message).
> 
> I would prefer something easier to spell and with more of a central "you
> should use this alot" feeling.

OK, let's make this a built-in: warning(category, message).

> > Possible implementation:
> > 
> > - Each module can has a dictionary __warnings__ in its global
> >   __dict__, which records the state of warnings.  It is created as an
> >   emprt dict if it doesn't exist when it is needed.  The keys are
> >   (message, linenumber) tuples (the module or file is implicit through
> >   the use of the module's __dict__).  The value is None if no more
> >   action is needed for this particular warning and location.  Some
> >   other values may indicate the options "always print warning" (1?)
> >   and "raise an exception" (-1?).
> 
> The problem with filtering based on line number is that it can't really
> be done in a static manner because it is too fragile to code changes. In
> my proposal every warning was assigned a "type" which would be the key
> for filtering. A string would also be fine.
> 
> In general, I'm uncomfortable that I don't understand the requirements
> enough. Are warnings something that the average Python programmer sees
> rarely and then immediately goes in to fix the code so they don't see it
> anymore (as compiler warnings are often handled)? Or do we expect most
> Python programs to issue hundreds of warnings unless they are filtered.
> Is filtering something you do constantly or as a partial workaround for
> half-broken code that you can't fix right now?

All of the above.

If I'm a programmer maintaining a piece of code, of course I turn all
warnings into errors and fix them as they occur.

But if I'm using someone else's code that happens to generate
warnings, it's better to disable warnings in that code until its
author has released a fixed version.  I want to be able to be very
specific, so that turning off warnings in 3rd party code doesn't
disable them in my own code.  If the 3rd party code generates a single
warning during normal use at a single location (e.g. if there's one
unconverted integer divide somehwere) then it's best to turn it off
just at that location, so that when I feed it other data which may
trigger other warnings elsewhere I will still get the benefit of the
warnings -- which *may* mean there's something wrong with my data, not
with the 3rd party code.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Tue Nov  7 14:17:02 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 14:17:02 +0100
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Tue, Nov 07, 2000 at 02:06:04PM +1300
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
Message-ID: <20001107141702.H27208@xs4all.nl>

On Tue, Nov 07, 2000 at 02:06:04PM +1300, Greg Ewing wrote:
> > It's a new keyword though, which has a much
> > higher threshold for acceptance than a new two-character operator
> > symbol.

> It could be non-reserved, since a div b is currently
> a syntax error.

Except for the fact our current parser can't handle the 'a div b' syntax
without making 'div' a reserved word, which also makes 'x.div', 'class div:'
and 'def div():' invalid syntax. It might be work around-able, but .... :P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From jim at interet.com  Tue Nov  7 14:28:16 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Tue, 07 Nov 2000 08:28:16 -0500
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com>
Message-ID: <3A080370.30C36339@interet.com>

"M.-A. Lemburg" wrote:

> Now I get this error after working in interactive Python
> mode with zipfile:
> 
> Exception exceptions.AttributeError:
> "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored

Reading the code, I don't see how this could have happened
unless __init__ has already raised an exception.
 
> I would like a method .copy(self, name, output) which
> reads the file name from the ZIP archive and writes it directly to
> the file-like object output. This should copy the file in chunks
> of say 64kB in order to reduce memory load.

This is only a few lines of Python, and I generally omit
any methods which are not absoultely necessary.  Does
anyone else think this should be added?

JimA



From guido at python.org  Tue Nov  7 14:33:46 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 08:33:46 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: Your message of "Tue, 07 Nov 2000 14:17:02 +0100."
             <20001107141702.H27208@xs4all.nl> 
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com> <200011070106.OAA00139@s454.cosc.canterbury.ac.nz>  
            <20001107141702.H27208@xs4all.nl> 
Message-ID: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>

> > > It's a new keyword though, which has a much
> > > higher threshold for acceptance than a new two-character operator
> > > symbol.
> 
> > It could be non-reserved, since a div b is currently
> > a syntax error.
> 
> Except for the fact our current parser can't handle the 'a div b' syntax
> without making 'div' a reserved word, which also makes 'x.div', 'class div:'
> and 'def div():' invalid syntax. It might be work around-able, but .... :P

No, we *could* use the 'import as' trick: define the syntax as

term: factor (('*'|'/'|'%'|NAME) factor)*

and add a check that NAME is "div" in the compiler.

But I don't know how comfy I am with a proliferation of hacks like
this -- and it's likely to cause more confusing error messages.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Tue Nov  7 15:46:15 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 07 Nov 2000 15:46:15 +0100
Subject: [Python-Dev] zipfile.py and InfoZIP files
References: <3A030D21.413B8C8A@lemburg.com> <3A06ADFD.5C735BBA@interet.com> <3A06B981.B3D73091@lemburg.com> <3A080370.30C36339@interet.com>
Message-ID: <3A0815B7.BD0D9C4C@lemburg.com>

"James C. Ahlstrom" wrote:
> 
> "M.-A. Lemburg" wrote:
> 
> > Now I get this error after working in interactive Python
> > mode with zipfile:
> >
> > Exception exceptions.AttributeError:
> > "ZipFile instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 0x824524c> ignored
> 
> Reading the code, I don't see how this could have happened
> unless __init__ has already raised an exception.

Probably has something to do with GC and cleaning up instances --
don't know. The error only shows up sometimes...

Instead of seeing the __del__ exception I now get the previous error
again:
zlib.error: Error -3 while decompressing: incomplete dynamic bit lengths tree

Nevermind... my setup must be broken in more way than I have
time to figure out now :-(
 
> > I would like a method .copy(self, name, output) which
> > reads the file name from the ZIP archive and writes it directly to
> > the file-like object output. This should copy the file in chunks
> > of say 64kB in order to reduce memory load.
> 
> This is only a few lines of Python, and I generally omit
> any methods which are not absoultely necessary.  Does
> anyone else think this should be added?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From pf at artcom-gmbh.de  Tue Nov  7 15:59:01 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Tue, 7 Nov 2000 15:59:01 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 7, 2000  8:33:46 am"
Message-ID: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum:
> No, we *could* use the 'import as' trick: define the syntax as
> 
> term: factor (('*'|'/'|'%'|NAME) factor)*
> 
> and add a check that NAME is "div" in the compiler.
> 
> But I don't know how comfy I am with a proliferation of hacks like
> this -- and it's likely to cause more confusing error messages.

In Modula-2 it is forbidden to declare a variable or procedure called
'DIV' or 'MOD', since both were reserved words in this language from
the very beginning.  But in Python the situation is different and
people might have used 'div' as an identifier.  So unless Guido is
able to fix this using the famous time machine ...sigh... for the
sake of backward compatibility using this "hack" seems to be the best
available choice.

I believe confusing error messages can be avoided and I see no 
"proliferation of hacks" in this two attempts to avoid defining
new keywords.  Keeping backward compatibility always had its price.
But it is often a price worth to pay.

Just my $0.02, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From gward at mems-exchange.org  Tue Nov  7 16:22:51 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Tue, 7 Nov 2000 10:22:51 -0500
Subject: [Python-Dev] Warning framework
In-Reply-To: <200011071255.HAA13355@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 07, 2000 at 07:55:26AM -0500
References: <200011032301.SAA26346@cj20424-a.reston1.va.home.com> <3A03E340.38F78FA2@lemburg.com> <200011060335.WAA04452@cj20424-a.reston1.va.home.com> <3A07A716.BDF9658B@prescod.net> <200011071255.HAA13355@cj20424-a.reston1.va.home.com>
Message-ID: <20001107102251.A15674@ludwig.cnri.reston.va.us>

On 07 November 2000, Guido van Rossum said:
> > Or it could be just a string. The main point is that filtering on a
> > number or enum is not flexible enough.
> 
> OK, let's make this a class then.  Just keep exceptions out of it
> -- this is a separate, disjoint set of classes.  Let's call this
> "warning category".  There will be standard categories and user code
> can add categories.

This sounds right -- I was going to suggest "warning class" instead of
"level", but "category" might be a better word.  My main rationale was
filtering: show me "integer divide" problems, but don't bother me with
"function argument not used".  (Hmm, those two sound more like specific
warnings rather than warning categories.  Probably the categories there
would be "arithmetic" and "dataflow".)

> > I would prefer something easier to spell and with more of a central "you
> > should use this alot" feeling.
> 
> OK, let's make this a built-in: warning(category, message).

Minor spelling nit: I would call it 'warn()' (or 'sys.warn()', or
'Py_Warn()', etc.) since that's a verb.

More importantly: if 'warn(ing)' is meant to be used mainly for
compiler-style warnings -- you're using this language or library feature
inappropriately -- then it should be left in sys.  But if it's meant to
also be used for printing some message to stderr (like Perl's 'warn'),
then there's a good case for making it a builtin.

Almost every Python script I write features

    def warn (msg):
        sys.stderr.write("warning: " + msg + "\n")

That might be a clue that something (albeit a tiny thing) is missing
from the language.  ;-)

        Greg



From fdrake at acm.org  Tue Nov  7 16:30:00 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 10:30:00 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
	<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
	<20001107141702.H27208@xs4all.nl>
	<200011071333.IAA13497@cj20424-a.reston1.va.home.com>
Message-ID: <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > But I don't know how comfy I am with a proliferation of hacks like
 > this -- and it's likely to cause more confusing error messages.

  If "div" is it, I'd rather see it made a keyword and a warning
published to the community soon so that people have a chance to check
their code and either make it compatible with the change or scream
ahead of time.
  A tool to help them out wouldn't hurt, either, and that could be
written before any actual changes are made or even final decisions are
made -- it could search everything on sys.path and report on uses that
would be affected by each candidate change.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Tue Nov  7 17:14:21 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 11:14:21 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."
             <3A06BEF4.95B773BD@tismer.com> 
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>  
            <3A06BEF4.95B773BD@tismer.com> 
Message-ID: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>

I've thought about it a little more, and drawn some pictures in my
head.

I still have to disagree with Christian when he says:

> Making Python completely coroutine aware, without
> tricking the C stack, is 90 percent of the problem.
> But after walking that far, there is no reason
> to leave the other 10 percent alone.

Without continuations, but with microthreads (uthreads) or coroutines,
each (Python) stack frame can simply be "paused" at a specific point
and continued later.  The semantics here are completely clear (except
perhaps for end cases such as unhandled exceptions and intervening C
stack frames).

With continuations, you have to decide how much state to save for a
future continuation.  It would seem easy enough: save all state kept
in the frame except for the local variables.  But now consider this:
the "evaluation stack" contained in each frame could easily be
replaced by a bunch of temporary variables, if we had a slightly
different instruction set (3-address opcodes instead of stack-based
opcodes).  Then would we save those temporary variables or not?  it
can make a difference!  Since the "save continuation" operation is a
function call, you can easily save a continuation while there are some
items on the value stack.  I believe the current implementation saves
these so they are restored when you jump to the continuation.  But if
the compiler were to use temporary variables instead of the evaluation
stack, they might not have been restored!

Here's another example.  Suppose you set up a for loop.  After three
iterations through the loop you save a continuation.  Then you finish
hree more iterations.  Then you return to the saved continuation.
Where does the loop continue: at 3 or at 6 iterations?  Try to answer
this without trying it.  My guess: it gets restarted at 3 iterations,
because the loop index is saved on the value stack.  If you rewrite
this using a while loop, however, it would get restarted at 6
iterations, because then your loop index is an unsaved local variable.
Ditto if you changed the bytecode compiler so for loops use an
anonymous local variable instead of an entry on the evaluation
stack.

This semantic imprecision is one of the reasons why I don't like the
concept of continuations.  (I've been told that the exact semantics of
continuations in Scheme differ greatly between Scheme implementations.)

Now let's look at Jython.  In Jython, we can simulate "paused frames"
well enough by using real Java threads.  However full continuations
would require modifications to the JVM -- which is unacceptable to a
language boasting "100% Pure Java".  Another reason against allowing
continuations.

So, all in all, I don't think of continuations as "the last 10% that
we might as well add to finish the job."  I see it as an ill-specified
hypergeneralization.

What *do* I want to see in a stackless PEP?

Not surprisingly, I'd like to see generators, coroutines, and
uthreads.  These all share a mechanism that pauses one frame and
resumes another.  I propose to make the concept of uthreads
fundamental -- this will simplify the emulation in Jython.


A strawman proposal:

The uthread module provides the new functionality at the lowest level.
Uthread objects represent microthreads.  An uthread has a chain of
stack frames linked by back pointers just like a regular thread.
Pause/resume operations are methods on uthread objects.  Pause/resume
operations do not address specific frames but specific uthreads;
within an uthread the normal call/return mechanisms can be used, and
only the top frame in the uthread's stack of call frames can be
paused/resumed (the ones below it are paused implicitly by the call to
the next frame, and resumed when that call returns).

- u = uthread.new(func) creates a new uthread object, u.  The new
  uthread is poised to call func() but doesn't execute yet.

- u = uthread.current() returns the uthread object for the current
  frame.

- u.yield() pauses the current uthread and resume the uthread u where
  it was paused.  The current uthread is resumed when some other
  uthread calls its yield() method.  Calling uthread.current().yield()
  is a no-op.

- When func() returns, the uthread that was executing it ceases to be
  runnable.  The uthread that most recently yielded to it is resumed,
  unless that is no longer runnable, in which case the uthread that
  most recently yielded to *it* is resumed, and so on until a runnable
  uthread is found or until no runnable uthreads are left, in which
  case the program terminates.  (XXX I need a proof here that this
  works.)

- When func() raises an unhandled exception, the exception gets
  propagated using the same rules as when it returns, and similarly
  its uthread ceases to be runnable.

- u.kill(exc) causes the yield() call that paused u to raise the
  exception exc.  (This can be caught in a try/except of course.)

- Calling u.yield() or u.kill() for a non-runnable uthread is an error
  and raises an exception.

I think this API should enough to implement Gordon's SelectDispatcher
code.  In general, it's easy to create a scheduler uthread that
schedules other uthreads.


Open issues:

- I'm not sure that I got the start conditions right.  Should func() be
  be allowed to run until its first yield() when uthread.new(func) is
  called?

- I'm not sure that the rules for returning and raising exceptions
  from func() are the right ones.

- Should it be possible to pass a value to another uthread by passing
  an argument to u.yield(), which then gets returned by the resumed
  yield() call in that uthread?

- How do uthreads interact with real threads?  Uthreads are explicitly
  scheduled through yield() calls; real threads use preemptive
  scheduling.  I suppose we could create a new "main" uthread for each
  real thread.  But what if we yield() to an uthread that's already
  executing in another thread?  How is that error detected?


Please help!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Tue Nov  7 17:30:16 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 10:30:16 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14856.11800.444274.851825@buffalo.fnal.gov>

 > Guido van Rossum:
 > > No, we *could* use the 'import as' trick: define the syntax as
 > > 
 > > term: factor (('*'|'/'|'%'|NAME) factor)*
 > > 
 > > and add a check that NAME is "div" in the compiler.
 > > 
 > > But I don't know how comfy I am with a proliferation of hacks like
 > > this -- and it's likely to cause more confusing error messages.

And what is the compelling reason for going through all this instead
of just using the '//' symbol?  Because it might be confused for a C++
comment?  This is a weak argument AFAIAC.  Python is not C++ and
everybody knows that.

I think that making "div" an infix operator would be setting a
horrible precedent.  Currently, all infix operators "look like"
operators, i.e. they are non-alphabetic characters, and things that
look like words are either functions or reserved words.

There's already a "divmod" builtin which is a function, not an infix
operator.  I think it would be rather inconsistent to write, on the
one hand:

    divmod(10, 2)

and on the other:

    10 div 2


Long before the creation of python-dev, this issue had been discussed
numerous times on c.l.py, and the '//' operator was suggested several
times, and I don't think anybody ever had a problem with it...



From gvwilson at nevex.com  Tue Nov  7 17:41:25 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Tue, 7 Nov 2000 11:41:25 -0500
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>

>  > Guido van Rossum:
>  > > No, we *could* use the 'import as' trick: define the syntax as
>  > > term: factor (('*'|'/'|'%'|NAME) factor)*
>  > > and add a check that NAME is "div" in the compiler.

> Charles G. Waldman:
> And what is the compelling reason for going through all this instead
> of just using the '//' symbol?  Because it might be confused for a C++
> comment?  This is a weak argument AFAIAC.  Python is not C++ and
> everybody knows that.
> Long before the creation of python-dev, this issue had been discussed
> numerous times on c.l.py, and the '//' operator was suggested several
> times, and I don't think anybody ever had a problem with it...

Greg Wilson:
As someone who teaches Python, I'm strongly opposed to using '//' in the
same language as '/', purely on readability grounds:

1. Every C/C++ book includes a warning about "=" vs. "==", because it's
   a common hard-to-spot error.

2. What mark would you give a student who had variables IO and I0 in the
   same module?

Greg

p.s. I was very disappointed to discover that Ruby uses both '..' and '...'
One means "up to but not including", the other means "up to and including".
It would be interesting to estimate the number of programmer-hours this will
cost... :-)




From thomas at xs4all.net  Tue Nov  7 18:26:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 7 Nov 2000 18:26:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <200011071544.HAA31147@slayer.i.sourceforge.net>; from gward@users.sourceforge.net on Tue, Nov 07, 2000 at 07:44:24AM -0800
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
Message-ID: <20001107182621.I27208@xs4all.nl>

On Tue, Nov 07, 2000 at 07:44:24AM -0800, Greg Ward wrote:

>   - when compiling with GCC on any platform, add "-fPIC" to OPT
>     (without this, "$(CC) -shared" dies horribly)

Sorry for the late remark (I did see your earlier message) but after reading
the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
for non-essential stuff: warnings, debug-info and optimizations. Removing
things from OPT shouldn't break anything, and neither should adding options
that fit in the categories above. (Barring broken compilers, of course.)

Instead, the -fPIC option should be added to CFLAGS, I think. The Python
autoconf setup is slightly less versatile than most, though, since it's
doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
without editing configure(.in) :P If noone else does it before me, I'll see
about fixing at least the -fPIC thing later, when I find some time ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From paulp at ActiveState.com  Tue Nov  7 18:29:10 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 09:29:10 -0800
Subject: [Python-Dev] Integer division transition
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
		<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
		<20001107141702.H27208@xs4all.nl>
		<200011071333.IAA13497@cj20424-a.reston1.va.home.com> <14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
Message-ID: <3A083BE6.4A502210@activestate.com>

"Fred L. Drake, Jr." wrote:
> 
> ...
>   A tool to help them out wouldn't hurt, either, and that could be
> written before any actual changes are made or even final decisions are
> made -- it could search everything on sys.path and report on uses that
> would be affected by each candidate change.

I think that the standard Python compiler is the appropriate tool for
this sort of thing. Anything that can be caught "statically" might as
well be implemented right in the compiler (at least from the user's
point of view) rather than in a separate "deprecation nanny."

 Paul Prescod



From nas at arctrix.com  Tue Nov  7 12:45:18 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 7 Nov 2000 03:45:18 -0800
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>; from gvwilson@nevex.com on Tue, Nov 07, 2000 at 11:41:25AM -0500
References: <14856.11800.444274.851825@buffalo.fnal.gov> <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <20001107034518.A12431@glacier.fnational.com>

On Tue, Nov 07, 2000 at 11:41:25AM -0500, Greg Wilson wrote:
> As someone who teaches Python, I'm strongly opposed to using '//' in the
> same language as '/', purely on readability grounds:

How do you feel about div(x, y)?

  Neil



From cgw at fnal.gov  Tue Nov  7 19:50:18 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Tue, 7 Nov 2000 12:50:18 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
References: <14856.11800.444274.851825@buffalo.fnal.gov>
	<NEBBIACCCGNFLECLCLMHKEBGCAAA.gvwilson@nevex.com>
Message-ID: <14856.20202.174151.647981@buffalo.fnal.gov>

Greg Wilson writes:
 > 
 > 2. What mark would you give a student who had variables IO and I0 in the
 >    same module?
 > 

I think this is a bit of a stretch - IO and I0 look almost identical
typographically (depending on the font) whereas // and / look pretty
different.  It would be a better analogy to say "What mark would you
give a student who used variables X and XX in the same program".  And,
I wouldn't have a problem with that.

How about URL's?  '/' and '//' have different meanings there and I
don't think people have a big problem with this.

The other point - "=" vs "==" - is a bit harder to answer.  Both of
these symbols are used in Python, but not in the same context.

All-the-good-symbols-are-already-taken-ly y'rs,
					//C





From gstein at lyra.org  Tue Nov  7 19:49:12 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 7 Nov 2000 10:49:12 -0800
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <Pine.GSO.4.10.10011070830200.13407-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 07, 2000 at 08:37:01AM +0200
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial>
Message-ID: <20001107104912.Q14054@lyra.org>

On Tue, Nov 07, 2000 at 08:37:01AM +0200, Moshe Zadka wrote:
> On 6 Nov 2000, Eric Gillespie, Jr. wrote:
> > There has been a problem in Debian's python-gdk-imlib package for
> > quite some time: Transparent PNGs do not display properly (see
> > attached example script). According to upstream
> > (http://www.daa.com.au/pipermail/pygtk/2000-September/000336.html),
> > the proper solution is either to have Python use RTLD_GLOBAL in
> > dlopen calls when loading extension modules,
> 
> Or, possible, using dlmodule to dlopen things ourselves rather then
> using Python's import facilities for that.

There was quite a long conversation [on python-dev] a while back (geez, a
year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
were pros and cons for both directions, and I believe some compatibility
issues.

You may be able to find the conversation, then figure out why Python chose
its current mechanism. Heck... maybe it should change :-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From akuchlin at mems-exchange.org  Tue Nov  7 20:11:06 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 7 Nov 2000 14:11:06 -0500
Subject: [Python-Dev] Re: python-gdk-imlib and Delinquent Maintainers
In-Reply-To: <20001107104912.Q14054@lyra.org>; from gstein@lyra.org on Tue, Nov 07, 2000 at 10:49:12AM -0800
References: <87ofzs39lo.fsf@indy.progeny.com> <Pine.GSO.4.10.10011070830200.13407-100000@sundial> <20001107104912.Q14054@lyra.org>
Message-ID: <20001107141106.A10897@kronos.cnri.reston.va.us>

On Tue, Nov 07, 2000 at 10:49:12AM -0800, Greg Stein wrote:
>There was quite a long conversation [on python-dev] a while back (geez, a
>year ago? more?) about RTLD_GLOBAL and whether Python should use it. There
>were pros and cons for both directions, and I believe some compatibility
>issues.

kronos Python>cvs log importdl.c 
  ...
revision 2.47
date: 1998/05/18 13:42:45;  author: guido;  state: Exp;  lines: +4 -6
Remove use of RTLD_GLOBAL.
----------------------------                                                    ...
revision 2.41
date: 1997/12/02 20:43:18;  author: guido;  state: Exp;  lines: +7 -3
Add the flag RTLD_GLOBAL to the dlopen() options.
This exports symbols defined by the loaded extension to other
extensions (loaded later).
 
(I'm not quite sure about this but suppose it can't hurt...)
----------------------------

Adding RTLD_GLOBAL in one version, removing it in the next: a new
Python tradition!

--amk




From est at hyperreal.org  Tue Nov  7 20:04:33 2000
From: est at hyperreal.org (est at hyperreal.org)
Date: Tue, 7 Nov 2000 11:04:33 -0800 (PST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com> from Guido
 van Rossum at "Nov 7, 2000 11:14:21 am"
Message-ID: <20001107190433.26578.qmail@hyperreal.org>

Guido van Rossum discourseth:
> 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest level.

I really like this as a primitive appropriate for Python's evolution.

> - When func() returns, the uthread that was executing it ceases to be
>   runnable.  The uthread that most recently yielded to it is resumed,
>   unless that is no longer runnable, in which case the uthread that
>   most recently yielded to *it* is resumed, and so on until a runnable
>   uthread is found or until no runnable uthreads are left, in which
>   case the program terminates.  (XXX I need a proof here that this
>   works.)

I'd like it added that when a uthread chains to its yielder it drops
(i.e., nulls and decrefs) the reference to that yielder.  I want
uthreads in some of the same applications in which i disable gc for
real-time purposes, and I don't want circular structures of unrunnable
uthreads leaking my memory.

> - Calling u.yield() or u.kill() for a non-runnable uthread is an error
>   and raises an exception.

A runnable() predicate might be nice.

> - I'm not sure that I got the start conditions right.  Should func() be
>   be allowed to run until its first yield() when uthread.new(func) is
>   called?

+1 for no on this.

> - I'm not sure that the rules for returning and raising exceptions
>   from func() are the right ones.

I'm particularly unsure about the exception propagation.  It could
always be disabled by a universal exception handler in the uthread,
but I'm not sure it's even worth the implementation effort.

> - Should it be possible to pass a value to another uthread by passing
>   an argument to u.yield(), which then gets returned by the resumed
>   yield() call in that uthread?

Certainly! :)

As written, the strawman seems to require that the dread intervening C
stack frames are handled transparently (since it doesn't say anything
about them).  This seems pretty important to me.  An instance method
may well know that it should yield, yet not know that it's being
called as a callback from a class/type that's just been moved to C.

OTOH..not handling this transparently would increase my market value
as a Python programmer.  Handling it right might get me some unpaid
work implementing some of the low-level details for Linux.  Hmm! :D

Eric




From paulp at ActiveState.com  Tue Nov  7 21:33:19 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 07 Nov 2000 12:33:19 -0800
Subject: [Python-Dev] Integer division transition
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
		<m13tADF-000Dm8C@artcom0.artcom-gmbh.de> <14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <3A08670F.A4703F32@activestate.com>

Charles G Waldman wrote:
> 
> ...
> 
> 
> I think that making "div" an infix operator would be setting a
> horrible precedent.  

I think it would be a good precedent because it is a cleaner upgrade
path to things like

matrixdiv, matrixmul, ...

 Paul Prescod



From tismer at tismer.com  Tue Nov  7 20:54:14 2000
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 07 Nov 2000 21:54:14 +0200
Subject: [Python-Dev] Re: uthread strawman
References: <200011060450.RAA00019@s454.cosc.canterbury.ac.nz>  
	            <3A06BEF4.95B773BD@tismer.com> <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <3A085DE6.28593202@tismer.com>

Just answering/clarifying a few bits,
since I can't change your opinion about
continuations, anyway.

Guido van Rossum wrote:
> 
> I've thought about it a little more, and drawn some pictures in my
> head.

I have to agree with Guido when he says:

> I still have to disagree with Christian when he says:
> 
> > Making Python completely coroutine aware, without
> > tricking the C stack, is 90 percent of the problem.
> > But after walking that far, there is no reason
> > to leave the other 10 percent alone.

... since I meant implementability. Of course there are
other reasons gainst continuations. I just did it since they
were in reach.

> Without continuations, but with microthreads (uthreads) or coroutines,
> each (Python) stack frame can simply be "paused" at a specific point
> and continued later.  The semantics here are completely clear (except
> perhaps for end cases such as unhandled exceptions and intervening C
> stack frames).

I agree. But also with continuations, the situation is identical,
as long as you don't try anything else where continuations
would be needed.
Note that they will not need to be created when the mentioned
structures are implemented well. We don't have to implement them,
but providing support for them in the interpreter framework
is simple. (that's the 10% issue).

> With continuations, you have to decide how much state to save for a
> future continuation.  It would seem easy enough: save all state kept
> in the frame except for the local variables.  But now consider this:
> the "evaluation stack" contained in each frame could easily be
> replaced by a bunch of temporary variables, if we had a slightly
> different instruction set (3-address opcodes instead of stack-based
> opcodes).  Then would we save those temporary variables or not?  it
> can make a difference!  Since the "save continuation" operation is a
> function call, you can easily save a continuation while there are some
> items on the value stack.  I believe the current implementation saves
> these so they are restored when you jump to the continuation.  But if
> the compiler were to use temporary variables instead of the evaluation
> stack, they might not have been restored!

I would consider these temporary variables registers which must
be preserved. They are immutable objects as part of the immutable
continuation, treated as values. Stack or registers, this is part
of an expression evaluation. Temporary results must conceptually
be read-only, whatever way I implement this.

> Here's another example.  Suppose you set up a for loop.  After three
> iterations through the loop you save a continuation.  Then you finish
> hree more iterations.  Then you return to the saved continuation.
> Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> this without trying it.  My guess: it gets restarted at 3 iterations,
> because the loop index is saved on the value stack.  If you rewrite
> this using a while loop, however, it would get restarted at 6
> iterations, because then your loop index is an unsaved local variable.
> Ditto if you changed the bytecode compiler so for loops use an
> anonymous local variable instead of an entry on the evaluation
> stack.

Wrong guess!
Exactly for that reason I changed the loop code to put a mutable
loopcounter object on the stack.
The loop works perfectly.

> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.  (I've been told that the exact semantics of
> continuations in Scheme differ greatly between Scheme implementations.)

In a sense, you have continuations already, also with the restriction
to gen/co/uthread structures. The only difference is to treat a
frame as exactly one continuation and to disallow to have more
than one at any time.
This saves the decision about the ambiguities you mentioned.

I agree that going to this point and not further for the
Python core is a good idea.
A PEP doesn't need to name continuations at all.

On the other hand, I don't see a reason why this hsould mean
that Python *must not* support them. What I'd like to keep
is the possibility to still write such an extension module.

Enabling this for educational purposes is a great value
that comes at a cheap price and no impact for the core.

> Now let's look at Jython.  In Jython, we can simulate "paused frames"
> well enough by using real Java threads.  However full continuations
> would require modifications to the JVM -- which is unacceptable to a
> language boasting "100% Pure Java".  Another reason against allowing
> continuations.

Not quite true, after I heard of a paper that shows how
to implement continuations in Java, using threads.
But I'll come back to that when I have the paper.

> So, all in all, I don't think of continuations as "the last 10% that
> we might as well add to finish the job."  I see it as an ill-specified
> hypergeneralization.

Can we agree to not support them without forbidding them?

...
> A strawman proposal:

Ok, this looks all very well to me. More on that later.
One question: Why do you want an explicit u.yield() ?
Uthreads are scheduled automatically now, like real
threads. Do you see a major drawback in supporting
this, maybe as an option? Or do you see automatic
scheduling as an extra construct on top with a special
"scheduler" uthread?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From jack at oratrix.nl  Tue Nov  7 23:57:53 2000
From: jack at oratrix.nl (Jack Jansen)
Date: Tue, 07 Nov 2000 23:57:53 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts 
In-Reply-To: Message by "M.-A. Lemburg" <mal@lemburg.com> ,
	     Tue, 07 Nov 2000 10:17:56 +0100 , <3A07C8C4.1F14985F@lemburg.com> 
Message-ID: <20001107225758.B77651301D9@oratrix.oratrix.nl>

> > In other words, what does this new type_id thing
> > actually *mean*?
> 
> For the interpreter it means that it can assume the type
> interface to be binary compatible to the "original"
> type, e.g. by setting the flag to say PyDict_TypeID
> the type assures that all PyDict_*() APIs will work
> on the type -- basically the same thing as PyDict_Check()
> does now except that the type object needn't be the same
> anymore.

I would be _very_ happy if this single type_id could somehow be
replaced by an array, or a bitset.

I have a lot of types in MacPython that are acceptable to the APIs of
other types, a sort of poor-mans-inheritance scheme. For instance, all 
operating system calls that accept a MacOS WindowPtr will also happily 
accept a DialogPtr. Major magic is needed to get this to work
reasonably in Python, and the Python user can still accidentally mess
up the refcounting scheme and free things s/he isn't aware of.

As the number of types in a given run of the interpreter appears to be 
limited (am I right here?) and type-identity-tests are valid within a
single interpreter run only (am I right here?) an API like
  typeindex = Py_TypeToTypeIndex(typeobject);
which would use a dictionary as storage for the mapping and generate
the index numbers on the fly would do the trick. Call it once during
module initalization and the
  Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
macro would be a oneliner to test the bit in the set.

A few hundred bits in the set would get us a long way, I guess.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen at oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 



From fdrake at acm.org  Wed Nov  8 00:06:25 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:06:25 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.11800.444274.851825@buffalo.fnal.gov>
References: <200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<m13tADF-000Dm8C@artcom0.artcom-gmbh.de>
	<14856.11800.444274.851825@buffalo.fnal.gov>
Message-ID: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > I think that making "div" an infix operator would be setting a
 > horrible precedent.  Currently, all infix operators "look like"
 > operators, i.e. they are non-alphabetic characters, and things that
 > look like words are either functions or reserved words.

  Like "is", "in", "is not", and "not in"?

 > Long before the creation of python-dev, this issue had been discussed
 > numerous times on c.l.py, and the '//' operator was suggested several
 > times, and I don't think anybody ever had a problem with it...

  I don't have a strong preference for either // or div, but
definately want this to be an operator.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov  8 00:17:10 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Nov 2000 18:17:10 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <3A083BE6.4A502210@activestate.com>
References: <200011061717.MAA07847@cj20424-a.reston1.va.home.com>
	<200011070106.OAA00139@s454.cosc.canterbury.ac.nz>
	<20001107141702.H27208@xs4all.nl>
	<200011071333.IAA13497@cj20424-a.reston1.va.home.com>
	<14856.8184.854926.597397@cj42289-a.reston1.va.home.com>
	<3A083BE6.4A502210@activestate.com>
Message-ID: <14856.36214.267784.220958@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I think that the standard Python compiler is the appropriate tool for
 > this sort of thing. Anything that can be caught "statically" might as
 > well be implemented right in the compiler (at least from the user's
 > point of view) rather than in a separate "deprecation nanny."

  For linting programs using the final specification, this fine.  I'm
thinking that a tool to read over people's sources and say "'div' is
used in 120 places out of 56K lines of code." would be helpful because
we could determine the extent of the effect of using "div" instead of
"//".  If there are a limited number of projects affected, it may be
entirely reasonable to find out that there aren't enough uses to worry
about, and it becomes acceptable to make it a keyword (like the other
textual infix operators).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From greg at cosc.canterbury.ac.nz  Wed Nov  8 01:42:40 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 13:42:40 +1300 (NZDT)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.20202.174151.647981@buffalo.fnal.gov>
Message-ID: <200011080042.NAA00284@s454.cosc.canterbury.ac.nz>

Charles G Waldman <cgw at fnal.gov>:

> The other point - "=" vs "==" - is a bit harder to answer.

I think the reason this causes so much trouble is that many
programming languages, not to mention mainstream mathematics,
use "=" to mean what C uses "==" for.

Other such pairs in C, e.g. "&" vs "&&" and "+" vs "++",
don't seem to cause anywhere near as much difficulty, so
I don't think the problem is one of visual confusion, but
of semantic confusion.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+





From guido at python.org  Wed Nov  8 02:34:49 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 20:34:49 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Tue, 07 Nov 2000 11:04:33 PST."
             <20001107190433.26578.qmail@hyperreal.org> 
References: <20001107190433.26578.qmail@hyperreal.org> 
Message-ID: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>

[Note: the stackless at starship.python.net list seems to have trouble
again.  BeOpen may have crashed the machine again. :-( ]

[Note: response to Christian at end.]

[Guido]
> > 
> > A strawman proposal:
> > 
> > The uthread module provides the new functionality at the lowest level.

Eric (Tiedemann, right?  There are two Erics here -- it would help if
you signed your full name :-) writes:

> I really like this as a primitive appropriate for Python's evolution.

Cool.  I think I spoke too soon when I called it uthreads -- these are
really more like coroutines.

I also forgot to mention that I am assuming with this strawman that we
can use an essentially stackless implementation of the PVM.  I hope
that it will be possible to make it a lot simpler than current
stackless though, by not doing continuations.  Freezing a frame in
place is a lot simpler than freezing for multiple uses, which requires
one to decide what to copy and what to share!

> > - When func() returns, the uthread that was executing it ceases to be
> >   runnable.  The uthread that most recently yielded to it is resumed,
> >   unless that is no longer runnable, in which case the uthread that
> >   most recently yielded to *it* is resumed, and so on until a runnable
> >   uthread is found or until no runnable uthreads are left, in which
> >   case the program terminates.  (XXX I need a proof here that this
> >   works.)
> 
> I'd like it added that when a uthread chains to its yielder it drops
> (i.e., nulls and decrefs) the reference to that yielder.  I want
> uthreads in some of the same applications in which i disable gc for
> real-time purposes, and I don't want circular structures of unrunnable
> uthreads leaking my memory.

Good point.

> > - Calling u.yield() or u.kill() for a non-runnable uthread is an error
> >   and raises an exception.
> 
> A runnable() predicate might be nice.

Yes.

> > - I'm not sure that I got the start conditions right.  Should func() be
> >   be allowed to run until its first yield() when uthread.new(func) is
> >   called?
> 
> +1 for no on this.

You're being unnecessarily cryptic.  "Yes for no"?  So you're for the
original proposal (which doesn't start func() at all until it is
yielded to for the first time).

> > - I'm not sure that the rules for returning and raising exceptions
> >   from func() are the right ones.
> 
> I'm particularly unsure about the exception propagation.  It could
> always be disabled by a universal exception handler in the uthread,
> but I'm not sure it's even worth the implementation effort.

Agreed.  We may have to experiment.

> > - Should it be possible to pass a value to another uthread by passing
> >   an argument to u.yield(), which then gets returned by the resumed
> >   yield() call in that uthread?
> 
> Certainly! :)

This affects the initial condition.  If u hasn't called func() yet,
and I call u.yield(42), where does the 42 go?  Does it call func(42)?
That may make it make it unnecessarily hard to get the end conditions
right for func().  Again, we'll have to write some sample code to see
how this turns out in practice.

> As written, the strawman seems to require that the dread intervening C
> stack frames are handled transparently (since it doesn't say anything
> about them).  This seems pretty important to me.  An instance method
> may well know that it should yield, yet not know that it's being
> called as a callback from a class/type that's just been moved to C.

Not sure what you meant by intervening.  I certainly intended Python
to Python calls to be handled without creating extra C stack frames.
When Python calls C which calls back into Python, this is considered
all part of the same uthread.

Where it gets tricky is when this spawns a new uthread, which also
calls C which calls Python.  Now the second uthread has a C stack
frame above the C stack frame that's part of the first uthread.  This
means that the second uthread must return from its C code before the
first uthread can return to its C code!  A little extra bookkeeping
will be necessary to check for this -- so that when it is attempted an
exception is raised, rather than a return attempted from the wrong
uthread back into C.  This is the same as for current stackless.  The
solution is simply that the application "shouldn't do that."

> OTOH..not handling this transparently would increase my market value
> as a Python programmer.  Handling it right might get me some unpaid
> work implementing some of the low-level details for Linux.  Hmm! :D

If you want do hack continuations in C, be my guest -- as long as you
stay 10,000 kilometers away from core Python. :-)

[Now replying to Christian:]

> Just answering/clarifying a few bits,
> since I can't change your opinion about
> continuations, anyway.

Right!

> Guido van Rossum wrote:
> > 
> > I've thought about it a little more, and drawn some pictures in my
> > head.
> 
> I have to agree with Guido when he says:
> 
> > I still have to disagree with Christian when he says:
> > 
> > > Making Python completely coroutine aware, without
> > > tricking the C stack, is 90 percent of the problem.
> > > But after walking that far, there is no reason
> > > to leave the other 10 percent alone.
> 
> ... since I meant implementability. Of course there are
> other reasons gainst continuations. I just did it since they
> were in reach.

Hm.  Having seen a few fragments of your implementation today (just a
very little bit, since we were having an all-day meeting) I feel that
there are a lot of extra hacks needed to make the reuse of
continuations necessary.  This shouldn't be needed in my version.

> > Without continuations, but with microthreads (uthreads) or coroutines,
> > each (Python) stack frame can simply be "paused" at a specific point
> > and continued later.  The semantics here are completely clear (except
> > perhaps for end cases such as unhandled exceptions and intervening C
> > stack frames).
> 
> I agree. But also with continuations, the situation is identical,
> as long as you don't try anything else where continuations
> would be needed.

But the complexity in the code still exists because a continuation
*could* be reused and you don't know if it will ever happen so you
must be prepared.

> Note that they will not need to be created when the mentioned
> structures are implemented well. We don't have to implement them,
> but providing support for them in the interpreter framework
> is simple. (that's the 10% issue).

Having seen your code (frankly, a mess!) I don't believe that it's
only 10% at all.

> > With continuations, you have to decide how much state to save for a
> > future continuation.  It would seem easy enough: save all state kept
> > in the frame except for the local variables.  But now consider this:
> > the "evaluation stack" contained in each frame could easily be
> > replaced by a bunch of temporary variables, if we had a slightly
> > different instruction set (3-address opcodes instead of stack-based
> > opcodes).  Then would we save those temporary variables or not?  it
> > can make a difference!  Since the "save continuation" operation is a
> > function call, you can easily save a continuation while there are some
> > items on the value stack.  I believe the current implementation saves
> > these so they are restored when you jump to the continuation.  But if
> > the compiler were to use temporary variables instead of the evaluation
> > stack, they might not have been restored!
> 
> I would consider these temporary variables registers which must
> be preserved. They are immutable objects as part of the immutable
> continuation, treated as values. Stack or registers, this is part
> of an expression evaluation. Temporary results must conceptually
> be read-only, whatever way I implement this.

I heard from Tim that he helped you get this right.  The fact that it
is so hard to know the requirements for a practical implementation
makes me very worried that continuations may have hidden bugs.

> > Here's another example.  Suppose you set up a for loop.  After three
> > iterations through the loop you save a continuation.  Then you finish
> > hree more iterations.  Then you return to the saved continuation.
> > Where does the loop continue: at 3 or at 6 iterations?  Try to answer
> > this without trying it.  My guess: it gets restarted at 3 iterations,
> > because the loop index is saved on the value stack.  If you rewrite
> > this using a while loop, however, it would get restarted at 6
> > iterations, because then your loop index is an unsaved local variable.
> > Ditto if you changed the bytecode compiler so for loops use an
> > anonymous local variable instead of an entry on the evaluation
> > stack.
> 
> Wrong guess!
> Exactly for that reason I changed the loop code to put a mutable
> loopcounter object on the stack.
> The loop works perfectly.

Wow.  i'm impressed.  You must have borrowed my time machine. :-)

Still, I believe there was a time when the loop *didn't* work
perfectly yet. It is really hard to know what is needed.  Are you
*sure* that it now *always* does the right thing?  What if I save a
continuation in the middle of a shortcut Boolean expression (and/or
stuff)?  Or in cases like a<b<c?  (Here b gets saved on the stack to
avoid loading it again.)

> > This semantic imprecision is one of the reasons why I don't like the
> > concept of continuations.  (I've been told that the exact semantics of
> > continuations in Scheme differ greatly between Scheme implementations.)
> 
> In a sense, you have continuations already, also with the restriction
> to gen/co/uthread structures. The only difference is to treat a
> frame as exactly one continuation and to disallow to have more
> than one at any time.
> This saves the decision about the ambiguities you mentioned.

Yeah, right.  You can call pausable/resumable frames use-once
continuations if you want to.  And if that gives you the happy feeling
that I "support" continuations, fine.

> I agree that going to this point and not further for the
> Python core is a good idea.
> A PEP doesn't need to name continuations at all.

Good.

> On the other hand, I don't see a reason why this hsould mean
> that Python *must not* support them. What I'd like to keep
> is the possibility to still write such an extension module.
> 
> Enabling this for educational purposes is a great value
> that comes at a cheap price and no impact for the core.

I doubt it.  I'm not going to allow any compromises just to make it
easier to reuse continuations.  (Such as using a mutable counter in
the for-loop code.)

> > Now let's look at Jython.  In Jython, we can simulate "paused frames"
> > well enough by using real Java threads.  However full continuations
> > would require modifications to the JVM -- which is unacceptable to a
> > language boasting "100% Pure Java".  Another reason against allowing
> > continuations.
> 
> Not quite true, after I heard of a paper that shows how
> to implement continuations in Java, using threads.
> But I'll come back to that when I have the paper.

I've heard that his margin was too small to contain the proof.  I
expect that it will be a disappointment from a practical point of
view: perhaps he emulates the JVM in Java.

> > So, all in all, I don't think of continuations as "the last 10% that
> > we might as well add to finish the job."  I see it as an ill-specified
> > hypergeneralization.
> 
> Can we agree to not support them without forbidding them?

I won't forbid them, but I won't make compromises to the core PVM that
would make them easier to implement.  Your patch set would still be a
heck of a lot smaller of course.

> ...
> > A strawman proposal:
> 
> Ok, this looks all very well to me. More on that later.
> One question: Why do you want an explicit u.yield() ?
> Uthreads are scheduled automatically now, like real
> threads. Do you see a major drawback in supporting
> this, maybe as an option? Or do you see automatic
> scheduling as an extra construct on top with a special
> "scheduler" uthread?

See my response near the top to Eric about this.  I was thinking of a
lower-level concept, like coroutines.  I might consider automatic
scheduling of uthreads too.  But I've noticed that there are some ugly
hacks in that code, too. :-)

I've lived for years with a system (early Amoeba) that had threads
with only explicit scheduling: other threads would only run when you
were blocked for I/O or waiting for a semaphore.  It made for very
easy coding in some cases, since you didn't need to protect critical
sections with mutexes.  Unles, that is, you invoke stuff that might do
I/O (maybe for debugging :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Wed Nov  8 03:12:09 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 15:12:09 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080134.UAA19392@cj20424-a.reston1.va.home.com>
Message-ID: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz>

Guido:

> I hope that it will be possible to make it a lot simpler than current
> stackless though, by not doing continuations.

My feeling is that this won't be the case. The fundamental
change of structure needed to make it stackless will be
much the same, as will the thought processes necessary
to understand how it works.

> Where it gets tricky is when this spawns a new uthread, which also
> calls C which calls Python... The solution is simply that the 
> application "shouldn't do that."

I worry that this is going to be a rather severe restriction.
For instance, it may make it impossible to spawn a uthread
from within a callback from a GUI framework. Since with many
GUI frameworks the entire application gets executed in
callbacks, you wouldn't be able to use uthreads at all with
such a framework.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Wed Nov  8 03:58:20 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 07 Nov 2000 21:58:20 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 15:12:09 +1300."
             <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> 
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>

> Guido:
> 
> > I hope that it will be possible to make it a lot simpler than current
> > stackless though, by not doing continuations.

[Greg Ewing]
> My feeling is that this won't be the case. The fundamental
> change of structure needed to make it stackless will be
> much the same, as will the thought processes necessary
> to understand how it works.

I hope you are wrong but you may be right.  I'll have to have a good
look -- or someone else (not Christian!  With all due respect his code
is unreadable :-).

> > Where it gets tricky is when this spawns a new uthread, which also
> > calls C which calls Python... The solution is simply that the 
> > application "shouldn't do that."
> 
> I worry that this is going to be a rather severe restriction.
> For instance, it may make it impossible to spawn a uthread
> from within a callback from a GUI framework. Since with many
> GUI frameworks the entire application gets executed in
> callbacks, you wouldn't be able to use uthreads at all with
> such a framework.

But that's the same problem that current stackless has.  I take it
that you don't see the point of stackless then?  That's fine.  Maybe
this is not an application that could use uthreads.  They seem more
something for servers anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From greg at cosc.canterbury.ac.nz  Wed Nov  8 04:18:10 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2000 16:18:10 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <200011080318.QAA00324@s454.cosc.canterbury.ac.nz>

Guido:

> I take it that you don't see the point of stackless then?

I have mixed feelings about it. I really like the idea of
uthreads, but I get the impression that Stackless as it
stands is only a partial implementation of the idea, with
no easy way in sight to evolve it into a full implementation.

> Maybe this is not an application that could use uthreads.

The point is that the user can be mixing Python and C stack
frames without even realising it. I was just giving one example
of how that can come about.

Saying "don't do that" isn't very helpful when "that" is
something which poeple often do quite unconsciously.

So, while I'd love to see uthreads as a core Python feature
one day, I think I'm in agreement with you that Stackless
isn't yet ready to be made into the standard Python
implementation.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From pf at artcom-gmbh.de  Wed Nov  8 08:33:00 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 08:33:00 +0100 (MET)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com> from "Fred L. Drake, Jr." at "Nov 7, 2000  6: 6:25 pm"
Message-ID: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>

> Charles G Waldman writes:
>  > I think that making "div" an infix operator would be setting a
>  > horrible precedent.  Currently, all infix operators "look like"
>  > operators, i.e. they are non-alphabetic characters, and things that
>  > look like words are either functions or reserved words.
 
Fred L. Drake, Jr.:
>   Like "is", "in", "is not", and "not in"?

And not to forget "and", "or" which were also infix operators from
the very beginning.  So "div" is no precedent at all.

IMHO the term "horrible" applies to operator symbols composed out
of non-alphabetic characters, where the meaning of these operators 
is hard to guess.  

counter-example: Using "><" as a vector cross product operator might 
still make some sense.  

But what would be the meaning of all those other arbitrary combinations
like ".+", ".%", ".*", "//", "@.", "~*" or what else has been proposed
to extend Python in the numeric area?  As long as the meaning of such
an operator isn't obvious from basic math knowledge, I clearly prefer
keyword operators.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Wed Nov  8 11:13:51 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 11:13:51 +0100
Subject: [Python-Dev] Starship down again ?!
References: <200011080620.WAA20659@slayer.i.sourceforge.net>
Message-ID: <3A09275F.CE4B01FA@lemburg.com>

> 
> +     Marc-Andr? Lemburg's mx.Proxy package.  These Web pages appear to
> +     be unavailable at the moment.
> + 
> +         http://starship.python.net/crew/lemburg/
> + 

Looks like Starship is down again. Is this due to the move from
BeOpen to DC or has someone pulled the plug on that ADSL line ? ...

 9  sl-bb1-rly-0-0-0.sprintlink.net (144.232.14.6)  120 ms  117 ms  113 ms
10  beth1sr2-2-0-0.md.us.prserv.net (165.87.97.226)  114 ms  116 ms  114 ms
11  beth1br2-ge-6-0-0-0.md.us.prserv.net (165.87.29.182)  122 ms  121 ms  116 ms
12  sfra1br1-so-2-1-0-0.ca.us.prserv.net (165.87.233.42)  193 ms  192 ms  191 ms
13  sfra1sr3-ge-2-0-0-0.ca.us.prserv.net (165.87.33.121)  191 ms  189 ms  190 ms
14  165.87.161.13 (165.87.161.13)  191 ms  191 ms  189 ms
15  core4-g2-0.snfc21.pbi.net (209.232.130.77)  197 ms  192 ms  190 ms
16  rback26-fe2-0.snfc21.pbi.net (216.102.187.153)  212 ms  197 ms  197 ms
17  adsl-63-202-160-65.dsl.snfc21.pacbell.net (63.202.160.65)  206 ms  212 ms  204 ms
18  * * *
19  * * *

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Wed Nov  8 12:23:46 2000
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 13:23:46 +0200
Subject: [Python-Dev] uthread strawman
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>
Message-ID: <3A0937C2.903D42@tismer.com>


Guido van Rossum wrote:
> 
> > Guido:
> >
> > > I hope that it will be possible to make it a lot simpler than current
> > > stackless though, by not doing continuations.
> 
> [Greg Ewing]
> > My feeling is that this won't be the case. The fundamental
> > change of structure needed to make it stackless will be
> > much the same, as will the thought processes necessary
> > to understand how it works.
> 
> I hope you are wrong but you may be right.  I'll have to have a good
> look -- or someone else (not Christian!  With all due respect his code
> is unreadable :-).

Are you talking of my changes to ceval.c or the continuationmodule?
I think it can't be the latter, since that does not matter at all
if we talk about Stackless. Stackless was written to make continuations
possible. It does not implement them.

My changes to ceval.c are written in the same style as your
original code, and it uses the same level of commenting as yours:
Nothing at all. :-)
With all due respect, I consider both versions equally unreadable,
unless one understands what the intent of the code is.

Until last October, I tried to keep everything as readable
and understandable as possible. Then it became clear that
this implementation would never make it into the core.
Then I gave up my efforts, and I also added a lot of
optimizations to the interpreter, by some systematic use
of macroes. Shurely his doesn't increase readability.

Forgetting about these optimizations, the code doesn't do
much more than the following:

eval_code2_setup
  is split off of the original eval_code2 function.
  It prepares a new frame for execution and puts it
  on top of the frame stack.

PyEval_Frame_Dispatch
  is a new function. It controls the execution of frames.
  Every initial or recursive interpreter call starts
  such a dispatcher. The nested dispatchers control the
  remaining "stackful" execution of Python.
  In its central loop, it runs the topmost frame of the
  frame stack, receives its return value and runs the
  next frame, until it sees the frame appear that
  spawned this dispatcher. Then it returns.

eval_code2_loop
  is the "real" part of the original eval_code2 function.
  It is not much different from the original. Major changes
  have been done to the entry code, the periodic checks
  in the loop, and the handling of function calls.

  The "big switch" has been simplified in the sense, that
  errors are no longer treated with various variables
  which have to be checked outside the switch.
  Instead, errors are directly mapped on a pseudo-opcode
  that allows to handle exceptions as just another case
  of the big switch.

  Every function call has got additional code that checks
  for the so-called unwind token, which tells us to leave
  this frame and to return to the scheduler.
  
  On entry to the frame, on every trip through the main loop,
  and after every function call, a callback f_callguard is
  checked for existence. If it exists, it is called, and
  if it returns -42, again the frame is left and we return
  to the scheduler.

  Entry into a frame has become a bit difficult, since we
  no longer know in advance whether a frame is expected
  to return a value or not. Due to uthread scheduling,
  switches occour between opcodes, and no values are
  transferred. When switching in the context of a function
  call, there *are* return values expected.
  This is all handled via some flags, in the frame entry
  code, line 948ff.

  Then, there are some very simple changes to the loop
  construct.

  Generally, more state variables are in the frames and
  kept up-to-date, like the instruction pointer.

  I'm omitting the extra code for uthread support here.

  Some functions were pulled out of the main loop, in order
  to make it smaller and easier to read. I would undo this
  today, since it makes comparison to the old version
  quite impossible, and it didn't yield more speed.

This is about all of it.
As you can see, there is no explicit support for co-anything
in the code. There are some generalisations to frame calling
and some callback hooks which actually do all co operations.

An implementation targeted for core integration would look
quite much different. It would provide more functionality
directly, without using callbacks.

A pure coroutine based implementation as you proposed
would not need the generalization of the frame parameter
passing, since switches can only occour in the context
of a function call.

Supporting auto-scheduled uthreads needs to distinguish
explicit and implicit switching, since implicit switching
occours between opcodes, not *in* opcodes.

The techniques for this can be written in quite a different
manner than I did.

Again, this code is not intended for inclusion in the core,
and not for drawing conclusions for the feasibility of
Stackless at all. The latter has been shown by applications
like the uthreads, and by its central use in the EVE game.
We need to judge the priciple, not the implementaiton.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From phil at river-bank.demon.co.uk  Wed Nov  8 13:26:54 2000
From: phil at river-bank.demon.co.uk (Phil Thompson)
Date: Wed, 08 Nov 2000 12:26:54 +0000
Subject: [Python-Dev] What to choose to replace Tkinter?
References: <049d01c0471f$d7899450$8119fea9@neil>
Message-ID: <3A09468E.9E767720@river-bank.demon.co.uk>

I've come to this discussion rather late...

Most modern GUI toolkits have (or will soon have) the widgets to compete
with Tk strengths. The difficult question with a Tkinter replacement is
the complete fragmentation of the GUI toolkit "market". I don't believe
that you can, today, identify a toolkit that you are sure is going to
have widespread support and the longevity needed (in 5 years time you
don't want to be in the position you are in today with Tk).

I see two alternatives...

- make the Tkinter replacement an abstraction layer between Python and
the *user's* choice of toolkit. The developer gets a consistent API, and
toolkits can be adopted and dropped as fashions change. This is the
approach taken by VeePee (http://www.thekompany.com/projects/vp/).

- don't bundle Tkinter with Python. At least you then make people think
a bit more about what they want from a toolkit and make an appropriate
choice - let Tkinter's replacement be found by natural selection. At the
very least let's have a more up-front presentation of the different
options, strengths/weaknesses etc on the web site. Cameron must be
getting bored of pointing people to his toolkit summary.

For the record, Qt has a good Canvas widget, Unicode support, user
selectable Windows/Mac/Unix look & feel etc, etc.

Phil



From guido at python.org  Wed Nov  8 14:20:02 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 08:20:02 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Wed, 08 Nov 2000 13:23:46 +0200."
             <3A0937C2.903D42@tismer.com> 
References: <200011080212.PAA00313@s454.cosc.canterbury.ac.nz> <200011080258.VAA19690@cj20424-a.reston1.va.home.com>  
            <3A0937C2.903D42@tismer.com> 
Message-ID: <200011081320.IAA21990@cj20424-a.reston1.va.home.com>

> Again, this code is not intended for inclusion in the core,
> and not for drawing conclusions for the feasibility of
> Stackless at all. The latter has been shown by applications
> like the uthreads, and by its central use in the EVE game.
> We need to judge the priciple, not the implementaiton.

Of course.  Thanks by the way for the clear explanation of what needs
to be done!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gward at mems-exchange.org  Wed Nov  8 15:02:49 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Wed, 8 Nov 2000 09:02:49 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001107182621.I27208@xs4all.nl>; from thomas@xs4all.net on Tue, Nov 07, 2000 at 06:26:21PM +0100
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>
Message-ID: <20001108090249.A28202@ludwig.cnri.reston.va.us>

On 07 November 2000, Thomas Wouters said:
> Sorry for the late remark (I did see your earlier message) but after reading
> the patch I realized 'OPT' isn't the right place for this. 'OPT' should be
> for non-essential stuff: warnings, debug-info and optimizations. Removing
> things from OPT shouldn't break anything, and neither should adding options
> that fit in the categories above. (Barring broken compilers, of course.)
> 
> Instead, the -fPIC option should be added to CFLAGS, I think. The Python
> autoconf setup is slightly less versatile than most, though, since it's
> doggone hard to near impossible to change things like OPT, CC, CFLAGS, etc,
> without editing configure(.in) :P If noone else does it before me, I'll see
> about fixing at least the -fPIC thing later, when I find some time ;P

Good point -- fixing CFLAGS instead of OPT sounds right to me.  I'm not
really sure on where to do this, though.

Ooh, I just noticed this in configure.in:

  # DG/UX requires some fancy ld contortions to produce a .so from an .a
  case $MACHDEP in
  dguxR4)
        LDLIBRARY='libpython$(VERSION).so'
        OPT="$OPT -pic"
        ;;

No prize for guessing that "-pic" on the DG/UX compiler has a similar
effect to GCC's -fPIC, and based on the comment this is required.  I'm
guessing this should be in CFLAGS as well.

Oh wait: CFLAGS is not exported from configure.in -- it's *only* defined
in the Makefile.  From Makefile.in:

  CFLAGS=		$(OPT) -I. $(DEFS)

IOW, it looks like OPT is used for all non-preprocessor compiler flags,
whether they're "frills" like optimization/debugging or not.

Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
right thing, but for the wrong reason.  Fixing it would require a little
more involved surgery on configure.in and the Makefiles.  And it would
also require reexamining every use of OPT in configure.in (not too hard,
"grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
so forth at config time or at make time.

I'm willing to spend some time on this; does anyone think it's a
pointless waste of time?

        Greg



From fdrake at acm.org  Wed Nov  8 15:04:32 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:04:32 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: <20001108090249.A28202@ludwig.cnri.reston.va.us>
References: <200011071544.HAA31147@slayer.i.sourceforge.net>
	<20001107182621.I27208@xs4all.nl>
	<20001108090249.A28202@ludwig.cnri.reston.va.us>
Message-ID: <14857.23920.646760.779849@cj42289-a.reston1.va.home.com>

Greg Ward writes:
 > I'm willing to spend some time on this; does anyone think it's a
 > pointless waste of time?

  I'm certainly willing to allocate some of your cycles to this.  ;)
I'll even help test it once you've checked it in.
  Seriously, I think you're right -- there needs to be a separation of
what's needed and optional stuff added from the make command line.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Wed Nov  8 15:21:20 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 09:21:20 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.177,1.178 configure,1.169,1.170
In-Reply-To: Your message of "Wed, 08 Nov 2000 09:02:49 EST."
             <20001108090249.A28202@ludwig.cnri.reston.va.us> 
References: <200011071544.HAA31147@slayer.i.sourceforge.net> <20001107182621.I27208@xs4all.nl>  
            <20001108090249.A28202@ludwig.cnri.reston.va.us> 
Message-ID: <200011081421.JAA22160@cj20424-a.reston1.va.home.com>

> Conclusion: my patch (add "-fPIC" to OPT instead of CFLAGS) does the
> right thing, but for the wrong reason.  Fixing it would require a little
> more involved surgery on configure.in and the Makefiles.  And it would
> also require reexamining every use of OPT in configure.in (not too hard,
> "grep -c OPT" only finds 16 matches).  IMHO this would be a good thing:
> if we do it right, it should make it easier to tweak OPT, CC, CFLAGS and
> so forth at config time or at make time.
> 
> I'm willing to spend some time on this; does anyone think it's a
> pointless waste of time?

No, please fix it right!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From cgw at fnal.gov  Wed Nov  8 15:23:47 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Wed, 8 Nov 2000 08:23:47 -0600 (CST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
	<m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14857.25075.482728.276846@buffalo.fnal.gov>

 > > Charles G Waldman writes:
 > >  > I think that making "div" an infix operator would be setting a
 > >  > horrible precedent.  Currently, all infix operators "look like"
 > >  > operators, i.e. they are non-alphabetic characters, and things that
 > >  > look like words are either functions or reserved words.
 >  
 > Fred L. Drake, Jr.:
 > >   Like "is", "in", "is not", and "not in"?
 > 
 > Peter Funk writes:
 > And not to forget "and", "or" which were also infix operators from
 > the very beginning.  So "div" is no precedent at all.

OK, I stand corrected and feel suitably foolish.  However I still
think it's quite inconsistent to have divmod(a,b) but a div b.




From fdrake at acm.org  Wed Nov  8 15:38:51 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 09:38:51 -0500 (EST)
Subject: [Python-Dev] Integer division transition
In-Reply-To: <14857.25075.482728.276846@buffalo.fnal.gov>
References: <14856.35569.55094.245631@cj42289-a.reston1.va.home.com>
	<m13tPjA-000Dm8C@artcom0.artcom-gmbh.de>
	<14857.25075.482728.276846@buffalo.fnal.gov>
Message-ID: <14857.25979.244131.879387@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > OK, I stand corrected and feel suitably foolish.  However I still
 > think it's quite inconsistent to have divmod(a,b) but a div b.

  I suspect div would be much more widely used than divmod(), which is
essentially a performance optimization when you need both results.
One reason *not* to make divmod() an operator, aside from issues of
legacy code, is that it really returns two results (never mind that it
returns exactly one tuple); I can't think of another operator that
conceptually returns two values.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From pf at artcom-gmbh.de  Wed Nov  8 16:32:22 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 16:32:22 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: <200011081517.HAA16919@slayer.i.sourceforge.net> from Guido van Rossum at "Nov 8, 2000  7:17:51 am"
Message-ID: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

this rather old module still contains string exceptions.
Since string exceptions are depreceated in favour of class based
exceptions wouldn't it be better to tweak those few lines into
class based exceptions now?

> Add 1994 Coroutine module by Tim Peters
[...]
> Killed = 'Coroutine.Killed'
> EarlyExit = 'Coroutine.EarlyExit'

Regards, Peter



From guido at python.org  Wed Nov  8 16:42:41 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 10:42:41 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE,1.1 README,1.7,1.8
In-Reply-To: Your message of "Wed, 08 Nov 2000 16:32:22 +0100."
             <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13tXD4-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011081542.KAA22738@cj20424-a.reston1.va.home.com>

> this rather old module still contains string exceptions.
> Since string exceptions are depreceated in favour of class based
> exceptions wouldn't it be better to tweak those few lines into
> class based exceptions now?
> 
> > Add 1994 Coroutine module by Tim Peters
> [...]
> > Killed = 'Coroutine.Killed'
> > EarlyExit = 'Coroutine.EarlyExit'

No.  This code is of historic interest only.  Don't touch it please!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From pf at artcom-gmbh.de  Wed Nov  8 17:06:56 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 8 Nov 2000 17:06:56 +0100 (MET)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: <200011081542.KAA22738@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 8, 2000 10:42:41 am"
Message-ID: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de>

I was nitpicking:
> > this rather old module still contains string exceptions.
> > Since string exceptions are depreceated in favour of class based
> > exceptions wouldn't it be better to tweak those few lines into
> > class based exceptions now?
> > 
> > > Add 1994 Coroutine module by Tim Peters
> > [...]
> > > Killed = 'Coroutine.Killed'
> > > EarlyExit = 'Coroutine.EarlyExit'

Guido van Rossum answered:
> No.  This code is of historic interest only.  Don't touch it please!

Hmmmm.... I always thought of the Demo directory as a repository for
example code, which may be used to teach Python programming to
beginners.  I know, that some pieces are a way out of date.  

But I think it would be a worthwile goal to update at least some of
those pieces step by step to reflect current Python coding habits.
The README in Demo says:

"""This directory contains various demonstrations of what you can do with
Python. [...]"""

If you want to turn the Demo directory into a museum of code snippets of
historic interest, at least the README should say so. ;-)

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From gmcm at hypernet.com  Wed Nov  8 18:09:48 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Wed, 8 Nov 2000 12:09:48 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com> 
Message-ID: <3A09428C.12529.1044B070@localhost>

[Guido]
> Without continuations, but with microthreads (uthreads) or
> coroutines, each (Python) stack frame can simply be "paused" at a
> specific point and continued later.  The semantics here are
> completely clear (except perhaps for end cases such as unhandled
> exceptions and intervening C stack frames).

Exceptions require some thought, particularly because the "for" 
protocol uses an IndexError as a signal. In my own stuff I've found I 
need to catch all exceptions in the coroutine, primarily because I've 
always got resources to clean up, but clearly the implementation has 
to do the right thing when an exception crosses the boundary.
 
> A strawman proposal:
> 
> The uthread module provides the new functionality at the lowest
> level. Uthread objects represent microthreads.  An uthread has a
> chain of stack frames linked by back pointers just like a regular
> thread. Pause/resume operations are methods on uthread objects. 
> Pause/resume operations do not address specific frames but
> specific uthreads; within an uthread the normal call/return
> mechanisms can be used, and only the top frame in the uthread's
> stack of call frames can be paused/resumed (the ones below it are
> paused implicitly by the call to the next frame, and resumed when
> that call returns).

I'm not convinced (though I'm not asking you to convince me - I need to 
ponder some more) that this is the right approach. My worry is that to 
do coroutines, we end up with a bunch of machinery on top of 
uthreads, just like Tim's old coroutine stuff, or implementations of 
coroutines in Java.

My mental test case is using coroutines to solve the impedance 
mismatch problem. SelectDispatcher is a simple example (write 
"client" code that looks like it's using blocking sockets, but multiplex 
them behind the curtain). Using a "pull" parser as a "push" parser is 
another case, (that is, letting it think it's doing its own reads).

But what about using a "pull" lexer and a "pull" parser, but tricking 
them with coroutines so you can "push" text into them? Tim's 
implementation of the  Dahl & Hoare example (which I 
rewrote in mcmillan-inc.com/tutorial4.html) shows 
you *can* do this kind of thing on top of a thread 
primitive, but might it not be much better done on 
a different primitive?

Again, I'm not really asking for an answer, but I think this type of problem is not uncommon, and a wonderful use of coroutines; so I'm wondering if this is a good trade-off.

> - u.yield() pauses the current uthread and resume the uthread
> u where it was paused.  The current uthread is resumed when
> some other uthread calls its yield() method.  Calling
> uthread.current().yield() is a no-op. 

This doesn't seem like enough: sort of as though you designed a 
language in which "call" and "return" were spelled the same way. 
Certainly for coroutines and generators, people gravitate towards 
paired operations (eg. "suspend" and "resume"). Again, Tim's 
demonstrated you can do that on top of threads, but it sure seems to 
me like they should be primitives.

> I think this API should enough to implement Gordon's
> SelectDispatcher code.  In general, it's easy to create a
> scheduler uthread that schedules other uthreads.

Thank you :-).

> Open issues:
> 
> - I'm not sure that I got the start conditions right.  Should
> func() be
>   be allowed to run until its first yield() when
>   uthread.new(func) is called?

For coroutine stuff, that doesn't bother me. For uthreads, I'd think (like 
real threads) that creation and starting are different things.
 
> - I'm not sure that the rules for returning and raising
> exceptions
>   from func() are the right ones.
> 
> - Should it be possible to pass a value to another uthread by
> passing
>   an argument to u.yield(), which then gets returned by the
>   resumed yield() call in that uthread?

Argument (in and / or out) passing is a necessity for generators and 
coroutines. But, as mentioned above, I don't think a symmetrical 
"yield" is the right answer.
 
> - How do uthreads interact with real threads?  Uthreads are
> explicitly
>   scheduled through yield() calls; real threads use preemptive
>   scheduling.  I suppose we could create a new "main" uthread for
>   each real thread.  But what if we yield() to an uthread that's
>   already executing in another thread?  How is that error
>   detected?

I think I would (perhaps naively) expect that I could create a uthread in 
one (real) thread, and then pass it off to another (real) thread to 
execute.

Another post brings up GUIs and uthreads. I already expect that I'm 
going to have to dedicate a (real) thread to the GUI and ponder very 
carefully how that thread interacts with others. Of course, that's from 
lessons learned the hard way; but personally I'm not expecting 
uthreads / coroutines to make that any easier.

[About continuations: while I love the fact that Christian has made 
these available for playing, I have so far not found them productive. I 
wrote a simple minded backtracking parser using them, but found it no 
better than a coroutine based one. But I am interested in how a *real* 
pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good* 
idea, boss", please.]

- Gordon



From akuchlin at mems-exchange.org  Wed Nov  8 19:11:26 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 08 Nov 2000 13:11:26 -0500
Subject: [Python-Dev] Catalog-SIG created
Message-ID: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>

A mailing list for the Catalog SIG has been created, to discuss the
design and construction of a Vaults/CPAN/LSM-like index for Python.
Web pages for the SIG don't exist yet, but will be created soon.

The SIG's charter:

The Python Catalog SIG aims at producing a master index of Python
software and other resources.  It will begin by figuring out what the
requirements are, converging on a design for the data schema, and
producing an implementation.  ("Implementation" will almost certainly
include mean a set of CGI scripts for browsing the catalog, and may
also contain a standard library module for automatically fetching &
installing modules, if the SIG decides that's a worthwhile feature.)

--amk



From guido at python.org  Wed Nov  8 19:21:39 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 13:21:39 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Demo/threads Coroutine.py,NONE,1.1 fcmp.py,NONE,1.1 squasher.py,NONE
In-Reply-To: Your message of "Wed, 08 Nov 2000 17:06:56 +0100."
             <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m13tXkW-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011081821.NAA23572@cj20424-a.reston1.va.home.com>

> Hmmmm.... I always thought of the Demo directory as a repository for
> example code, which may be used to teach Python programming to
> beginners.  I know, that some pieces are a way out of date.  
> 
> But I think it would be a worthwile goal to update at least some of
> those pieces step by step to reflect current Python coding habits.
> The README in Demo says:
> 
> """This directory contains various demonstrations of what you can do with
> Python. [...]"""
> 
> If you want to turn the Demo directory into a museum of code snippets of
> historic interest, at least the README should say so. ;-)

If you want to make a proposal for reorganizing the Demo directory,
please do so.  There are more important problems with the Demo
directlry than the fact that some code still uses string exceptions.
Please don't start fixing the small nits without seeing the big
picture.

(That's all I have time for now.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tismer at tismer.com  Wed Nov  8 18:20:43 2000
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 08 Nov 2000 19:20:43 +0200
Subject: [Python-Dev] uthread strawman
References: Your message of "Mon, 06 Nov 2000 16:23:48 +0200."             <3A06BEF4.95B773BD@tismer.com> <3A09428C.12529.1044B070@localhost>
Message-ID: <3A098B6B.522EA5E8@tismer.com>


Gordon McMillan wrote:

[snipped all the good stuff away for bigger brains than mine ]

> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

Yes, I saw Gordon making heavy use of naked continuations,
but actually they were not really what he needed. I believe
this since he made much use of co.update(), which moves
a continaution to the most current state of the frame.

In fact, what Gordon would need (and most probably most of
us as well) is just the handle to a frame, and the ability
to switch to it. In Gordon's case, these would probably
be "continuation" which are not frozen, but simply track
the frame as it is. I'm not absolutely shure, but quite.

I'm happy to toss continuations for core Python, if we
can find the right building blocks for coro/gen/uthreads.
I think Guido comes quite near this, already.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From fdrake at acm.org  Wed Nov  8 20:50:50 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 8 Nov 2000 14:50:50 -0500 (EST)
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us>
	<Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net>
Message-ID: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>

Mark W. Alexander writes:
 > Is there another way to subscribe, or did I just jump the gun?

  You can get to the Mailman interface at:

	http://www.python.org/mailman/listinfo/catelog-sig/

  The Web pages aren't actually there yet; Andrew will get to this
when he can, I'm sure.  ;)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Wed Nov  8 21:34:49 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Nov 2000 21:34:49 +0100
Subject: [Python-Dev] Re: Class/type dichotomy thoughts
References: <20001107225758.B77651301D9@oratrix.oratrix.nl>
Message-ID: <3A09B8E9.E247125F@lemburg.com>

Jack Jansen wrote:
> 
> > > In other words, what does this new type_id thing
> > > actually *mean*?
> >
> > For the interpreter it means that it can assume the type
> > interface to be binary compatible to the "original"
> > type, e.g. by setting the flag to say PyDict_TypeID
> > the type assures that all PyDict_*() APIs will work
> > on the type -- basically the same thing as PyDict_Check()
> > does now except that the type object needn't be the same
> > anymore.
> 
> I would be _very_ happy if this single type_id could somehow be
> replaced by an array, or a bitset.

I guess a bit array would be a possibility...

#define PyList_Check(obj) ((obj)->ob_type->\
                            capabilities[Py_ListType->cap_slot])
 
cap_slot could be set at type object creation time using
some Python slot id generator (a function which outputs integers
up to the maximum length of the capabilities array and raises
a Py_FatalError() if this limited is excceded).

> I have a lot of types in MacPython that are acceptable to the APIs of
> other types, a sort of poor-mans-inheritance scheme. For instance, all
> operating system calls that accept a MacOS WindowPtr will also happily
> accept a DialogPtr. Major magic is needed to get this to work
> reasonably in Python, and the Python user can still accidentally mess
> up the refcounting scheme and free things s/he isn't aware of.
> 
> As the number of types in a given run of the interpreter appears to be
> limited (am I right here?) and type-identity-tests are valid within a
> single interpreter run only (am I right here?) 

Right * 2

> an API like
>   typeindex = Py_TypeToTypeIndex(typeobject);
> which would use a dictionary as storage for the mapping and generate
> the index numbers on the fly would do the trick. Call it once during
> module initalization and the
>   Py_ISOBJECTCOMPATIBLEWITH(object, typeindex)
> macro would be a oneliner to test the bit in the set.
> 
> A few hundred bits in the set would get us a long way, I guess.

One thing I'm unsure about is whether changing cap_slot ids
between runs of the interpreter are a good idea. Also, I think that
the basic types should be given constant cat_slot ids to enhance
performance (the id generator could be made to start at say 10 
and the basic types be fixed in the range 0-9).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From trentm at ActiveState.com  Wed Nov  8 22:07:39 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 8 Nov 2000 13:07:39 -0800
Subject: [Python-Dev] Re: [Distutils] Catalog-SIG created
In-Reply-To: <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 08, 2000 at 02:50:50PM -0500
References: <E13tZh0-0003oe-00@kronos.cnri.reston.va.us> <Pine.A41.4.21.0011081450360.50480-100000@dakota.gate.net> <14857.44698.672928.206695@cj42289-a.reston1.va.home.com>
Message-ID: <20001108130739.H27185@ActiveState.com>

On Wed, Nov 08, 2000 at 02:50:50PM -0500, Fred L. Drake, Jr. wrote:
> 	http://www.python.org/mailman/listinfo/catelog-sig/ 

.replace("catelog", "catalog")

:)


-- 
Trent Mick
TrentM at ActiveState.com



From jeremy at alum.mit.edu  Thu Nov  9 01:14:45 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:14:45 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
Message-ID: <14857.60533.766291.786182@bitdiddle.concentric.net>

I just uploaded a patch to sourceforge that revises the CALL_FUNCTION
implementation to use a bunch of functions and avoid the long block of
inline code in eval_code2.

The overall performance of the patch is about the same.  The current
patch causes a big slowdown for the use of keyword arguments, but is
otherwise as fast or faster then the old version.  The keyword
slowdown should be avoidable without too much effort.

I wrote a short benchmark that demonstrates the effect on many
variations of function calls.  The output lists the test time, and the
median, min, and max time of 10 executions.  (The benchmark script is
attached to this message.)

The vanilla CVS tree produces these results:

time_arg1 0.09 0.09 0.11
time_arg2 0.1 0.09 0.12
time_arg3 0.11 0.1 0.12
time_fact 0.12 0.1 0.14
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.1 0.08 0.11
time_callable 0.37 0.33 0.38
time_keyword 0.14 0.13 0.18
time_star 0.12 0.12 0.14
time_kw 0.25 0.24 0.27
time_kw2 0.69 0.66 0.73
time_starkw 0.24 0.23 0.26
time_init 0.64 0.63 0.68
total 3.18

The CVS tree with the CALL_FUNCTION patch applied produces these
results: 
time_arg1 0.09 0.09 0.1
time_arg2 0.1 0.09 0.1
time_arg3 0.11 0.09 0.13
time_fact 0.11 0.11 0.14
time_meth 0.09 0.09 0.1
time_umeth 0.1 0.1 0.11
time_builtin 0.08 0.07 0.09
time_callable 0.35 0.34 0.38
time_keyword 0.42 0.4 0.44      (*** big slowdown ***)
time_star 0.13 0.13 0.15
time_kw 0.25 0.23 0.29
time_kw2 0.66 0.61 0.79
time_starkw 0.24 0.22 0.27
time_init 0.66 0.6 0.72
total 3.39

Jeremy

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: callbench.py
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001108/b099a7be/attachment-0001.asc>

From jeremy at alum.mit.edu  Thu Nov  9 01:25:11 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 8 Nov 2000 19:25:11 -0500 (EST)
Subject: [Python-Dev] Re: [Patch #102337] revised CALL_FUNCTION implementation
In-Reply-To: <14857.60533.766291.786182@bitdiddle.concentric.net>
References: <200011090008.QAA22011@sf-web2.i.sourceforge.net>
	<14857.60533.766291.786182@bitdiddle.concentric.net>
Message-ID: <14857.61159.339908.704603@bitdiddle.concentric.net>

Looks like I jumped the gun with my last message.  A trivial change to
the logic prevented the keyword arguments slowdown.  I've revised the
SF patch.  The new numbers show that the patch is just a tad faster on
the benchmark.  (And the difference between gcc -O2 and -O3 makes a
big difference for this ceval-intensive benchmark.)

Jeremy

time_arg1 0.1 0.09 0.12
time_arg2 0.1 0.1 0.12
time_arg3 0.1 0.1 0.12
time_fact 0.12 0.11 0.13
time_meth 0.1 0.09 0.11
time_umeth 0.11 0.1 0.12
time_builtin 0.09 0.08 0.1
time_callable 0.36 0.34 0.41
time_keyword 0.13 0.13 0.14
time_star 0.12 0.12 0.13
time_kw 0.23 0.22 0.24
time_kw2 0.64 0.61 0.66
time_starkw 0.24 0.23 0.26
time_init 0.64 0.64 0.69
total 3.08



From tim.one at home.com  Thu Nov  9 09:44:07 2000
From: tim.one at home.com (Tim Peters)
Date: Thu, 9 Nov 2000 03:44:07 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011071614.LAA18276@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>

[Guido]

[examples of presumably fuzzy semantics in the presence of continuations]

> ...
> But now consider this:  the "evaluation stack" contained in each
> frame could easily be replaced by a bunch of temporary variables,
> if we had a slightly different instruction set ...
> Then would we save those temporary variables or not?

Yes, provided they remained anonymous (to the user) they're simply internal
implementation details, and continuations preserve all things "like that".
It's only visible name bindings that continuations let change across
resumptions.

> it can make a difference!

Indeed yes.

> ...
> But if the compiler were to use temporary variables instead of the
> evaluation stack, they might not have been restored!

The technical term for that would be "bug" <0.5 wink>.

Christian already covered the for-loop example.  A more interesting
variation is a for loop without a named "indexing vrbl":

    for x in sequence:
        etc

Scheme has nothing direct to say about this because Scheme has no loops.
Writing it as a recursive function instead leads to the same kind of result,
though.

> ...
> This semantic imprecision is one of the reasons why I don't like the
> concept of continuations.

It's clearer in Scheme because Scheme has fewer "primitive concepts" than
Python.

> (I've been told that the exact semantics of continuations in Scheme
> differ greatly between Scheme implementations.)

I believe you've been told that, but not by a clueful Scheme user!
Continuations are rigorously well-defined in Scheme.  What *isn't*
well-defined in Scheme is order of evaluation in many cases, so an
expression like

   (+ (f) (g))

can display wildly different behavior across implementations if one or both
of the functions {f, g}, directly or indirectly, creates or invokes a
continuation (say that g does:  it's not defined whether f has been invoked
by the time g is -- indeed, it's even OK to invoke f and g in parallel).

Note:  the Python eye sees something like that and leaps to the rule "OK,
chowderhead, so don't muck with continuations in the middle of
expressions!".  What that misses is that *everything* in Scheme is "an
expression".  Scheme implementations do "differ greatly" in *this* respect.
BTW, note that Scheme implementations can also display wildly different
behavior if f and g merely have side effects (i.e., this really has nothing
to do with continuations specifically:  they're just another form of
side-effect you can't always predict without knowing the order of evaluation
first).

[skipping the proposal because we talked about it instead]

[Gordon McMillan]
> ...
> [About continuations: while I love the fact that Christian has made
> these available for playing, I have so far not found them productive. I
> wrote a simple minded backtracking parser using them, but found it no
> better than a coroutine based one. But I am interested in how a *real*
> pervert (eg, Tim) feels about it - and no "Gee, that sounds like a *good*
> idea, boss", please.]

I don't know of any comprehensible application of continuations that can't
be done without them.  The real appeal of continuations is in their
theoretical elegance (they're a single mechanism that can be used to build
all sorts of stuff, much as all of "if" and "while" and "for" can be built
out of compares and gotos -- continuations can be used to implement all of
calls, non-resumable and resumable exceptions, generators, coroutines,
non-deterministic evaluation, "thread like" stuff, ...).  In any specific
case, though, you want to live at the higher level, not at the raw
continuation level.

WRT a backtracking parser, note that this is what Icon *lives for*, and
generators alone suffice (and are indeed very pleasant) for natural
expression of that task.  Icon became what it is when Griswold decided to
open up the backtracking pattern-matching engine underlying SNOBOL4, and
make it the basis for all expression evaluation.  It took him 3 full
languages (SL5 and Rebus came before Icon) and 20 years to get this right.

Building a backtracking parser directly out of continuations sounds to me
mostly painful.  Building generators out of continuations *is* painful (I've
done it).  Curiously, the symmetry of coroutines appears to make building
them out of continuations easier (than building generators).

I'm not in love w/ continuations:  I *could* be if Guido got Continuation
Religion and wanted to redo exceptions and calls on top of continuations
too, but given that whatever happens here is destined (Christian will say
"doomed" <wink>) to co-exist with everything that's already here, the appeal
of continuations is minimal.  I've got no desire to play with novel new
control structures in Python (note that I don't consider generators-- or
even coroutines --to be "novel", not after they've been in multiple
languages for more than 30 years), and Python doesn't have the syntactic
flexibility that makes such experiments *pleasant* in Scheme anyway.  So
it's enough for me if Python supports the handful of new control-flow
gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
if we don't need continuations for those, fine by me.  BTW, I'd also like to
pickle a pure-Python computation in mid-stream, save it to disk, and resume
it later after a reboot (or on another machine!); we don't need
continuations for that either (although we do need much of what Stackless
does).

switching-from-redefining-truth-to-redefining-falsehood-ly y'rs  - tim




From tismer at tismer.com  Thu Nov  9 12:39:48 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 13:39:48 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com>
Message-ID: <3A0A8D04.3881E4FC@tismer.com>


Tim Peters wrote:
...

> Building a backtracking parser directly out of continuations sounds to me
> mostly painful.  Building generators out of continuations *is* painful (I've
> done it).  Curiously, the symmetry of coroutines appears to make building
> them out of continuations easier (than building generators).

Some things work very well, built with plain continuations.
See the attached ICON-style generator/backtrack framework
(going to post the 8 queens puzzle, soon).

> I'm not in love w/ continuations:  I *could* be if Guido got Continuation
> Religion and wanted to redo exceptions and calls on top of continuations
> too, but given that whatever happens here is destined (Christian will say
> "doomed" <wink>) to co-exist with everything that's already here, the appeal
> of continuations is minimal.  I've got no desire to play with novel new
> control structures in Python (note that I don't consider generators-- or
> even coroutines --to be "novel", not after they've been in multiple
> languages for more than 30 years), and Python doesn't have the syntactic
> flexibility that makes such experiments *pleasant* in Scheme anyway.

That's a very good point. Tricking Python to make continuations
useful is a pain in the a** and has led me to a quite weird API.
After the "sane" constructs are defined well, there is no much
reason to support continuations in the first place.

> So it's enough for me if Python supports the handful of new control-flow
> gimmicks (generators, coroutines, maybe uthreads) tasteful *users* ask for;
> if we don't need continuations for those, fine by me.  BTW, I'd also like to
> pickle a pure-Python computation in mid-stream, save it to disk, and resume
> it later after a reboot (or on another machine!); we don't need
> continuations for that either (although we do need much of what Stackless
> does).

There is one application of continuations which I still consider
worthy. I'm shure that many people find it incredibly ugly.
Using continuations, I can build method-like functions
without classes and instances, which perform incredibly
fast. This cannot be done with simple one-shot continuations;
of course a class method would do the same, but slower:

<script labguage="python">
function expensive_to_init_and_cheap_to_call(*args):
    pass # initialize many local variables
    param = continuation.return_current() # ***
    return do_cheap_calculation(param)

# generate and initialize such a function    

fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

# now we are at ***

fastfunc(42)  # can call it many times, at high speed.
</script>

This is a function with expensive initialization and many
local variables involved. After initializing, the
continuation of *** is returned as a callable object.
All initialization is done, all locals are set, and
now we can pull out many results by repeatedly calling
this continuation.
This cannot be modelled as efficiently today with classes.

ciao - chris

p.s.: Here the simple ICON-like generator/backtrack framework.



---------------------------------------------------------------------

import continuation

class control:
    """ ICON style generators """
    
    def __init__(self):
        # the chain of alternatives is a stack of tuples
        self.more = None

    def suspend(self, value):
        """ return a value, but keep the caller for re-use """
        # push the caller on the alternatives stack
        self.more = (continuation.caller(), self.more)
        # simulate a return of the caller with the current value
        continuation.caller(2)(value)

    def fail(self):
        """ restart an alternative, if there is some.
        Otherwise do nothing """
        if self.more:
            back, self.more = self.more
            back()

    def clear(self):
        """ clear alternatives stack """
        self.more = None

    def asn(self, old, val):
        """ an undoable assignment. Usage:
        var = c.asn(var, value)
        Like the ICON operator "<-"
        """
        self.suspend(val)
        print "asn returning"
        return(old)

    def choice(self, *args):
        """ iterator over a fixed sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        for val in args[:-1]:
            self.suspend(val)
        return args[-1]

    # the above works only for sequences of known size.
    # The version below is better since it does not need to
    # know the size, but it has to do a look-ahead.
    
    def choice(self, *args):
        """ iterator over a general sequence of values
        Like the ICON operator "|"
        """
        if len(args) == 1:
            args = args[0]
        
        # problem: how do we *return* the last element for any sequence?
        # solution: do a look-ahead by 1
        first = 1
        for nextval in args:
            if first:
                val = nextval
                first = 0
                continue
            self.suspend(val)
            val = nextval
        return val

    def to_by(self, a, b=None, c=None):
        """ a loop construct, modelled after the ICON
        for .. to .. by expression, but using xrange style
        """
        if c is None:
            if b is None:
                iterator = xrange(a)
            else:
                iterator = xrange(a,b)
        else:
            iterator = xrange(a,b,c)
        # problem: how do we *return* the last element?
        # solution: splitting the xrange is handy!
        for i in iterator[:-1]:
            self.suspend(i)
        return iterator[-1]

    # trying to get *slightly* more structure, here an
    # attempt to introduce something like "every".

---------------------------------------------------------------------

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From tismer at tismer.com  Thu Nov  9 14:04:00 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:04:00 +0200
Subject: [Python-Dev] uthread strawman
References: <LNBBLJKPBEHFEDALKOLCIEHGHPAA.tim.one@home.com> <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <3A0AA0C0.EA9C137B@tismer.com>


Christian Tismer wrote:

I wrote:
> There is one application of continuations which I still consider
> worthy. I'm shure that many people find it incredibly ugly.
> Using continuations, I can build method-like functions
> without classes and instances, which perform incredibly
> fast. This cannot be done with simple one-shot continuations;
> of course a class method would do the same, but slower:
> 
> <script language="python">
> function expensive_to_init_and_cheap_to_call(*args):
>     pass # initialize many local variables
>     param = continuation.return_current() # ***
>     return do_cheap_calculation(param)
> 
> # generate and initialize such a function
> 
> fastfunc = expensive_to_init_and_cheap_to_call(manyargs)

But this is again no argument for continuations, since
generators provide an equally performant solution:
(assuming a generator implementation, which passes the
generator object as first argument to itself. The
initialization part ends at self.start(), which
leaves the generator frame as is, and acts as an
entry point.)

function easily_done_with_generators(self, *args):
    pass # initialize many local variables
    param = self.start()
    while 1:
        result = do_cheap_calculation(param)
        param = self.suspend(result)

fastfunc = generator(easily_done_with_generators, manyargs)

fastfunc(42) # ...

----------------------

This implementation works without continuations and gives
equally fast generators.

Note the self.start() method: It is equivalent to the
continuation.return_current() function:
Leave the current frame as its own result, and provide
an entry point which accepts a value.

self.suspend() returns from the frame without ending it.
It returns a value, but preserves the entry point.

Does anybody know of a useful example where continuations
are really needed? The only remaining idea is at the moment
modelling of a case construct, but this reason is easily
removed by adding a switch statement to Python. :-)

cheers - chris   (stackless and continuationless)

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at math.huji.ac.il  Thu Nov  9 15:07:14 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:07:14 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA0C0.EA9C137B@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>

On Thu, 9 Nov 2000, Christian Tismer wrote:

> Does anybody know of a useful example where continuations
> are really needed?

Well, it may be a bit unpythonic (for several reasons), but restartable
exceptions AFAICS cannot be implemented by generators or coroutines.
Restartable exceptions are not always appropriate, but tend to be
a pain to simulate when they are needed.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From tismer at tismer.com  Thu Nov  9 14:26:17 2000
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 09 Nov 2000 15:26:17 +0200
Subject: [Python-Dev] uthread strawman
References: <Pine.GSO.4.10.10011091604530.5020-100000@sundial>
Message-ID: <3A0AA5F9.67C782E2@tismer.com>


Moshe Zadka wrote:
> 
> On Thu, 9 Nov 2000, Christian Tismer wrote:
> 
> > Does anybody know of a useful example where continuations
> > are really needed?
> 
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

(sorry for posting to python-dev/null but Starship is still down)

How would restartable exceptions work? Like so?

try:     # here, a continuation is saved
    pass # some operations which may fail,
         # raising something that inherits from RestartableException

except RestartableException:
    pass # repair the condition
    sys.try_again()   # go back to the try statement

Probably not, since this would be doable by just an internal
jump operation.
But if the restartable exception were a callable object,
continuations might be necessary, since we now have a fork
of two concurrently existing execution paths in the frame:
We might continue with the exception handling but pass the
restartable to someone else, who tries to call it later.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From moshez at math.huji.ac.il  Thu Nov  9 15:38:30 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 9 Nov 2000 16:38:30 +0200 (IST)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0AA5F9.67C782E2@tismer.com>
Message-ID: <Pine.GSO.4.10.10011091627510.5020-100000@sundial>

[Christian Tismer]
> Does anybody know of a useful example where continuations
> are really needed?

[Moshe Zadka]
> Well, it may be a bit unpythonic (for several reasons), but restartable
> exceptions AFAICS cannot be implemented by generators or coroutines.
> Restartable exceptions are not always appropriate, but tend to be
> a pain to simulate when they are needed.

[Christian Tismer]
> How would restartable exceptions work? Like so?
> 
> try:     # here, a continuation is saved
>     pass # some operations which may fail,
>          # raising something that inherits from RestartableException
> 
> except RestartableException:
>     pass # repair the condition
>     sys.try_again()   # go back to the try statement

I imagined something like

try:
	raise RestartableException, continuation.current_continuation()
except RestartableException, ex:
	# correct the problem
	ex.saved_continuation()

IOW, just put the continuation in the exception object.
I don't think you can use generators/coroutines to simulate something
like this
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov  9 16:30:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 10:30:03 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 16:38:30 +0200."
             <Pine.GSO.4.10.10011091627510.5020-100000@sundial> 
References: <Pine.GSO.4.10.10011091627510.5020-100000@sundial> 
Message-ID: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>

[Moshe]
> I imagined something like
> 
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
> 
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

You can make the exception handler into a separate coroutine -- then
no reusable continuations are needed.  The coroutine can repair the
exception and resume the other.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at pythonware.com  Thu Nov  9 17:02:31 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 9 Nov 2000 17:02:31 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
Message-ID: <00d801c04a66$7828bff0$0900a8c0@SPIFF>

> Remove AC_C_INLINE test from configure.in, since the only place the symbol
> occurs in the Python sources appears to be as text in comments.  We do not
> want to interfere with C++ keywords!

did you benchmark SRE before and after this change?

</F>




From fdrake at acm.org  Thu Nov  9 17:16:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Nov 2000 11:16:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: <00d801c04a66$7828bff0$0900a8c0@SPIFF>
References: <200011082023.MAA12997@slayer.i.sourceforge.net>
	<00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <14858.52676.775791.32394@cj42289-a.reston1.va.home.com>

Fredrik Lundh writes:
 > did you benchmark SRE before and after this change?

  No, but I don't see how this change would affect SRE.  _sre looks
like it takes care of itself with regard to inlining; the only thing
you might lose is:

	#undef inline

  Am I misreading something?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fredrik at effbot.org  Thu Nov  9 21:00:59 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Thu, 9 Nov 2000 21:00:59 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>
Message-ID: <00c101c04a87$c8d497f0$3c6340d5@hagrid>

I wrote:


> > Remove AC_C_INLINE test from configure.in, since the only place the symbol
> > occurs in the Python sources appears to be as text in comments.  We do not
> > want to interfere with C++ keywords!
> 
> did you benchmark SRE before and after this change?

following up on myself: SRE uses inline only if USE_INLINE is enabled,
and it's appears to be disabled in the 2.0 code base, for all platforms
except Windows (don't recall doing this; maybe someone else thought
that 10-20% performance hit was good enough?).  and now that Fred
has removed portable inline support, I cannot switch it on again :-(

besides, do people really expect to be able to use one compiler when
running configure, and another to compile Python?  sounds like they're
asking for it...

</F>




From billtut at microsoft.com  Thu Nov  9 21:28:41 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 12:28:41 -0800 
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com>

Christian Tismer wrote:
> Does anybody know of a useful example where continuations
> are really needed? 

Well, I don't know about needed, but let me explain a possible sever
architecture, and then see what new fangled control structure could help it
become more manageable. (irrespective if the perf #s of such an architecture
would actually be worth the architecture overhead in current CPython)

In a high performance server, any non-CPU intensive operation which blocks
the thread you're on reduces your scalability potential.

A fairly common way of writing multi-threaded servers is to have one client
per thread , whether this is either a thread pool, or just a simplistic
"Create thread, execute work, end thread" approach.

Threads are very expensive, and increases the context switch penalty your
server is inflicted with.

An alternative which reduces the context switch penalty dramatically is to
use a thread safe work item queue, and N threads where N is usually some
small multiple of the # of CPUs, and is < than the thread pool # of threads.
The solution to avoid these threads from blocking is an asynchronous state
machine. You asynchronously start the long-time operation, and on completion
of this operation insert the request state back in the thread safe queue.

You can further extend this idea to reduce the # of threads that you have so
that you have only 1 thread/CPU. Each thread being bound directly to the
CPU, and not allowed to run on other CPUs.
This tries to prevent the threads from CPU switching and ruining the CPU
cache, etc.
A work item must be able to be executed on any of these threads.

Another extension of this idea is to bundle these work items into separate
queues based on the state in the state machine.
The reason for doing this is trying to prevent unnecessary CPU cache
flushing.

The downside of this approach is that asynchronous state machines are a pain
to debug, maintain, understand, write, etc...
(BTW for the curious, the above architecture does allow you to achieve much
higher perf for certain tasks than other ways of handling the code, it's
been tested and used extensively in some internal C/C++ code (not mine))

The thought occurs to me that continuations would definitely help in this
situation. 
* You'd have more debugging state
* The code is organized around other needs besides boundaries between
blocking operations.

But it's not clear to me (mostly because I haven't applied a lot of thought
about it) if coroutines would suffice here.

Thoughts?

Bill




From loewis at informatik.hu-berlin.de  Thu Nov  9 22:11:00 2000
From: loewis at informatik.hu-berlin.de (Martin von Loewis)
Date: Thu, 9 Nov 2000 22:11:00 +0100 (MET)
Subject: [Python-Dev] [Python-Help] Driver?????
Message-ID: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>

> My name is Kelli Collins, and I work for the Data Management Team at
> IBM.  I am currently working with a customer that is considering
> using DB2 as their database software, and has asked me to see if I
> can locate a driver for Python and DB2...... can you tell if one
> exists??????

Hi Kelli,

I believe there is actually more than one way to get a database module
for Python and DB2. Please have a look at the DB SIG pages, at

  http://www.python.org/topics/database/

You'll see a DB/2 module being maintained at

  ftp://people.linuxkorea.co.kr/pub/DB2/

In addition, since DB2 offers the CLI, and an ODBC driver, you can
also use the ODBC module. For that, you have two options again: on
Windows, you can use the ODBC module that is in PythonWin. On all
systems, you can use mxODBC (make sure the licensing is acceptable,
though).

In any case, all of these comply with the DB API, so application code
should port easily among these options. I guess the difference then is
ease-of-installation, performance, and access to advanced features;
since I don't use DB2 myself, I can't recommend a particular choice.

Good luck,
Martin



From guido at python.org  Thu Nov  9 22:19:21 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 16:19:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure.in,1.178,1.179 config.h.in,2.78,2.79
In-Reply-To: Your message of "Thu, 09 Nov 2000 21:00:59 +0100."
             <00c101c04a87$c8d497f0$3c6340d5@hagrid> 
References: <200011082023.MAA12997@slayer.i.sourceforge.net> <00d801c04a66$7828bff0$0900a8c0@SPIFF>  
            <00c101c04a87$c8d497f0$3c6340d5@hagrid> 
Message-ID: <200011092119.QAA32327@cj20424-a.reston1.va.home.com>

> following up on myself: SRE uses inline only if USE_INLINE is enabled,
> and it's appears to be disabled in the 2.0 code base, for all platforms
> except Windows (don't recall doing this; maybe someone else thought
> that 10-20% performance hit was good enough?).  and now that Fred
> has removed portable inline support, I cannot switch it on again :-(

Can you explain yourself better, Fredrik?  I don't think that
USE_INLINE was ever defined.  I'd gladly accept a patch to
configure.in that defines USE_INLINE if 'inline' seems accepted by the
compiler.

> besides, do people really expect to be able to use one compiler when
> running configure, and another to compile Python?  sounds like they're
> asking for it...

Yes!  C++ uses include the same headers that were used to compile
Python, and C++ definitely qualifies as a different compiler...

--Guido van Rossum (home page: http://www.python.org/~guido/)




From greg at cosc.canterbury.ac.nz  Thu Nov  9 23:30:08 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Nov 2000 11:30:08 +1300 (NZDT)
Subject: [Python-Dev] uthread strawman
In-Reply-To: <3A0A8D04.3881E4FC@tismer.com>
Message-ID: <200011092230.LAA00785@s454.cosc.canterbury.ac.nz>

Christian Tismer <tismer at tismer.com>:

> Using continuations, I can build method-like functions
> without classes and instances

Couldn't you achieve the same thing with a closure, if 
Python acquires nested scopes?

My first attempt at implementing classes in Scheme
used continuations, until I realised that closures
were all that you needed.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From guido at python.org  Thu Nov  9 23:54:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 17:54:16 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: Your message of "Thu, 09 Nov 2000 12:28:41 PST."
             <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com> 
References: <58C671173DB6174A93E9ED88DCB0883D0A61B3@red-msg-07.redmond.corp.microsoft.com> 
Message-ID: <200011092254.RAA04553@cj20424-a.reston1.va.home.com>

> Christian Tismer wrote:
> > Does anybody know of a useful example where continuations
> > are really needed? 

[Bill Tutt]
> Well, I don't know about needed, but let me explain a possible sever
> architecture, and then see what new fangled control structure could help it
> become more manageable. (irrespective if the perf #s of such an architecture
> would actually be worth the architecture overhead in current CPython)
[...]
> But it's not clear to me (mostly because I haven't applied a lot of thought
> about it) if coroutines would suffice here.

Indeed, this kind of server architecture is typically done with
coroutines -- it's not a good argument for the full power of
continuations. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)




From mal at lemburg.com  Fri Nov 10 00:30:20 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 10 Nov 2000 00:30:20 +0100
Subject: [Python-Dev] [Python-Help] Driver?????
References: <200011092111.WAA13220@pandora.informatik.hu-berlin.de>
Message-ID: <3A0B338C.D1DD7CB2@lemburg.com>

Martin von Loewis wrote:
> 
> > My name is Kelli Collins, and I work for the Data Management Team at
> > IBM.  I am currently working with a customer that is considering
> > using DB2 as their database software, and has asked me to see if I
> > can locate a driver for Python and DB2...... can you tell if one
> > exists??????
> 

FYI, the next version of mxODBC will support DB/2 on all
platforms. It won't be freeware anymore, but in return you'll
get better support for Unicode and other ODBC 3.5 features.

The next round of mx.Modules will be published in the first
quarter of 2001. Until then you can use the existing mxODBC 1.1.1
which will work nicely on Windows and with some effort on Unix
for DB/2 too -- it'll give you a consistent interface to rely
on fo Windows and all supported Unix platforms.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From billtut at microsoft.com  Fri Nov 10 01:30:20 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Thu, 9 Nov 2000 16:30:20 -0800 
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>

Guido van Rossum [mailto:guido at python.org] wrote:
> > Christian Tismer wrote:
> > > Does anybody know of a useful example where continuations
> > > are really needed? 
> [Bill Tutt]
> > Well, I don't know about needed, but let me explain a possible sever
> > architecture, and then see what new fangled control structure could help
it
> > become more manageable. (irrespective if the perf #s of such an
architecture
> > would actually be worth the architecture overhead in current CPython)
> [...]
> > But it's not clear to me (mostly because I haven't applied a lot of
thought
> > about it) if coroutines would suffice here.

> Indeed, this kind of server architecture is typically done with
> coroutines -- it's not a good argument for the full power of
> continuations. :-)

Hrm, but can I get enough info from the coroutine support to
automatically determine which "work item queue" the paused routine
should go into after the "blocking" operation completes?
Additionally, the coroutine can't be stuck to only one system
thread. That would be bad.

If the above can't be done with coroutines, then I'm either stuck with
an async state machine, or something that might not scale as well as
it could. :(

(Ignoring the interesting discussion about how exactly you make this
architecture portable, since AFAIK only NT lets you bind threads to
CPUs)

I wish I had the time to play with stackless and hack up something
like I mentioned. The performance benefits to async state machine C++ code
that
use this idea are pretty impressive. CPU cache misses hurt. :(

Bill



From tismer at tismer.com  Fri Nov 10 12:39:24 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 10 Nov 2000 13:39:24 +0200
Subject: [Python-Dev] uthread strawman
References: <58C671173DB6174A93E9ED88DCB0883D0A61B4@red-msg-07.redmond.corp.microsoft.com>
Message-ID: <3A0BDE6C.1712F87E@tismer.com>


Bill Tutt wrote:
> 
> Guido van Rossum [mailto:guido at python.org] wrote:
> > > Christian Tismer wrote:
> > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > [Bill Tutt]

Bill on [server architecture, async state machine]

> > Indeed, this kind of server architecture is typically done with
> > coroutines -- it's not a good argument for the full power of
> > continuations. :-)
> 
> Hrm, but can I get enough info from the coroutine support to
> automatically determine which "work item queue" the paused routine
> should go into after the "blocking" operation completes?
> Additionally, the coroutine can't be stuck to only one system
> thread. That would be bad.

Coroutine vs. continuation isn't the point here, I think.
What you would likely need is gathering information about
the frames and variables in your co-whatever. This is quite
simple to implement, with either architecture.
My next version will not only allow to inspect the eval
stack of a frame, but also inspection and modification
of local variables; still this is no continuation feature.
We can simply inspect anything.

What makes the difference is callability:
Whenever we expose arbitrary frames as callable objects,
or we provide an unrestricted other way to execute them,
then exactly all the problems and consequences of
continuations pop up: we need to clone execution state,
since now it is possible to have more than one state for
a frame.
If we avoid this, we can still have all control and debugging
features with coroutines.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From billtut at microsoft.com  Fri Nov 10 20:49:40 2000
From: billtut at microsoft.com (Bill Tutt)
Date: Fri, 10 Nov 2000 11:49:40 -0800
Subject: [Python-Dev] uthread strawman
Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A61B7@red-msg-07.redmond.corp.microsoft.com>


From: 	Christian Tismer [mailto:tismer at tismer.com] 

> Bill Tutt wrote:
> > 
> > Guido van Rossum [mailto:guido at python.org] wrote:
> > > > Christian Tismer wrote:
> > > > > Does anybody know of a useful example where continuations
> > > > are really needed?
> > > [Bill Tutt]

> Bill on [server architecture, async state machine]

> > > Indeed, this kind of server architecture is typically done with
> > > coroutines -- it's not a good argument for the full power of
> > > continuations. :-)
> 
> > Hrm, but can I get enough info from the coroutine support to
> > automatically determine which "work item queue" the paused routine
> > should go into after the "blocking" operation completes?
> > Additionally, the coroutine can't be stuck to only one system
> > thread. That would be bad.

> Coroutine vs. continuation isn't the point here, I think.
> What you would likely need is gathering information about
> the frames and variables in your co-whatever. This is quite
> simple to implement, with either architecture.
> My next version will not only allow to inspect the eval
> stack of a frame, but also inspection and modification
> of local variables; still this is no continuation feature.
> We can simply inspect anything.

Coolness. 

> What makes the difference is callability:
> Whenever we expose arbitrary frames as callable objects,
> or we provide an unrestricted other way to execute them,
> then exactly all the problems and consequences of
> continuations pop up: we need to clone execution state,
> since now it is possible to have more than one state for
> a frame.
> If we avoid this, we can still have all control and debugging
> features with coroutines.

Ah, well this definitely would cause all sorts of confusing things.
I did mention I hadn't spent much time thinking about all of the
implementation details, I just know the requirements the architecture idea
has. :) As long as the co-whatevers can migrate to any system thread, I'm
one happy camper.

Thanks for helping clear stuff up folks,
Bill




From tim_one at email.msn.com  Sat Nov 11 10:47:10 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Sat, 11 Nov 2000 04:47:10 -0500
Subject: [Python-Dev] uthread strawman
In-Reply-To: <200011091530.KAA27521@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEMOHPAA.tim_one@email.msn.com>

[Moshe]
> I imagined something like
>
> try:
> 	raise RestartableException, continuation.current_continuation()
> except RestartableException, ex:
> 	# correct the problem
> 	ex.saved_continuation()
>
> IOW, just put the continuation in the exception object.
> I don't think you can use generators/coroutines to simulate something
> like this

[Guido]
> You can make the exception handler into a separate coroutine -- then
> no reusable continuations are needed.  The coroutine can repair the
> exception and resume the other.

Since I'm not a fan of resumable exceptions, I won't beat it to death.

Note that there are at least three meanings for "resumable exception" in
actual use (and I believe, e.g., that modern (ANSI) Smalltalk supports all
three):

1. Handler returns a value, and the failing operation acts as if that were
what it had computed from the start.  Very common in HW (e.g., a page fault
on a memory load acts this way; ditto 1e300**2 returning +Inf under 754
double arithmetic when the overflow signal is masked out).  Note that no
lexically enclosing "try block" is needed in this case (it is in the next
two, although they don't need "except" clauses attached to their lexically
enclosing "try").

2. Handler tells the entire originating "try block" to start over.

3. Handler resumes right after the originating try block.

In all of these, you have to search "up the call stack" for a handler, which
then wants to resume as if the call stack had not been unwound (albeit that
in #2 and #3, wants to resume at different places than where the exception
was raised!).  "Up the call stack" means within your current thread of
control, so it's hard to see how to make an independent coroutine out of
that on demand (the handler needs to work in its natural lexical context,
incl. modifying values of *its* scope's locals that will be visible again
when the deeper routine with the exception eventually does normal returns
back to it).

The details are excruciating -- so it's a good thing nobody in their right
mind wants any of these <wink>.

relievedly y'rs  - tim





From mal at lemburg.com  Sat Nov 11 10:52:27 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 10:52:27 +0100
Subject: [Python-Dev] Help wanted...
Message-ID: <3A0D16DB.A8724B9B@lemburg.com>

Hi everybody,

I just wanted to inform you that I won't be able to work on the
patches needed to fix the bugs assigned to me for the next month.
I currently have to finish a huge project with a deadline in mid-
December which has high priority (anyone ever write a B2B
eCommerce order solution in one month ?) ... I hope Python
get's me there in time.

Anyway, if any of the bugs currently assigned to me need a
fast fix, I'd suggest someone else taking a look at them, since
I won't be able to do much about these until around Christmas.
If you need help about some Unicode detail, just drop me line.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov 11 15:05:38 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Nov 2000 15:05:38 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <3A0D5232.CF6A884@lemburg.com>

I am looking for a way to do DNS resolution with a timeout
on the lookup. The easy way to do these lookups using the
socket module often result in processes hanging for several
seconds (up to minutes on Windows), so I figured emulating the
standard DNS lookup (including the /etc/hosts and /etc/resolv.conf
files) would be a good idea.

I know of the Demo/dns/, but I'm unsure whether it can be
tweaked to allow timeouts.

Are there any other such implementations out there ?

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From thomas at xs4all.net  Sat Nov 11 15:23:58 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 11 Nov 2000 15:23:58 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: <3A0D5232.CF6A884@lemburg.com>; from mal@lemburg.com on Sat, Nov 11, 2000 at 03:05:38PM +0100
References: <3A0D5232.CF6A884@lemburg.com>
Message-ID: <20001111152358.O27208@xs4all.nl>

On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:

> I am looking for a way to do DNS resolution with a timeout on the lookup.
> [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> allow timeouts.

> Are there any other such implementations out there ?

Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
timeouts, but it does do async lookups ;) He posted this a couple of days
ago, to the python list:

---------------------------------------------------------------------------
From: Anthony Baxter <anthony at interlink.com.au>
To: support at internetdiscovery.com (Mike Clarkson)
Cc: python-list at python.org
Subject: Re: Async DNS? 

The python DNS library I released a while ago (based on Guido's code)
did this. I just had a poke around and realised that it's not visible
anywhere on the net - I just dropped a copy into

http://www.zope.org/Members/anthony/software/dns.tar.gz
(sorry DC guys for abusing the zope.org site - to make it fair, I'll
whip up a DNS zope product at some point :)

I haven't looked at it in an age or two, but it used to work fine. :)

Anthony

>>> Mike Clarkson wrote
> Does anyone have asynchronous DNS lookup for Python?
> 
> Presumably it could be done in Python using sockets, but someone
> must have done a C implementation.
> 
> Many thanks in advance,
> 
> Mike.
> -- 
> http://www.python.org/mailman/listinfo/python-list
> 

-- 
Anthony Baxter     <anthony at interlink.com.au>   
It's never too late to have a happy childhood.
---------------------------------------------------------------------------

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Sat Nov 11 19:15:52 2000
From: guido at python.org (Guido van Rossum)
Date: Sat, 11 Nov 2000 13:15:52 -0500
Subject: [Python-Dev] DNS lookups with timeouts ?
In-Reply-To: Your message of "Sat, 11 Nov 2000 15:05:38 +0100."
             <3A0D5232.CF6A884@lemburg.com> 
References: <3A0D5232.CF6A884@lemburg.com> 
Message-ID: <200011111815.NAA12217@cj20424-a.reston1.va.home.com>

> I know of the Demo/dns/, but I'm unsure whether it can be
> tweaked to allow timeouts.

Hm, typically the protocol used is UDP, and it's easy to set timeouts
there: just use select on the socket.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From martin at loewis.home.cs.tu-berlin.de  Sun Nov 12 00:58:12 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 12 Nov 2000 00:58:12 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
Message-ID: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>

> Are there any other such implementations out there ?

How about http://dustman.net/andy/python/adns-python?

Regards,
Martin

P.S. Is this really on-topic for python-dev?



From moshez at zadka.site.co.il  Sun Nov 12 21:03:00 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Sun, 12 Nov 2000 22:03:00 +0200
Subject: [Python-Dev] Weak References
Message-ID: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>

For those of you who do not read c.l.py

From: shindich at my-deja.com
Newsgroups: comp.lang.python
Subject: Weak References
Date: Sun, 12 Nov 2000 11:07:32 GMT

Ever wanted to have weak references in Python?
Pre-alpha release of PyWeakReference project is now available on Source
Forge (http://sourceforge.net/projects/pyweakreference/)
Documentation is coming soon... (some docs are available in the Doc
subdirectory of the zip file)


Sent via Deja.com http://www.deja.com/
Before you buy.
- -- 
http://www.python.org/mailman/listinfo/python-list



------- End of Forwarded Message

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Sun Nov 12 13:55:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:55:42 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <200011112358.AAA00730@loewis.home.cs.tu-berlin.de>
Message-ID: <3A0E934E.8BC64ED3@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Are there any other such implementations out there ?
> 
> How about http://dustman.net/andy/python/adns-python?

Thanks for the hint.
 
> Regards,
> Martin
> 
> P.S. Is this really on-topic for python-dev?

Not really... but I thought since this question often arises
when using sockets, it might be a good idea to have something
like dnslib in the standard lib together with some support
for doing DNS lookups with predefined timeouts.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 13:58:44 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 13:58:44 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <200011111815.NAA12217@cj20424-a.reston1.va.home.com>
Message-ID: <3A0E9404.68BD77D0@lemburg.com>

Guido van Rossum wrote:
> 
> > I know of the Demo/dns/, but I'm unsure whether it can be
> > tweaked to allow timeouts.
> 
> Hm, typically the protocol used is UDP, and it's easy to set timeouts
> there: just use select on the socket.

Hmm, yes that would be a possibility. I'll see if I can come
up with some code which implements this.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 14:00:05 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:00:05 +0100
Subject: [Python-Dev] DNS lookups with timeouts ?
References: <3A0D5232.CF6A884@lemburg.com> <20001111152358.O27208@xs4all.nl>
Message-ID: <3A0E9455.83AE479E@lemburg.com>

Thomas Wouters wrote:
> 
> On Sat, Nov 11, 2000 at 03:05:38PM +0100, M.-A. Lemburg wrote:
> 
> > I am looking for a way to do DNS resolution with a timeout on the lookup.
> > [..] I know of the Demo/dns/, but I'm unsure whether it can be tweaked to
> > allow timeouts.
> 
> > Are there any other such implementations out there ?
> 
> Anthony Baxter did a lot of work on Demo/dns. Unsure whether it does
> timeouts, but it does do async lookups ;) He posted this a couple of days
> ago, to the python list:

I did know of that lib -- but it seems a bit over the top for the
simple lookups I hav in mind.
 
> ---------------------------------------------------------------------------
> From: Anthony Baxter <anthony at interlink.com.au>
> To: support at internetdiscovery.com (Mike Clarkson)
> Cc: python-list at python.org
> Subject: Re: Async DNS?
> 
> The python DNS library I released a while ago (based on Guido's code)
> did this. I just had a poke around and realised that it's not visible
> anywhere on the net - I just dropped a copy into
> 
> http://www.zope.org/Members/anthony/software/dns.tar.gz
> (sorry DC guys for abusing the zope.org site - to make it fair, I'll
> whip up a DNS zope product at some point :)
> 
> I haven't looked at it in an age or two, but it used to work fine. :)
> 
> Anthony
> 
> >>> Mike Clarkson wrote
> > Does anyone have asynchronous DNS lookup for Python?
> >
> > Presumably it could be done in Python using sockets, but someone
> > must have done a C implementation.
> >
> > Many thanks in advance,
> >
> > Mike.

Thanks,
-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sun Nov 12 14:17:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Nov 2000 14:17:33 +0100
Subject: [Python-Dev] Weak References
References: <E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <3A0E986D.43B170BA@lemburg.com>

Moshe Zadka wrote:
> 
> For those of you who do not read c.l.py
> 
> From: shindich at my-deja.com
> Newsgroups: comp.lang.python
> Subject: Weak References
> Date: Sun, 12 Nov 2000 11:07:32 GMT
> 
> Ever wanted to have weak references in Python?
> Pre-alpha release of PyWeakReference project is now available on Source
> Forge (http://sourceforge.net/projects/pyweakreference/)
> Documentation is coming soon... (some docs are available in the Doc
> subdirectory of the zip file)

Since more and more of these implementations are popping up
like mushrooms ;-) ... could you add some discussion about the
different methods of adding weak refs to Python to the PEP 205 ?

--
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Sun Nov 12 14:55:51 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Sun, 12 Nov 2000 15:55:51 +0200 (IST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011121554400.1528-100000@sundial>

On Sun, 12 Nov 2000, M.-A. Lemburg wrote:

> Since more and more of these implementations are popping up
> like mushrooms ;-) ... could you add some discussion about the
> different methods of adding weak refs to Python to the PEP 205 ?

Well, it's one of the few PEPs which are not yet mine <wink>:
I suppose Fred will do that pretty soon, though.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Mon Nov 13 13:54:43 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 07:54:43 -0500
Subject: [Python-Dev] Nested functions and class scope
In-Reply-To: Your message of "Mon, 13 Nov 2000 10:24:51 +1100."
             <3A0F26C3.7CD823BF@ozemail.com.au> 
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
            <3A0F26C3.7CD823BF@ozemail.com.au> 
Message-ID: <200011131254.HAA22379@cj20424-a.reston1.va.home.com>

John Max Skaller wrote:
> Guido van Rossum wrote:
> ===============>>>>
> > Other issues that need sorting out:
> > 
> > x = 2
> > class C:
> >   x = 1
> >   C = 'some string'
> >   def a(self):
> >      print x
> >   def b(self):
> >      global x
> >      x = 3
> > 
> > class D(C):
> >   C = 'some string'
> >   def a(self):
> >      C.a(self)
> >      print C
> > 
> > o = C()
> > o.a()
> > o.b()
> > o.a()
> > 
> > o = D()
> > o.a()
> > 
> > What would the output look like under your proposal ?
> 
> This is a good point!  If we considered the class as a nested scope
> here, I think it might break too much code, plus it would allow a new
> coding style where you could reference class variables without a self
> or <classname> prefix.  I don't like that prospect, so I'm in favor
> for ruling this out.
> <<<===================
> 
> You're missing the point. you can refer to class variables without
> a prefix right now:
> 
> class X:
> 	x = 1
> 	y = x
> 
> 
> It would be madness NOT to support lexically scoped nested classes
> as well as functions. But note that these scopes only exist
> when a class is executed, that is, when the class is defined.
> This has NO impact on finding class variables from an instance.
> 
> output of above from Vyper:
> --------------------------------------------------------------
> Viperi 2.0.1
> Copyright Maxtal P/L, John Skaller, Australia, 1999
> >>>x = 2
> ...class C:
> ...  x = 1
> ...  C = 'some string'
> ...  def a(self):
> ...     print x
> ...  def b(self):
> ...     global x
> ...     x = 3
> ...
> >>>class D(C):
> ...  C = 'some string'
> ...  def a(self):
> ...     C.a(self)
> ...     print C
> ...
> >>>o = C()
> ...o.a()
> ...o.b()
> ...o.a()
> ...
> 1 
> 3 
> >>>o = D()
> ...o.a()
> ...
> Error Near line 4
> 
> Uncaught Python Exception at top level!!
>   .. Kind: Instance of AttributeError
>   .. Attributes:
>       args --> Cannot find attribute of "some string" "a"
> 
> Traceback (innermost first)
> File: <string> Line: 4
> <cannot get line from file>
> File: <string> Line: 2
> <cannot get line from file>
> 
> -- 
> John (Max) Skaller, mailto:skaller at maxtal.com.au
> 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
> checkout Vyper http://Vyper.sourceforge.net
> download Interscript http://Interscript.sourceforge.net

Looks like you are missing the point.  Of course I want to keep the
class as a local scope *inside the class def*.  But I need it out of
the way when executing methods.  The example code should print

2			# a() references global x, not C.x
3			# global x got set to three by b()
2			# C refers to global class C
<class C ...>		# ditto

I agree that this is surprising.  But I believe it would break too
much code if we implemented your (simpler) scheme.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Mon Nov 13 15:09:46 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 13 Nov 2000 09:09:46 -0500 (EST)
Subject: [Python-Dev] Weak References
In-Reply-To: <3A0E986D.43B170BA@lemburg.com>
References: <3A0E986D.43B170BA@lemburg.com>
	<Pine.GSO.4.10.10011121554400.1528-100000@sundial>
	<E13v3LB-0002SH-00@darjeeling.zadka.site.co.il>
Message-ID: <14863.63018.678116.930743@cj42289-a.reston1.va.home.com>

M.-A. Lemburg writes:
 > Since more and more of these implementations are popping up
 > like mushrooms ;-) ... could you add some discussion about the
 > different methods of adding weak refs to Python to the PEP 205 ?

Moshe Zadka writes:
 > Well, it's one of the few PEPs which are not yet mine <wink>:
 > I suppose Fred will do that pretty soon, though.

  I plan to do this sometime this week.  I have a bunch of text that
isn't already checked in, so this isn't the time to open up editorial
control just yet.
  The notice Moshe forwarded didn't slip past me, but I've been a
little under the weather the past few days.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From nas at arctrix.com  Mon Nov 13 09:36:53 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 13 Nov 2000 00:36:53 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001113003653.A13421@glacier.fnational.com>

I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil



From mal at lemburg.com  Mon Nov 13 16:32:04 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 13 Nov 2000 16:32:04 +0100
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
References: <20001113003653.A13421@glacier.fnational.com>
Message-ID: <3A100974.969045C0@lemburg.com>

Neil Schemenauer wrote:
> 
> I making this available now in the hope that it will generate
> some dicussion:
> 
>     http://arctrix.com/nas/python/coerce-2.diff
> 
> The patch is based on Marc-Andre Lemburg's coercion patch for
> 1.5.1 downloaded from the starship:
> 
>     http://starship.python.net/~lemburg/CoercionProposal.html
> 
> All credit goes to him.  The bugs are probably introduced by me.
> What the patch does:
> 
>   - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
>     set then the tp_as_number methods are treated differently.
>     Coercion is not done before passing arguments to the method.
>     New style methods do their own coercion or return the
>     NotImplemented singleton.  See Marc's page for more details.

Very nice !

I wanted to suggest the usage of a type flag
instead of the slot magic in the original patch too -- your time
machine was faster ;-)
 
>   - Decide on handling of nb_coerce slot (eg. should it be called
>     on new style numbers).

Probably not, since all coercion is now handled by the slots
themselves.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From trentm at ActiveState.com  Mon Nov 13 22:16:26 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:16:26 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011131948.LAA04697@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Mon, Nov 13, 2000 at 11:48:25AM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net>
Message-ID: <20001113131626.B4553@ActiveState.com>

On Mon, Nov 13, 2000 at 11:48:25AM -0800, Guido van Rossum wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv4689
> 
> Modified Files:
> 	fileobject.c 
> Log Message:
> Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> hack for TELL64()...  Sounds like there's something else going on
> really.  Does anybody have a clue I can buy?

Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
when it should not really be. Essentially, these platforms lie when they say
they support largefiles, at least according to the configure test for
HAVE_LARGEFILE_SUPPORT>

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From guido at python.org  Mon Nov 13 22:38:01 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 16:38:01 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:16:26 PST."
             <20001113131626.B4553@ActiveState.com> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net>  
            <20001113131626.B4553@ActiveState.com> 
Message-ID: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>

> > Log Message:
> > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > hack for TELL64()...  Sounds like there's something else going on
> > really.  Does anybody have a clue I can buy?
> 
> Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> when it should not really be. Essentially, these platforms lie when they say
> they support largefiles, at least according to the configure test for
> HAVE_LARGEFILE_SUPPORT>

Mh...  Thanks.  And how are we testing for large file support?
Perhaps the test should be improved rather than hacks piled on top of
each other?  What's wrong with the current test?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From trentm at ActiveState.com  Mon Nov 13 22:59:05 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 13:59:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132138.QAA25483@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 04:38:01PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>
Message-ID: <20001113135905.B7508@ActiveState.com>

On Mon, Nov 13, 2000 at 04:38:01PM -0500, Guido van Rossum wrote:
> > > Log Message:
> > > Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
> > > hack for TELL64()...  Sounds like there's something else going on
> > > really.  Does anybody have a clue I can buy?
> > 
> > Yes, there is. For all of these platforms HAVE_LARGEFILE_SUPPORT is defined
> > when it should not really be. Essentially, these platforms lie when they say
> > they support largefiles, at least according to the configure test for
> > HAVE_LARGEFILE_SUPPORT>
> 
> Mh...  Thanks.  And how are we testing for large file support?
> Perhaps the test should be improved rather than hacks piled on top of
> each other?  What's wrong with the current test?
> 

I agree: the test should improve. Here it is:

AC_MSG_CHECKING(whether to enable large file support)
if test "$have_long_long" = yes -a \
    "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
    "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
  AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
  AC_MSG_RESULT(yes)
else
  AC_MSG_RESULT(no)
fi


BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
(i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
think). In windows is is _tell64, I think. ...anyway there are a lot of ways
to spell it.

I don't know right away how to translate that into an appropriate configure
test. I.e. how do you test that the platform has an ftell/fseek-like function
that uses an index variable whose sizeof() is at least 8. 

Note that, IIRC, windows had some funny case where the test above would have
failed but that was okay because there were lower level I/O functions that
used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
details. 



Trent


-- 
Trent Mick
TrentM at ActiveState.com



From akuchlin at mems-exchange.org  Mon Nov 13 23:13:58 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Mon, 13 Nov 2000 17:13:58 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113135905.B7508@ActiveState.com>; from trentm@ActiveState.com on Mon, Nov 13, 2000 at 01:59:05PM -0800
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com>
Message-ID: <20001113171358.A7407@kronos.cnri.reston.va.us>

On Mon, Nov 13, 2000 at 01:59:05PM -0800, Trent Mick wrote:
>I don't know right away how to translate that into an appropriate configure
>test. I.e. how do you test that the platform has an ftell/fseek-like function
>that uses an index variable whose sizeof() is at least 8. 

If you look at the very latest autoconf alpha release, autoconf 2.49a,
at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
macro that may be what's needed.  Perhaps we can back-port this macro
to the current autoconf release.

--amk



From thomas at xs4all.net  Mon Nov 13 23:20:20 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:20:20 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <20001113171358.A7407@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 13, 2000 at 05:13:58PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <20001113171358.A7407@kronos.cnri.reston.va.us>
Message-ID: <20001113232020.R27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:13:58PM -0500, Andrew Kuchling wrote:

> If you look at the very latest autoconf alpha release, autoconf 2.49a,
> at ftp://alpha.gnu.org/pub/gnu/autoconf/, it has an AC_SYS_LARGEFILE
> macro that may be what's needed.  Perhaps we can back-port this macro
> to the current autoconf release.

Aahh, hmm. I'll look at what this macro does tomorrow, unless someone else
beats me to it. I've been busy, between gobs of work, trying to find out how
*other* applications determine large file support... that's suprisingly
tricky in some cases -- I still haven't figured out how zsh does it.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Mon Nov 13 23:36:06 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:36:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 13:59:05 PST."
             <20001113135905.B7508@ActiveState.com> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com>  
            <20001113135905.B7508@ActiveState.com> 
Message-ID: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>

> > Mh...  Thanks.  And how are we testing for large file support?
> > Perhaps the test should be improved rather than hacks piled on top of
> > each other?  What's wrong with the current test?
> > 
> 
> I agree: the test should improve. Here it is:
> 
> AC_MSG_CHECKING(whether to enable large file support)
> if test "$have_long_long" = yes -a \
>     "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \
>     "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then
>   AC_DEFINE(HAVE_LARGEFILE_SUPPORT)
>   AC_MSG_RESULT(yes)
> else
>   AC_MSG_RESULT(no)
> fi
> 
> 
> BSDI, etc pass this test but do not have 64-bit capable ftell/fseek functions
> (i.e. ones that use that off_t variable that is as big as a LONG_LONG). In
> posix these are called ftello/fseeko. We also check for ftell64/fseek64 (I
> think). In windows is is _tell64, I think. ...anyway there are a lot of ways
> to spell it.
> 
> I don't know right away how to translate that into an appropriate configure
> test. I.e. how do you test that the platform has an ftell/fseek-like function
> that uses an index variable whose sizeof() is at least 8. 
> 
> Note that, IIRC, windows had some funny case where the test above would have
> failed but that was okay because there were lower level I/O functions that
> used a 64-bit capable fpos_t (instead of off_t). I can't remember the exact
> details. 

After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
means that the low-level system calls (lseek(), stat() etc.)  support
large files, through an off_t type that is at least 8 bytes (assumed
to be equivalent with a long long in some places, given the use of
PyLong_FromLongLong() and PyLong_AsLongLong()).

But the problems occur in fileobject.c, where we're dealing with the
stdio library.  Not all stdio libraries seem to support long files in
the same way, and they use a different typedef, fpos_t, which may be
larger or smaller in size than off_t.

Aha!  It looks like the problem is that under a variety of
circumstances TELL64(fd) is required; but these have in common that
HAVE_LARGEFILE_SUPPORT is defined, and in all cases except MS_WIN64 it
is defined as equal to lseek((fd),0,SEEK_CUR).

So wouldn't the problem be solved if we changed the #ifdefs so that
that is the default definition, instead of only defining that for
specific platforms?  As more and more platforms develop kernel support
for 64-bit offsets but their stdio libraries lag behind (or use APIs
we don't know about, or assume that fsetpos() is sufficient), we'll
run into this more and more often.

Here's the current logic for defining TELL64() in fileobject.c:

#if defined(MS_WIN64)
#define TELL64 _telli64
#elif defined(__NetBSD__) || ... /* etc. */
/* NOTE: this is only used on older
   NetBSD prior to f*o() funcions */
#define TELL64(fd) lseek((fd),0,SEEK_CUR)
#endif

There's exactly one place where TELL64() is subsequently used, which
is inside

#elif defined(HAVE_LARGEFILE_SUPPORT) && SIZEOF_FPOS_T >= 8
#else

Any objections???

--Guido van Rossum (home page: http://www.python.org/~guido/)



From thomas at xs4all.net  Mon Nov 13 23:41:21 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 13 Nov 2000 23:41:21 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132236.RAA25734@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:36:06PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>
Message-ID: <20001113234121.S27208@xs4all.nl>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:

> After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> means that the low-level system calls (lseek(), stat() etc.)  support
> large files, through an off_t type that is at least 8 bytes (assumed
> to be equivalent with a long long in some places, given the use of
> PyLong_FromLongLong() and PyLong_AsLongLong()).

> But the problems occur in fileobject.c, where we're dealing with the
> stdio library.  Not all stdio libraries seem to support long files in
> the same way, and they use a different typedef, fpos_t, which may be
> larger or smaller in size than off_t.

This isn't the problem. The problem is that we assume that because off_t is
8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
bytes, but none of the available fseek/ftell variations take an off_t as
argument ;P The TELL64() workaround works around that problem, but still
doesn't enable large file support, because there isn't any such support in
BSDI.

(Trust me... we've had logfiles and IP-traffic databases truncated because
of that... 2Gb is the limit, currently, on BSDI.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From guido at python.org  Mon Nov 13 23:52:46 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2000 17:52:46 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: Your message of "Mon, 13 Nov 2000 23:41:21 +0100."
             <20001113234121.S27208@xs4all.nl> 
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com>  
            <20001113234121.S27208@xs4all.nl> 
Message-ID: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>

> On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> 
> > After a bit of grepping, it seems that HAVE_LARGEFILE_SUPPORT reliably
> > means that the low-level system calls (lseek(), stat() etc.)  support
> > large files, through an off_t type that is at least 8 bytes (assumed
> > to be equivalent with a long long in some places, given the use of
> > PyLong_FromLongLong() and PyLong_AsLongLong()).
> 
> > But the problems occur in fileobject.c, where we're dealing with the
> > stdio library.  Not all stdio libraries seem to support long files in
> > the same way, and they use a different typedef, fpos_t, which may be
> > larger or smaller in size than off_t.
> 
> This isn't the problem. The problem is that we assume that because off_t is
> 8 bytes, we have_LARGE_FILE_SUPPORT. This isn't true. On BSDI, off_t *is* 8
> bytes, but none of the available fseek/ftell variations take an off_t as
> argument ;P The TELL64() workaround works around that problem, but still
> doesn't enable large file support, because there isn't any such support in
> BSDI.
> 
> (Trust me... we've had logfiles and IP-traffic databases truncated because
> of that... 2Gb is the limit, currently, on BSDI.)

Sure, but the #ifdef isn't really about how well the kernel supports
large files -- it is about what code you must use.  There's one set of
places that uses off_t, and those are guided just fine by
HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
larger than 2Gb!

But there's a *different* set of test that must be used to determine
what to do for the stdio calls.  Note that on all platforms so far
where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
and Mac OSX), there were only compilation problems in fileobject.c,
and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).

What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
doesn't actually support files larger than 2Gb?  I suppose you get
some sort of I/O error when you try to write such files.  Since they
can't be created, you can't run into trouble when trying to read
them! :-)

Or am I still missing something?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From trentm at ActiveState.com  Tue Nov 14 00:07:05 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Mon, 13 Nov 2000 15:07:05 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects fileobject.c,2.91,2.92
In-Reply-To: <200011132252.RAA25873@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 13, 2000 at 05:52:46PM -0500
References: <200011131948.LAA04697@slayer.i.sourceforge.net> <20001113131626.B4553@ActiveState.com> <200011132138.QAA25483@cj20424-a.reston1.va.home.com> <20001113135905.B7508@ActiveState.com> <200011132236.RAA25734@cj20424-a.reston1.va.home.com> <20001113234121.S27208@xs4all.nl> <200011132252.RAA25873@cj20424-a.reston1.va.home.com>
Message-ID: <20001113150705.A12976@ActiveState.com>

On Mon, Nov 13, 2000 at 05:36:06PM -0500, Guido van Rossum wrote:
> Sure, but the #ifdef isn't really about how well the kernel supports
> large files -- it is about what code you must use.  There's one set of
> places that uses off_t, and those are guided just fine by
> HAVE_LARGEFILE_SUPPORT -- whether or not you can actually have files
> larger than 2Gb!
> 
> But there's a *different* set of test that must be used to determine
> what to do for the stdio calls.  Note that on all platforms so far
> where TELL64 was needed (5 in total: MS_WIN64, NetBSD, OpenBSD, BSDI,
> and Mac OSX), there were only compilation problems in fileobject.c,
> and they went away by defining TELL64 as lseek((fd),0,SEEK_CUR).
> 
> What goes wrong if HAVE_LARGEFILE_SUPPORT is defined but the system
> doesn't actually support files larger than 2Gb?  I suppose you get
> some sort of I/O error when you try to write such files.  Since they
> can't be created, you can't run into trouble when trying to read
> them! :-)
> 
> Or am I still missing something?

I don't *think* so. Though having HAVE_LARGEFILE_SUUPORT still be defined for
a platform where this is not true still nags.

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From jeremy at alum.mit.edu  Tue Nov 14 05:16:01 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 13 Nov 2000 23:16:01 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
Message-ID: <14864.48257.324726.976892@bitdiddle.concentric.net>

I just saw your note about whether IDLE was packaged with the Python
RPMs.  It isn't, nor is anything in the Tools directory.  I did the
simplest thing possible with the RPM and ran "make install".  The RPM
installs whatever is installed as a result of that.

We've discussed installation of other tools once or twice, but I've
never decided what exactly to do.  The compiler package, for example,
comes with a distutils setup.py script that will do the job.  I think
the best way to move forward is to write setup.py scripts for all the
other tools that ought to be installed.

I don't know how to package up idle cleanly, though.  I need to decide
what to install and where to install it.  The Tools/idle directory is
nominally a package, but it doesn't seem like that will help much for
installing it.  If we want to call the executable script idle.py, it
will clash with the idle package name.  Any advice here?  It doesn't
seem like there is any obvious way to "install" IDLE; it looks like it
is intended to be run out of the source directory.

If anyone else owns (or otherwise feels responsible for) a package in
the Tools directory that ought to be included with the RPMs, please
write a setup script for it.

Jeremy



From moshez at zadka.site.co.il  Tue Nov 14 14:19:11 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 14 Nov 2000 15:19:11 +0200
Subject: [Python-Dev] installing IDLE & other Tools 
In-Reply-To: Message from Jeremy Hylton <jeremy@alum.mit.edu> 
   of "Mon, 13 Nov 2000 23:16:01 EST." <14864.48257.324726.976892@bitdiddle.concentric.net> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> 
Message-ID: <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>

[Jeremy Hylton]
> We've discussed installation of other tools once or twice, but I've
> never decided what exactly to do.  The compiler package, for example,
> comes with a distutils setup.py script that will do the job.  I think
> the best way to move forward is to write setup.py scripts for all the
> other tools that ought to be installed.

That does seem reasonable. It seems the most non-intrusive, since we
get out of the way of proffessional packages, while still giving them
something to start from.

> I don't know how to package up idle cleanly, though.  I need to decide
> what to install and where to install it.  The Tools/idle directory is
> nominally a package, but it doesn't seem like that will help much for
> installing it.  If we want to call the executable script idle.py, it
> will clash with the idle package name.  Any advice here?  It doesn't
> seem like there is any obvious way to "install" IDLE; it looks like it
> is intended to be run out of the source directory.

I don't think we want to call the executable script idle.py. Each platform
will have it's own problems with it: for UNIXish stuff, call it "idle".
For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
should *not* be in the same directory as the idle package. In the UNIX
case it should be in $prefix/bin, and in the windows case, it should be
where python.exe is.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From tim.one at home.com  Tue Nov 14 07:06:01 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 14 Nov 2000 01:06:01 -0500
Subject: [Python-Dev] RE: This Project Has Not Released Any Files
In-Reply-To: <200010191731.TAA01114@loewis.home.cs.tu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECAIAAA.tim.one@home.com>

[Tim, on 19 October]
> From BeOpen.com's POV, so long as they were paying major bills, they
> would rather have download traffic tickle their ad banners than SF's
> ad banners.

[Martin v. Loewis]
> Even though this should have been clear to me all the time, stating it
> explicitly triggers alarms for me.
> ...
> Of course, having ftp mirrors would mean that many downloads do not
> tickle anybody's ad banners - which would probably be in the interest
> of other users as well, just not in the interest of BeOpen. So I'm
> curious how this conflict of interest is resolved...

I hope it's clear now how it got resolved:  at the time I wrote that, the
PythonLabs guys were officially but secretly unemployed.  The issue of
distributing Python was one bone of contention at The End, but not the
mongoose that pecked the python's eyes out.

or-if-you're-not-american-it-wasn't-the-straw-that-tickled-the-
    camel-to-death-ly y'rs  - tim




From moshez at math.huji.ac.il  Tue Nov 14 10:20:01 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 14 Nov 2000 11:20:01 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: <200011132021.MAA09003@slayer.i.sourceforge.net>
Message-ID: <Pine.GSO.4.10.10011141118470.16939-100000@sundial>

On Mon, 13 Nov 2000, Guido van Rossum wrote:

> +     - Killing a thread from another thread.  Or maybe sending a
> +       signal.  Or maybe raising an asynchronous exception.
> + 
> +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470

In general, killing threads in considered a bad idea. It is usually much
better to design the application properly, and being aware that a thread
doing nothing is not that big a deal.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Tue Nov 14 13:06:48 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 07:06:48 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0042.txt,1.43,1.44
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:20:01 +0200."
             <Pine.GSO.4.10.10011141118470.16939-100000@sundial> 
References: <Pine.GSO.4.10.10011141118470.16939-100000@sundial> 
Message-ID: <200011141206.HAA28215@cj20424-a.reston1.va.home.com>

> On Mon, 13 Nov 2000, Guido van Rossum wrote:
> 
> > +     - Killing a thread from another thread.  Or maybe sending a
> > +       signal.  Or maybe raising an asynchronous exception.
> > + 
> > +       http://sourceforge.net/bugs/?func=detailbug&bug_id=121115&group_id=5470
> 
> In general, killing threads in considered a bad idea. It is usually much
> better to design the application properly, and being aware that a thread
> doing nothing is not that big a deal.

Sorry.  I agree 99% (see the referenced thread).  But this keeps
coming up as a feature request, so I figured that there should be
*something* we could offer -- if only some support library calls to
make implementing this common pattern smooth.

For that 1%, I believe that raising an asynchronous exception takes away
*most* of the objections against killing threads -- it gives the
thread control over recovery, since exceptions are to be expected
anyway.

Also note that in a long-running server, losing a thread every once in
a while means leaking a lot of memory!  So (in that case) the
application design cannot afford to just "park" unproductive threads
forever -- it must contain a robust way of recovering their resources.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From skaller at ozemail.com.au  Tue Nov 14 14:25:27 2000
From: skaller at ozemail.com.au (John Max Skaller)
Date: Wed, 15 Nov 2000 00:25:27 +1100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
	            <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>
Message-ID: <3A113D47.ABF1D625@ozemail.com.au>

Guido van Rossum wrote:

> Looks like you are missing the point.  Of course I want to keep the
> class as a local scope *inside the class def*.  

	Ah, ok: seem's I was missing your intent.

> But I need it out of the way when executing methods.  

	I don't think this is the right idea. Consider:

	class X:
		x = 'x'
		def f(self): print x
	a = X()
	a.f() # should print 'x'

and then

	def g(self): print x
	class X:
		f = g
	a = X()
	a.f() # should fail

and then:

	def maker():
		x = 'x'
		def f(self): print x
		return f
	class X:
		f = maker()
	a = X()
	a.f() # should work

The point is that I think there is not really much choice
in the matter: when you lookup an unqualified name in a function,
you must look in the function locals, then in the scope of the
object in which the function is defined, and then in the scope
in which that is defined .. and so ontil you reach the bottom
of the stack (which presumably is 'builtins').

	There's no real issue of keeping the class of the object
of a bound method 'out of the way' when executing the function
of the bound method. The scope of that class should be searched
if, and only if, the function happens to have been defined
inside that class. Note there is no such thing as a method
in python: executing a bound (or unbound) method results
in executing a plain old function.

	You _could_ exclude class scopes from the search stack
associated with a function, and just search locals, then
enclosing functions, then the enclosing module and finally
builtins. But it would be highly inconsistent, and not what
people used to lexical scoping would expect. It will be hard
enough to explain the _consistent_ rule that only local
bindings can be changed by assignments -- unless a global
directive changes that. But that rule IS consistent (and the
global directive much less so!)

	I think there is general agreement that lexically
scoping functions inside functions will not break much code.
Do you have an example of real application code where searching a class
in
which a function is defined will break code?

-- 
John (Max) Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
checkout Vyper http://Vyper.sourceforge.net
download Interscript http://Interscript.sourceforge.net



From guido at python.org  Tue Nov 14 14:29:17 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 08:29:17 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 15:19:11 +0200."
             <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net>  
            <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> 
Message-ID: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>

> [Jeremy Hylton]
> > We've discussed installation of other tools once or twice, but I've
> > never decided what exactly to do.  The compiler package, for example,
> > comes with a distutils setup.py script that will do the job.  I think
> > the best way to move forward is to write setup.py scripts for all the
> > other tools that ought to be installed.
> 
> That does seem reasonable. It seems the most non-intrusive, since we
> get out of the way of proffessional packages, while still giving them
> something to start from.

Agreed.

> > I don't know how to package up idle cleanly, though.  I need to decide
> > what to install and where to install it.  The Tools/idle directory is
> > nominally a package, but it doesn't seem like that will help much for
> > installing it.  If we want to call the executable script idle.py, it
> > will clash with the idle package name.  Any advice here?  It doesn't
> > seem like there is any obvious way to "install" IDLE; it looks like it
> > is intended to be run out of the source directory.
> 
> I don't think we want to call the executable script idle.py. Each platform
> will have it's own problems with it: for UNIXish stuff, call it "idle".
> For windowish stuff, it's better to call it "idle.pyw" anway. Both of those
> should *not* be in the same directory as the idle package. In the UNIX
> case it should be in $prefix/bin, and in the windows case, it should be
> where python.exe is.

For Windows, the IDLE problem is solved: it's installed in directory
<topdir>/Tools/idle/ and there's a link to it in the start menu.  It's
not directly accessible from the command line -- but on Windows,
nobody cares about the command line.

Maybe we can extend this idea to Unix.  Concrete proposal: install
IDLE as <prefix>/lib/python<version>/site-packages/Tools/idle/, and
place a symlink or 2-line bootstrap script in <prefix>/bin/idle.

The introduction of the extra directory level "Tools" implies that
idle is not importable as a package from regular Python.  This is how
I want it -- otherwise I could/would have made IDLE a package in the
Lib/lib-tk directory instead of a directory in Tools.

I'd gladly accept a contribution of a setup.py file that does this for
me.  (I looked at the distutils manual for a bit but decided that I'd
have to read it cover-to-cover to figure out whether what I want is
even possible...)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Tue Nov 14 15:00:39 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 09:00:39 -0500
Subject: [Python-Dev] Re: Nested functions and class scope
In-Reply-To: Your message of "Wed, 15 Nov 2000 00:25:27 +1100."
             <3A113D47.ABF1D625@ozemail.com.au> 
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com> <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com>  
            <3A113D47.ABF1D625@ozemail.com.au> 
Message-ID: <200011141400.JAA28465@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> 
> > Looks like you are missing the point.  Of course I want to keep the
> > class as a local scope *inside the class def*.  

John Max Skaller replied:

> 	Ah, ok: seem's I was missing your intent.
> 
> > But I need it out of the way when executing methods.  
> 
> 	I don't think this is the right idea. Consider:
> 
> 	class X:
> 		x = 'x'
> 		def f(self): print x
> 	a = X()
> 	a.f() # should print 'x'

We understand each other perfectly.  I just disagree with you on what
this example should do!  I believe that in the sake of backwards
compatibility *as well as* not "giving in" to Java and C++ conventions
that I believe are a bad idea for Python, the reference to 'x' from
inside 'f' should not find the class variable 'x' but only a
module-global 'x'.

Backwards compatibility: currently this program prints '1':

    x = 1
    class C:
	x = 2
	def f(self): print x
    C().f()

I believe it should not be broken; I expect that this is, for better
or for worse, a popular idiom, especially when you look at methods
rather than varibles.  I believe it might be common to find this:

    from lower_level_module import some_call
    class C:
	def some_call(self):
	    "OO wrapper around some_call()"
	    return some_call(self.data)

Your rule would break this, because the reference to some_call()
inside C.some_call() would be changed to make a (senseless, recursive)
reference to itself instead of to the globally imported some_call()
function.

Java/C++ expectations: in Java/C++, the rules are different.  Inside a
method, not just names defined in the current class are in scope, but
also all names defined in its base classes.  Look at this:

    # Module a.py
    class B:
        def f(self): pass

    # Module b.py
    import a
    class C(a.B):
	def g(self): pass
        def h(self):
	    g(self)
            f(self)

Under your rules, the reference to g() in h() would work, but the
reference to f() would fail.  In similarly structured Java or C++
code, h() could reference both g() and f() without a naming
disambiguation.

Python requires that you specify the class when you reference a class
attribute (method or class variable).  I don't want to lose this rule,
it is important (to me) for readability.

[...more examples elided...]
> The point is that I think there is not really much choice
> in the matter: when you lookup an unqualified name in a function,
> you must look in the function locals, then in the scope of the
> object in which the function is defined, and then in the scope
> in which that is defined .. and so ontil you reach the bottom
> of the stack (which presumably is 'builtins').

I claim different, and above I have explained why I think this is
necessary.  I'd agree if you said "the scope of the *function* in
which it is defined."

> 	There's no real issue of keeping the class of the object
> of a bound method 'out of the way' when executing the function
> of the bound method. The scope of that class should be searched
> if, and only if, the function happens to have been defined
> inside that class. Note there is no such thing as a method
> in python: executing a bound (or unbound) method results
> in executing a plain old function.

If the code generator keeps track of the distinction between class and
function scopes, my rules can be implemented easily without
introducing a distinction between functions and methods.

> 	You _could_ exclude class scopes from the search stack
> associated with a function, and just search locals, then
> enclosing functions, then the enclosing module and finally
> builtins. But it would be highly inconsistent, and not what
> people used to lexical scoping would expect.


Yet, this is what I want.  It's impossible to design things so that
people coming from very different backgrounds will never encounter a
surprise.  Since Python is primarily an OO language, and there's ten
years of established Python rules, I prefer to give the OO people and
existing Python users an easier time (note that the OO people at least
can count on *consistently* not seeing the class scope in the
"methods") and tell the lexical-scoping people about the exceptions.

> It will be hard
> enough to explain the _consistent_ rule that only local
> bindings can be changed by assignments -- unless a global
> directive changes that. But that rule IS consistent (and the
> global directive much less so!)

> 	I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't found one yet, but I've seen plenty of examples where a
class defines a method with the same name as a global function
(including lots in the standard library -- it really is a common
idiom).  I think this comes too close to risk breakage.

But even if we find that this doesn't break anything, I still think
it's unwise to allow unqualified access to class attributes from
within methods.  People would discover that in this example:

    class C:
	def m1(self):
	    pass
	def m2a(self):
	    m1(self)
	def m2b(self):
	    self.m1()

m2a() and m2b() would be equivalent, i.e. m1(self) and self.m1() are
equivalent.  That's evil, because it opens up a meaningless choice
between alternative (and the m1(self) notation is inferior, because it
doesn't look in base classes).

But I'm beginning to repeat myself, so I think I've said all that can
be said about this.  Sorry the scope rules in Vyper are wrong.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at pythonware.com  Tue Nov 14 15:09:37 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2000 15:09:37 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>              <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <013701c04e44$871496b0$0900a8c0@SPIFF>

John Max Skaller wrote:
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

I haven't been following this thread closely, but are we talking
about a scoping model where a class can override any built-in
function or method by defining a method (or a class attribute)
with the same name?

like in urllib.py, for example:

class URLopener:
    ...
    def open(self, fullurl, data=None):
        fullurl = unwrap(fullurl)
        if self.tempcache and self.tempcache.has_key(fullurl):
            filename, headers = self.tempcache[fullurl]
            fp = open(filename, 'rb')
            return addinfourl(fp, headers, fullurl)
    ...

this won't fly.

</F>




From mal at lemburg.com  Tue Nov 14 15:23:50 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 14 Nov 2000 15:23:50 +0100
Subject: [Python-Dev] Re: Nested functions and class scope
References: <200011061702.MAA07735@cj20424-a.reston1.va.home.com> <3A09462A.63881D07@ozemail.com.au> <200011081311.IAA21962@cj20424-a.reston1.va.home.com> <3A096AB0.D754E2B6@ozemail.com.au> <200011081507.KAA22524@cj20424-a.reston1.va.home.com> <3A0D6782.8747BA94@ozemail.com.au> <200011111827.NAA12288@cj20424-a.reston1.va.home.com>  
		            <3A0F26C3.7CD823BF@ozemail.com.au> <200011131254.HAA22379@cj20424-a.reston1.va.home.com> <3A113D47.ABF1D625@ozemail.com.au>
Message-ID: <3A114AF6.EC584A17@lemburg.com>

John Max Skaller wrote:
> 
>         I think there is general agreement that lexically
> scoping functions inside functions will not break much code.
> Do you have an example of real application code where searching a class
> in which a function is defined will break code?

class Object:

   type = 'typename'

   def __init__(self, obj):

       if type(obj) == types.IntType:
           ...

Basically all builtins which are in common use like open(), id()
type(), etc. would no longer be accessible.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From klm at digicool.com  Tue Nov 14 16:53:47 2000
From: klm at digicool.com (Ken Manheimer)
Date: Tue, 14 Nov 2000 10:53:47 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <14849.38941.59576.682495@bitdiddle.concentric.net>
Message-ID: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>

On Thu, 2 Nov 2000, Jeremy Hylton wrote:

> There are few languages that use dynamic scoping for normal name
> resolution.  Many early Lisp implementations did, but I think all the
> modern ones use lexical scoping instead.  It is hard to write modular
> code using dynamic scope, because the behavior of a function with free
> variables can not be determined by the module that defines it.  Not
> saying it isn't useful, just that it makes it much harder to reason
> about how a particular modular or function works in isolation from the
> rest of the system.

There's a seminal steele and sussman paper, _The Art of the Interpreter,
or the Modularity Complex (Parts Zero, One, and Two)_, which, among other
things, introduces closures and describes why dynamic scoping is evil.  
(The paper sketches out an elementary, lexically scoped lisp interpreter
which i think is the precursor to scheme.)  It's been over 20 years since
i read it, but i recall something known as the funarg problem, where
dynamic scoping + recursion (maybe, + functions as first class objects)
inherently breaks certain parameter passing arrangements.

If you're interested, see:

  ftp://publications.ai.mit.edu/ai-publications/0-499/AIM-453.ps

(I'm working at home today, so can't even glance at it to refresh my
memory, and oughtn't at the moment, anyway.)

In any case, dynamic scoping breaks locality of reference for just about
any definition of "locality".

With lexical scoping, you can trace through the text of the program to
track down implicit references.  Even with acquisition, the containment
connections are made explicitly (though it seems magic with web
programming, because the zope publisher applies .__of__() for you), and
can be nailed down by introspection on the objects (.aq_base, .aq_parent,
.aq_inner...).

With dynamic scoping, variable references go up the stack to the context
of *any* intervening call frame, with no way for the programmer to know
where it's being resolved.  In practice, even without the funarg breakage,
it doesn't scale up - it's too promiscuous a state-communication
mechanism.  You can work around it - a lot of emacs lisp code does so -
but it's a terrible burden, not a benefit.

Ken




From esr at golux.thyrsus.com  Tue Nov 14 17:59:24 2000
From: esr at golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 11:59:24 -0500
Subject: [Python-Dev] Making popen2 actually work
Message-ID: <200011141659.LAA01557@golux.thyrsus.com>

OK, so Guido told me at one point that 2.0 was supposed to have
included changes to the way buffering of I/O to subprocesses is
handled.  I've installed 2.0, and I need to know how to make the
following program work:

import os, sys

(child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
print "* Child process spawned"

while 1:
    line = raw_input(">");
    child_stdin.write(line); child_stdin.flush()
    print "* Line written to child process"
    transformed = child_stdout.readline(); child_stdout.flush()
    print "* Line read back from child process."
    sys.stdout.write(transformed)

If anybody can tell me what the right magic is here, I'll write up 
a patch for the library docs explaining it and including the corrected
example.  This has to be a FAQ.
--
						>>esr>>



From esr at golux.thyrsus.com  Tue Nov 14 18:08:28 2000
From: esr at golux.thyrsus.com (Eric S. Raymond)
Date: Tue, 14 Nov 2000 12:08:28 -0500
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
Message-ID: <200011141708.MAA01643@golux.thyrsus.com>

I installed from the Beopen-Python-2.0 RPM at Source Forge.  Now:

golux:~/WWW/fetchmail$ fetchmailconf
Traceback (most recent call last):
  File "./fetchmailconf", line 9, in ?
    from Tkinter import *
  File "/usr/local/lib/python2.0/lib-tk/Tkinter.py", line 35, in ?
    import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter

This is a serious problem.  Is there a non-broken RPM out there?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The right of self-defense is the first law of nature: in most
governments it has been the study of rulers to confine this right
within the narrowest limits possible.  Wherever standing armies
are kept up, and when the right of the people to keep and bear
arms is, under any color or pretext whatsoever, prohibited,
liberty, if not already annihilated, is on the brink of
destruction." 
	-- Henry St. George Tucker (in Blackstone's Commentaries)



From jeremy at alum.mit.edu  Tue Nov 14 18:38:09 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 12:38:09 -0500 (EST)
Subject: [Python-Dev] Installing 2.0 seems to have broken Tkinter
In-Reply-To: <200011141708.MAA01643@golux.thyrsus.com>
References: <200011141708.MAA01643@golux.thyrsus.com>
Message-ID: <14865.30849.276890.227441@bitdiddle.concentric.net>

The RPM isn't "broken."  It just doesn't include _tkinter.  You'll
need to get the separate Tkinter RPM for that.

The separation is intentional, because different Linux distriubtions
have different Tk versions installed.  Rather than have the main RPM
depend on a single Tk version, they are packaged separately.  You can
also use the distutils package to install _tkinter.so.  It will work
with any version of Tk, while the RPMs will only work with the version
of Tk installed on my machine.

Jeremy



From guido at python.org  Wed Nov 15 04:48:14 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 22:48:14 -0500
Subject: [Python-Dev] Making popen2 actually work
In-Reply-To: Your message of "Tue, 14 Nov 2000 11:59:24 EST."
             <200011141659.LAA01557@golux.thyrsus.com> 
References: <200011141659.LAA01557@golux.thyrsus.com> 
Message-ID: <200011150348.WAA05535@cj20424-a.reston1.va.home.com>

> OK, so Guido told me at one point that 2.0 was supposed to have
> included changes to the way buffering of I/O to subprocesses is
> handled.  I've installed 2.0, and I need to know how to make the
> following program work:
> 
> import os, sys
> 
> (child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
> print "* Child process spawned"
> 
> while 1:
>     line = raw_input(">");
>     child_stdin.write(line); child_stdin.flush()
>     print "* Line written to child process"
>     transformed = child_stdout.readline(); child_stdout.flush()
>     print "* Line read back from child process."
>     sys.stdout.write(transformed)
> 
> If anybody can tell me what the right magic is here, I'll write up 
> a patch for the library docs explaining it and including the corrected
> example.  This has to be a FAQ.

First, raw_input() strips the newline, so you should change the second
line of the loop body to

     child_stdin.write(line+'\n'); child_stdin.flush()

Second, with "cat" instead of "/usr/bin/rev", this works fine for me,
so your logic is fine.

The trick seems to be to get the subprocess not to buffer its output
(or input).  There's nothing that Python can do for you here!

Unfortunately, most programs use stdio for all their console I/O, and
stdio automatically buffers its output when connected to a pipe... :-(

If you absolutely, positively have to do this with a program that
won't unbuffer its output, your only resort would be to use pseudo
ttys -- but that's quite a different story... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jeremy at alum.mit.edu  Wed Nov 15 03:13:36 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 14 Nov 2000 21:13:36 -0500 (EST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011141329.IAA28404@cj20424-a.reston1.va.home.com>
References: <14864.48257.324726.976892@bitdiddle.concentric.net>
	<E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il>
	<200011141329.IAA28404@cj20424-a.reston1.va.home.com>
Message-ID: <14865.61776.50840.609922@bitdiddle.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

  >> > I don't know how to package up idle cleanly, though.  I need to
  >> > decide what to install and where to install it.  The Tools/idle
  >> > directory is nominally a package, but it doesn't seem like that
  >> > will help much for installing it.  If we want to call the
  >> > executable script idle.py, it will clash with the idle package
  >> > name.  Any advice here?  It doesn't seem like there is any
  >> > obvious way to "install" IDLE; it looks like it is intended to
  >> > be run out of the source directory.
  >>

  GvR> For Windows, the IDLE problem is solved: it's installed in
  GvR> directory <topdir>/Tools/idle/ and there's a link to it in the
  GvR> start menu.  It's not directly accessible from the command line
  GvR> -- but on Windows, nobody cares about the command line.

  GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
  GvR> install IDLE as
  GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
  GvR> place a symlink or 2-line bootstrap script in
  GvR> <prefix>/bin/idle.

How about
<prefix>/lib/python<version>/Tools/idle
instead.  I suggest the change to support an until-now-implicit policy
that anything in site-packages actually be a package.  If things in
site-packages aren't supposed to be packages, it ought to be called
something like site-tools <0.2 wink>.

  GvR> The introduction of the extra directory level "Tools" implies
  GvR> that idle is not importable as a package from regular Python.
  GvR> This is how I want it -- otherwise I could/would have made IDLE
  GvR> a package in the Lib/lib-tk directory instead of a directory in
  GvR> Tools.

I think the extra Tools directory still implies this, even if it isn't
in site-packages.

  GvR> I'd gladly accept a contribution of a setup.py file that does
  GvR> this for me.  (I looked at the distutils manual for a bit but
  GvR> decided that I'd have to read it cover-to-cover to figure out
  GvR> whether what I want is even possible...)

Is there a reason why IDLE has this peculiar installation process?
Might there be a better way to do it now that we have distutils.  If
there is, we could put it in PEP 42.

Jeremy



From guido at python.org  Wed Nov 15 05:10:06 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2000 23:10:06 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Tue, 14 Nov 2000 21:13:36 EST."
             <14865.61776.50840.609922@bitdiddle.concentric.net> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com>  
            <14865.61776.50840.609922@bitdiddle.concentric.net> 
Message-ID: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>

>   GvR> Maybe we can extend this idea to Unix.  Concrete proposal:
>   GvR> install IDLE as
>   GvR> <prefix>/lib/python<version>/site-packages/Tools/idle/, and
>   GvR> place a symlink or 2-line bootstrap script in
>   GvR> <prefix>/bin/idle.
> 
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Fine.

>   GvR> The introduction of the extra directory level "Tools" implies
>   GvR> that idle is not importable as a package from regular Python.
>   GvR> This is how I want it -- otherwise I could/would have made IDLE
>   GvR> a package in the Lib/lib-tk directory instead of a directory in
>   GvR> Tools.
> 
> I think the extra Tools directory still implies this, even if it isn't
> in site-packages.

Sure.

>   GvR> I'd gladly accept a contribution of a setup.py file that does
>   GvR> this for me.  (I looked at the distutils manual for a bit but
>   GvR> decided that I'd have to read it cover-to-cover to figure out
>   GvR> whether what I want is even possible...)
> 
> Is there a reason why IDLE has this peculiar installation process?
> Might there be a better way to do it now that we have distutils.  If
> there is, we could put it in PEP 42.

IDLE doesn't have a peculiar installation process, but it may not be
what distutils is currently good at.  IDLE is an application that
needs its files installed somewhere but not on the module search path
for other apps.  I don't know how to do that!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From klm at digicool.com  Wed Nov 15 06:05:10 2000
From: klm at digicool.com (Ken Manheimer)
Date: Wed, 15 Nov 2000 00:05:10 -0500 (EST)
Subject: [Python-Dev] Re: Dynamic nested scopes
In-Reply-To: <Pine.LNX.4.21.0011141010410.8955-100000@serenade.digicool.com>
Message-ID: <Pine.LNX.4.21.0011142355310.8955-100000@serenade.digicool.com>

On Tue, 14 Nov 2000, Ken Manheimer wrote:

> In any case, dynamic scoping breaks locality of reference for just about
> any definition of "locality".

(Huh - i realized that "locality of reference" isn't the right phrase, not
sure what is.  I was trying to refer to something along the lines of
modularity - basically, the virtue of lexical scoping is that you can,
eventually, trace the connections by reading the program text.  Dynamic
scoping is not so well behaved - any routine that calls your routine may
contain a variable whose value you're expecting to get from elsewhere.  
It's too promiscuous...)

Ken




From moshez at math.huji.ac.il  Wed Nov 15 08:11:29 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 15 Nov 2000 09:11:29 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <200011150410.XAA05616@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011150904510.16783-100000@sundial>

On Tue, 14 Nov 2000, Guido van Rossum wrote:

> > How about
> > <prefix>/lib/python<version>/Tools/idle
> > instead.  I suggest the change to support an until-now-implicit policy
> > that anything in site-packages actually be a package.  If things in
> > site-packages aren't supposed to be packages, it ought to be called
> > something like site-tools <0.2 wink>.
> 
> Fine.

Sorry, just noticed a problem with that:

Say you put idle in .../Tools/idle
And compiler in .../Tools/compiler

Then the fact that some script does

sys.path.insert(0, ".../Tools") means that it would get all the tools
as packages to import, instead of only those it needs (say, for
idle-callers, idle). That means a lot of backwards compat. issues.
Why not have each tool be in things like ..../idle-support/idle/
And .../compiler/compiler-support/compiler, etc.?

Actually, once we do that, why bring the Python path into that at all?
"idle" is a program. The fact it is written in Python is incidental --
it might as well have been written in C, for all most users care. The
FHS policy would seem to indicate that the support files would need
to be in things like /usr/lib and /usr/local/lib
Say, the default installation would put it in /usr/local/lib/idle-support/
and would add a line to sys.path.insert(), and put "idle" (the script)
under autoconf control too.

sorry-for-challenging-the-consensus-again-ly y'rs, Z.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From pf at artcom-gmbh.de  Wed Nov 15 08:32:59 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Wed, 15 Nov 2000 08:32:59 +0100 (MET)
Subject: Using telnetlib instead of 'popen2' (was: Re: [Python-Dev] Making popen2 actually work)
In-Reply-To: <200011150348.WAA05535@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 14, 2000 10:48:14 pm"
Message-ID: <m13vx3z-000Dm8C@artcom0.artcom-gmbh.de>

Hi,

Guido van Rossum wrote:
> The trick seems to be to get the subprocess not to buffer its output
> (or input).  There's nothing that Python can do for you here!
> 
> Unfortunately, most programs use stdio for all their console I/O, and
> stdio automatically buffers its output when connected to a pipe... :-(
> 
> If you absolutely, positively have to do this with a program that
> won't unbuffer its output, your only resort would be to use pseudo
> ttys -- but that's quite a different story... :-(

Yes.  But it should be noted here, that sometimes you can avoid the rather
complicated and platform depending handling of pseudo ttys, if you use
the 'telnetlib' module from the Python standard library.  The fine lib
documentation contains a good example ready for cut'n'paste.  Disadvantage:
This only works, if the telnet service is not disabled due to security
reasons.

Regards, Peter



From gward at mems-exchange.org  Wed Nov 15 19:11:30 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Wed, 15 Nov 2000 13:11:30 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <14865.61776.50840.609922@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 14, 2000 at 09:13:36PM -0500
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>
Message-ID: <20001115131130.A22029@ludwig.cnri.reston.va.us>

On 14 November 2000, Jeremy Hylton said:
> How about
> <prefix>/lib/python<version>/Tools/idle
> instead.  I suggest the change to support an until-now-implicit policy
> that anything in site-packages actually be a package.  If things in
> site-packages aren't supposed to be packages, it ought to be called
> something like site-tools <0.2 wink>.

Actually, lots of non-packagized modules get installed to site-packages
-- they just wind up right in site-packages, rather than in a
subdirectory of it.

I've always thought this directory was misnamed -- should have been
"site-lib" or something.  Oh well.

For the record, the Distutils don't currently support the "application"
model of installation that Guido wants.  It's on the (unofficial,
notional, blue-sky) to-do list.

        Greg



From nas at arctrix.com  Wed Nov 15 18:35:59 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 15 Nov 2000 09:35:59 -0800
Subject: [Python-Dev] Work in progress - coercion/cmp overhaul
Message-ID: <20001115093559.A19286@glacier.fnational.com>

[The python-dev list server seems to have gotten hungry and eaten my
original message.  I'm trying again.]


I making this available now in the hope that it will generate
some dicussion:

    http://arctrix.com/nas/python/coerce-2.diff

The patch is based on Marc-Andre Lemburg's coercion patch for
1.5.1 downloaded from the starship:

    http://starship.python.net/~lemburg/CoercionProposal.html

All credit goes to him.  The bugs are probably introduced by me.
What the patch does:

  - Introduces a new type flag "NEWSYTLENUMBER".  If this flag is
    set then the tp_as_number methods are treated differently.
    Coercion is not done before passing arguments to the method.
    New style methods do their own coercion or return the
    NotImplemented singleton.  See Marc's page for more details.

  - Updates the PyInt, PyFloat and PyInstance types to use new
    style methods.

  - Changes PyNumber_Multiply to only swap arguments if second
    argument implements sq_repeat.

Why:

  - It gives extension types more control over how they implement
    number methods.

  - Removes the special handling of PyInstance in abstract.c.

  - Slightly speeds up the interpreter.

Todo:

  - Clean up cmp() operator and implement rich comparsions for
    the new nb_cmp slot.

  - Decide on handling of nb_coerce slot (eg. should it be called
    on new style numbers).

  - Convert PyComplex and PyLong to use new style.

Comments?

  Neil




From moshez at math.huji.ac.il  Thu Nov 16 08:06:14 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 16 Nov 2000 09:06:14 +0200 (IST)
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: <20001115131130.A22029@ludwig.cnri.reston.va.us>
Message-ID: <Pine.GSO.4.10.10011160905260.24460-100000@sundial>

On Wed, 15 Nov 2000, Greg Ward wrote:

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.
> 
> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

On a similar note, there's a discussion going on about this in 
debian-python at lists.debian.org (there are archives on the Debian
site)

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From guido at python.org  Thu Nov 16 17:26:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 11:26:16 -0500
Subject: [Python-Dev] installing IDLE & other Tools
In-Reply-To: Your message of "Wed, 15 Nov 2000 13:11:30 EST."
             <20001115131130.A22029@ludwig.cnri.reston.va.us> 
References: <14864.48257.324726.976892@bitdiddle.concentric.net> <E13vfzU-0003nQ-00@darjeeling.zadka.site.co.il> <200011141329.IAA28404@cj20424-a.reston1.va.home.com> <14865.61776.50840.609922@bitdiddle.concentric.net>  
            <20001115131130.A22029@ludwig.cnri.reston.va.us> 
Message-ID: <200011161626.LAA10243@cj20424-a.reston1.va.home.com>

> Actually, lots of non-packagized modules get installed to site-packages
> -- they just wind up right in site-packages, rather than in a
> subdirectory of it.
> 
> I've always thought this directory was misnamed -- should have been
> "site-lib" or something.  Oh well.

The name doesn't matter that much.  I think this is a fine use of
site-packages.

> For the record, the Distutils don't currently support the "application"
> model of installation that Guido wants.  It's on the (unofficial,
> notional, blue-sky) to-do list.

At least this admission saves me wasting time trying to force
distutils into submssion. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From akuchlin at mems-exchange.org  Thu Nov 16 20:44:59 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 14:44:59 -0500
Subject: [Python-Dev] curses demos
Message-ID: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us>

I'd like to add a Demo/curses directory to the CVS tree.  If anyone
has reasonably small programs that use the curses module, please let
me know.

Right now I have two programs: a Life game that I wrote, and Alexei
Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
manager.  I also want to write scripts to demonstrate reading the
mouse, and writing a threaded curses program.

--amk



From guido at python.org  Thu Nov 16 21:03:47 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 15:03:47 -0500
Subject: [Python-Dev] curses demos
In-Reply-To: Your message of "Thu, 16 Nov 2000 14:44:59 EST."
             <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us> 
References: <E13wUxv-0006Ot-00@kronos.cnri.reston.va.us> 
Message-ID: <200011162003.PAA14322@cj20424-a.reston1.va.home.com>

> I'd like to add a Demo/curses directory to the CVS tree.  If anyone
> has reasonably small programs that use the curses module, please let
> me know.
> 
> Right now I have two programs: a Life game that I wrote, and Alexei
> Gilchrist's cfm (http://cfm.sourceforge.net/), which is a little file
> manager.  I also want to write scripts to demonstrate reading the
> mouse, and writing a threaded curses program.

Cool.  Go for it!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Fri Nov 17 01:42:56 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 19:42:56 -0500
Subject: [Python-Dev] coroutines and microthreads
Message-ID: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>

(I would cc this to the Stackless list as well, except that starship
mail is still down it seems.  I *am* posting to the python-coro list,
which seems to be appropriate here.  Will Ware, are you listening?)

Coroutines!
-----------

Inspired by Gordon McMillan's Stackless tutorial
(http://www.mcmillan-inc.com/stackless.html), I've been thinking more
about a coroutine API for Python.

I proposed one before (see the thread "uthread strawman" in
python-dev).  I've been thinking a bit more and am now revising my
proposal a bit.

I'll first give an example.  Tim Peters gave a standard example to
return the fringe of a list (checked in as Demo/threads/fcmp.py).

Using my proposed API this example could be rewritten a bit
cleaner, as follows:

    from coro import coroutine, suspend, EarlyExit  # Provisional module name

    def fringe(L):
	for x in L:
	    if type(x) is type(L):
		fringe(x)
	    else:
		suspend(x)

    def printinorder(L):
	c = coroutine(f, L)
	try:
	    while 1:
		print c(),
	except EarlyExit:
	    pass
	print

    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
    printinorder(x) # prints "0 1 2 3 4 5 6"

This is about as simple as it gets.  It supports both generators (like
the example here) and coroutines (like Tim's Demo/threads/squasher.py
example).

Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
propose a function current() which returns the current coroutine
object.  There should also be a way to kill a coroutine (or at least
to send an exception).  When a coroutine falls through at its end,
*some* other coroutine needs to be resumed.  

I believe this can be implemented with a much simplified stackless
approach, that doesn't cater towards continuations (but still borrows
a lot of wisdom from Christian's Stackless).  It can also be
implemented using threads, which should give some hope for getting the
same API supported in JPython, making it more attractive.  I am hoping
to create an implementation on top of Stackless, just to experiment
with the idiom.

Microthreads?
-------------

I'm much less clear about microthreads (uthreads).  Last time, I
thought this was a fancy name for coroutines.  It isn't!  Microthreads
are "almost real" threads, with round-robin scheduling.  What makes
them attractive to some is the fact that they don't use operating
system resources: there's no OS-level stack or kernel process table
entry associated with them.  This also accounts for their biggest
weakness: when a microthread is suspended for I/O, *all* microthreads
are suspended.  In limited contexts, such as a network server, this
can be solved by an approach similar to that in Gordon's
SelectDispatcher.  (It would be a titanic effort to do this for every
potentially blocking I/O operation, and it probably wouldn't work well
with C extensions.)

I'm not sure what applications require the round-robin scheduling
property of uthreads -- certainly Gordon's application would seem to
be doing just fine without it (I'm not sure if it is being scheduled
round-robin or not).

Proper round-robin scheduling for uthreads requires explicit switching
code in the Python interpreter.  Stackless provides this, at the same
place where in regular Python the global interpreter lock is released
and re-acquired to give other threads a chance to run.  Is this
needed?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gmcm at hypernet.com  Fri Nov 17 03:29:52 2000
From: gmcm at hypernet.com (Gordon McMillan)
Date: Thu, 16 Nov 2000 21:29:52 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <3A1451D0.12627.17914E75@localhost>

[Guido]
> I'll first give an example.  Tim Peters gave a standard example to
> return the fringe of a list (checked in as Demo/threads/fcmp.py).
> 
> Using my proposed API this example could be rewritten a bit
> cleaner, as follows:
> 
>     from coro import coroutine, suspend, EarlyExit  # Provisional module name
> 
>     def fringe(L):
> 	for x in L:
> 	    if type(x) is type(L):
> 		fringe(x)
> 	    else:
> 		suspend(x)
> 
>     def printinorder(L):
> 	c = coroutine(f, L)
> 	try:
> 	    while 1:
> 		print c(),
> 	except EarlyExit:
> 	    pass
> 	print
> 
>     x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>     printinorder(x) # prints "0 1 2 3 4 5 6"
> 
> This is about as simple as it gets.  It supports both generators (like
> the example here) and coroutines (like Tim's Demo/threads/squasher.py
> example).

This looks very nice and clean. It looks sufficient for the type 
of thing I'm doing (and planning to do), but is it really a full 
coroutine API? That is, doesn't the fact that you always 
suspend to the guy who just activated you make this a 
generator API? (OTOH, if I want A and B to talk to each other 
as "coroutines", is it sufficient to make them both "generators" 
and then glue them together with another routine that just 
swaps results?)
 
> Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> propose a function current() which returns the current coroutine
> object.  There should also be a way to kill a coroutine (or at least
> to send an exception).  When a coroutine falls through at its end,
> *some* other coroutine needs to be resumed.  

Random thoughts:
 - I've found it handy that Christian's stuff lets you grab a 
coroutine as well as return one (that is, either side can 
instigate it). Not sure if that's necessary.
 - What do you mean by "kill a coroutine"? You can't interrupt 
one, so isn't it sufficient that when it goes out of scope it gets 
GC'd somehow?
 - It appears from your example that falling off the end 
automatically raises an EarlyExit. I think I see more 
arguments for that than against it :-).
 
> Microthreads?
> -------------
> 
> I'm much less clear about microthreads (uthreads).  Last time, I
> thought this was a fancy name for coroutines.  It isn't!  Microthreads
> are "almost real" threads, with round-robin scheduling.  What makes
> them attractive to some is the fact that they don't use operating
> system resources: there's no OS-level stack or kernel process table
> entry associated with them.  This also accounts for their biggest
> weakness: when a microthread is suspended for I/O, *all* microthreads
> are suspended.  In limited contexts, such as a network server, this
> can be solved by an approach similar to that in Gordon's
> SelectDispatcher.  (It would be a titanic effort to do this for every
> potentially blocking I/O operation, and it probably wouldn't work well
> with C extensions.)

Using the raw Win32 API, I think you could come pretty close. 
I've been wondering if it's possible to do something that would 
get Cameron to quit raving about Tcl's event loop ;-).
 
> I'm not sure what applications require the round-robin scheduling
> property of uthreads -- certainly Gordon's application would seem to
> be doing just fine without it (I'm not sure if it is being scheduled
> round-robin or not).

Each coroutine (or whatever they are) runs until it calls one of 
the SelectDispatcher methods that suspends it.  The socket 
methods suspend it until select says the socket is ready; 
yield suspends it till the next time round the select loop 
(which has a timeout). So piggish routines are encouraged to 
yield once in a while.
 
(While I'm doing things the other way 'round, I believe I could 
use your API without changing SelectDispatcher's API at all.)

> Proper round-robin scheduling for uthreads requires explicit switching
> code in the Python interpreter.  Stackless provides this, at the same
> place where in regular Python the global interpreter lock is released
> and re-acquired to give other threads a chance to run.  Is this
> needed?

While I'm not really a uthread user, I think they would give you 
an unqualified "yes". The advantage to explicitly yeilding is 
that (with proper thought) you don't need nasty things like 
locks; the disadvantage (as demonstrated by a particular OS 
with a rabidly fanatical following) is that one jerk can ruin it for 
everybody.

- Gordon



From guido at python.org  Fri Nov 17 04:15:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 22:15:06 -0500
Subject: [Python-Dev] coroutines and microthreads
In-Reply-To: Your message of "Thu, 16 Nov 2000 21:29:52 EST."
             <3A1451D0.12627.17914E75@localhost> 
References: <3A1451D0.12627.17914E75@localhost> 
Message-ID: <200011170315.WAA21528@cj20424-a.reston1.va.home.com>

> > This is about as simple as it gets.  It supports both generators (like
> > the example here) and coroutines (like Tim's Demo/threads/squasher.py
> > example).
> 
> This looks very nice and clean. It looks sufficient for the type 
> of thing I'm doing (and planning to do), but is it really a full 
> coroutine API? That is, doesn't the fact that you always 
> suspend to the guy who just activated you make this a 
> generator API? (OTOH, if I want A and B to talk to each other 
> as "coroutines", is it sufficient to make them both "generators" 
> and then glue them together with another routine that just 
> swaps results?)

Yes, it is a full coroutine API -- you don't have to use suspend() at
all.  Appended is my version of squasher.py, an almost mechanical
translation from Tim's version (co.tran(cofoobar, arg) => cofoobar(arg)).
That's built out of a bunch of real coroutines with only explicit
transfers!

> > Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> > propose a function current() which returns the current coroutine
> > object.  There should also be a way to kill a coroutine (or at least
> > to send an exception).  When a coroutine falls through at its end,
> > *some* other coroutine needs to be resumed.  
> 
> Random thoughts:
>  - I've found it handy that Christian's stuff lets you grab a 
> coroutine as well as return one (that is, either side can 
> instigate it). Not sure if that's necessary.

Sorry, not sure what this means.

>  - What do you mean by "kill a coroutine"? You can't interrupt 
> one, so isn't it sufficient that when it goes out of scope it gets 
> GC'd somehow?

That's one solution.  You can't interrupt threads either, and most
people are happy with that (although there's a request for thread
interrupts in PEP 42).

>  - It appears from your example that falling off the end 
> automatically raises an EarlyExit. I think I see more 
> arguments for that than against it :-).

Yes, this follows Tim's lead.  What would be an argument against it?

> > Microthreads?
> > -------------
> > 
> > I'm much less clear about microthreads (uthreads).  Last time, I
> > thought this was a fancy name for coroutines.  It isn't!  Microthreads
> > are "almost real" threads, with round-robin scheduling.  What makes
> > them attractive to some is the fact that they don't use operating
> > system resources: there's no OS-level stack or kernel process table
> > entry associated with them.  This also accounts for their biggest
> > weakness: when a microthread is suspended for I/O, *all* microthreads
> > are suspended.  In limited contexts, such as a network server, this
> > can be solved by an approach similar to that in Gordon's
> > SelectDispatcher.  (It would be a titanic effort to do this for every
> > potentially blocking I/O operation, and it probably wouldn't work well
> > with C extensions.)
> 
> Using the raw Win32 API, I think you could come pretty close. 

You can do it on Unix too, by trapping open() to always set files in
nonblocking mode.  It's just a lot of work because there are so many
calls to trap (cf. your dispatcher, which just traps read, write,
accept, connect).  Long ago, there was a user-mode thread package on
Solaris that did this; it's still immortalized in Python/thread_lwp.h
although I doubt that it still works.  GNU Pth uses the same approach
for user-mode threads.  Hm...  This makes me wonder...  Maybe instead
of uthreads we could promote Python linked with Pth?

> I've been wondering if it's possible to do something that would 
> get Cameron to quit raving about Tcl's event loop ;-).

Cameron is now lobbying for a separation between Tcl and Tk.  I guess
that will mean that the event loop will have to be moved back to Tk,
where it started many years ago. :-)

> > I'm not sure what applications require the round-robin scheduling
> > property of uthreads -- certainly Gordon's application would seem to
> > be doing just fine without it (I'm not sure if it is being scheduled
> > round-robin or not).
> 
> Each coroutine (or whatever they are) runs until it calls one of 
> the SelectDispatcher methods that suspends it.  The socket 
> methods suspend it until select says the socket is ready; 
> yield suspends it till the next time round the select loop 
> (which has a timeout). So piggish routines are encouraged to 
> yield once in a while.

That's what I figured.

> (While I'm doing things the other way 'round, I believe I could 
> use your API without changing SelectDispatcher's API at all.)

I think so too.  Maybe that'll be my next exercise.  I've already done
8 queens with coroutines (slower than recursion, alas -- should be
faster when coroutines are implemented in C though because you save on
number of function calls).  I've also made tho working implementations of
my proposed coro API: one using threads, one using Stackless and the
continuation module.  (Found a few quirks in the examples in your
tutorial.  Interested in details?)

> > Proper round-robin scheduling for uthreads requires explicit switching
> > code in the Python interpreter.  Stackless provides this, at the same
> > place where in regular Python the global interpreter lock is released
> > and re-acquired to give other threads a chance to run.  Is this
> > needed?
> 
> While I'm not really a uthread user, I think they would give you 
> an unqualified "yes". The advantage to explicitly yeilding is 
> that (with proper thought) you don't need nasty things like 
> locks; the disadvantage (as demonstrated by a particular OS 
> with a rabidly fanatical following) is that one jerk can ruin it for 
> everybody.

Let me guess -- you don't like the Mac much, do you? :-)

And yes, I noticed that the first thing the uthread API does is define
a bunch of synchronization primitives. :-(  I think it wouldn't be
hard to implement the round-robin scheduling, but it's yet more added
complexity...

BTW, how's PEP 219 coming along? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

# Coroutine example:  general coroutine transfers
# (After Tim Peters)
#
# The program is a variation of a Simula 67 program due to Dahl & Hoare,
# (Dahl/Dijkstra/Hoare, Structured Programming; Academic Press, 1972)
# who in turn credit the original example to Conway.
#
# We have a number of input lines, terminated by a 0 byte.  The problem
# is to squash them together into output lines containing 72 characters
# each.  A semicolon must be added between input lines.  Runs of blanks
# and tabs in input lines must be squashed into single blanks.
# Occurrences of "**" in input lines must be replaced by "^".
#
# Here's a test case:

test = """\
   d    =   sqrt(b**2  -  4*a*c)
twoa    =   2*a
   L    =   -b/twoa
   R    =   d/twoa
  A1    =   L + R
  A2    =   L - R\0
"""

# The program should print:

# d = sqrt(b^2 - 4*a*c);twoa = 2*a; L = -b/twoa; R = d/twoa; A1 = L + R;
#A2 = L - R
#done

# getline: delivers the next input line to its invoker
# disassembler: grabs input lines from getline, and delivers them one
#    character at a time to squasher, also inserting a semicolon into
#    the stream between lines
# squasher:  grabs characters from disassembler and passes them on to
#    assembler, first replacing "**" with "^" and squashing runs of
#    whitespace
# assembler: grabs characters from squasher and packs them into lines
#    with 72 character each, delivering each such line to putline;
#    when it sees a null byte, passes the last line to putline and
#    then kills all the coroutines
# putline: grabs lines from assembler, and just prints them

from coro import coroutine, suspend, main, EarlyExit

def getline(text):
    for line in string.splitfields(text, '\n'):
        codisassembler(line)

def disassembler():
    while 1:
        card = cogetline()
        for i in range(len(card)):
            cosquasher(card[i])
        cosquasher(';')

def squasher():
    while 1:
        ch = codisassembler()
        if ch == '*':
            ch2 = codisassembler()
            if ch2 == '*':
                ch = '^'
            else:
                coassembler(ch)
                ch = ch2
        if ch in ' \t':
            while 1:
                ch2 = codisassembler()
                if ch2 not in ' \t':
                    break
            coassembler(' ')
            ch = ch2
        coassembler(ch)

def assembler():
    line = ''
    while 1:
        ch = cosquasher()
        if ch == '\0':
            break
        if len(line) == 72:
            coputline(line)
            line = ''
        line = line + ch
    line = line + ' ' * (72 - len(line))
    coputline(line)

def putline():
    try:
        while 1:
            line = coassembler()
            print line
    except EarlyExit:
        main() # Hack -- resume main coroutine to exit

import string
cogetline = coroutine(getline, test)
coputline = coroutine(putline)
coassembler = coroutine(assembler)
codisassembler = coroutine(disassembler)
cosquasher = coroutine(squasher)

coputline()
print "done"

raw_input("Exit?")

# end of example



From akuchlin at mems-exchange.org  Fri Nov 17 05:17:15 2000
From: akuchlin at mems-exchange.org (A.M. Kuchling)
Date: Thu, 16 Nov 2000 23:17:15 -0500
Subject: [Python-Dev] Python/Distutil draft PEP
Message-ID: <200011170417.XAA00509@207-172-146-153.s153.tnt3.ann.va.dialup.rcn.com>

PEP: XXX
Title: Using Distutils to Build Python
Version: $Revision:  $
Author: akuchlin at mems-exchange.org (A.M. Kuchling)
Status: Draft
Type: Standards
Created: 16-Nov-2000

Introduction

    The Modules/Setup mechanism has some flaws:

    * People have to remember to uncomment bits of Modules/Setup in
      order to get all the possible modules.

    * Moving Setup to a new version of Python is tedious; new modules
      have been added, so you can't just copy the older
      version, but have to reconcile the two versions.

    * Users have to figure out where the needed libraries, such as
      zlib, are installed.


Proposal

    Use the Distutils to build the modules that come with Python.

    The changes can be broken up into several pieces:

    1. The Distutils needs some Python modules to be able to build
       modules.  Currently I believe the minimal list is posix, _sre,
       and string.

       These modules will have to be built before the Distutils can be
       used, so they'll simply be hardwired into Modules/Makefile and 
       be automatically built.  

    2. A top-level setup.py script will be written that checks the
       libraries installed on the system and compiles 
       as many modules as possible.       

    3. Modules/Setup will be kept and settings in it will override
       setup.py's usual behaviour, so you can disable a module known
       to be buggy, or specify particular compilation or linker flags.
       However, in the common case where setup.py works correctly, 
       everything in Setup will remain commented out.  The other
       Setup.*  become unnecessary, since nothing will be generating 
       Setup automatically.


Unresolved Issues
 
    Do we need to make it possible to disable the 3 hard-wired modules
    without manually hacking the Makefiles?  If yes, perhaps a
    configure switch is sufficient.

    The Distutils always compile modules as shared libraries.  How do
    we support compiling them statically into the resulting Python
    binary?
   
    makesetup and the other contents of $(LIBPL)/config need to be
    preserved for compatibility with existing modules; for how many
    versions do we need to keep them around?


Copyright

    This document has been placed in the public domain.



From moshez at zadka.site.co.il  Fri Nov 17 15:04:53 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 16:04:53 +0200
Subject: [Python-Dev] PEP-228 (fw)
Message-ID: <E13wm8M-00085z-00@darjeeling.zadka.site.co.il>

I'm forwarding this message about PEP-228, but I haven't fully
understood it. In any case, since PEP-228 is apparently not
a 2.1 candidate, I've told Michael that his remarks will be considered,
but they are not relevant yet.

------- Forwarded Message

From: Michael McLay <mclay at nist.gov>
Date: Fri, 17 Nov 2000 04:20:45 -0500 (EST)
To: pep at zadka.site.co.il
Subject: PEP:228

I've dropped off the Python mailing lists for the last couple years
because the volume is too high and I don't have time to contribute to
the discussion.  I do read the summaries and PEP:228 was mentioned in
reference to the 2.1 release.  I'd like to see this problem solve in
the 2.1 release and I have a few thoughts on the matter that may be of
use to this task.  Is there a mailing list specific to PEP:228 or
would I have to join python-dev to participate in the discussion?
Please feel free to forward my message to the python-dev list or use
my comments in PEP:228.

Python's use in the XML community is growing and broad-based XML
support in Python may be key to widespread adoption of Python.  There
is some support for XML Schema, the XML Schema Validation software is
written in Python, but there is no direct support in the core Python
XML library.  One problem area for an XML Schema implementation is
related to the numerical types in Python.  The ability to implement
XML Schema types could be improved if the requirements are considered
when creating the PEP:228 implementation.

And from the Python language perspective a review of the numerical
model used in XML Schema may be useful to the discussion of the
revised numerical model for Python.  The authors of the XML Schema
specification put a good deal of effort into defining a modern,
portable numerical model and I think it might be a good fit with the
Python object model.  I might also be possible that the XML Schema
type and entity definition mechanism would be helpful in the
discussion of defining type interfaces for Python.

A table of simple types built into XML Schema are defined in:

   http://www.w3.org/TR/xmlschema-0/#simpleTypesTable

These are defined more fully in "XML Schema Part 2: DataTypes" at:

    http://www.w3.org/TR/2000/CR-xmlschema-2-20001024/

This includes a discussion of the difference between built-inprimitive
and built-inderived datatypes.

The derived types are defined in the XML Schema at: 

    http://www.w3.org/2000/10/datatypes.xsd

In this document, for instance, an integer is derived from a decimal
type in XML Schema. 

 <simpleType name="integer">
   <restriction base="decimal">
      <scale value="0" fixed="true"/>
   </restriction>
 </simpleType>

The use of XML Schema in conjunction with Python would be very helpful
to a soon to be large domain of applications.  I am working with a
group that is currently defining 20 standards using XML Schema and
this is probably just the tip of the iceburg.  I need to be able to
define constrained numerical types as defined in XML Schma.  In Python
this might look like:

  size = ConstrainedDecimal(initial=2, minInclusive=2, 
				       maxInclusive=3000, scale=0)

  for i in range(3000-2):
    size += 1

  print size

The  ConstrainedDecimal would define a restriction range on the value
of "size".  





------- End of Forwarded Message

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From just at letterror.com  Fri Nov 17 09:55:48 2000
From: just at letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 09:55:48 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011170042.TAA16847@cj20424-a.reston1.va.home.com>
Message-ID: <l03102803b63a9e816862@[193.78.237.158]>

At 7:42 PM -0500 16-11-2000, Guido van Rossum wrote:
[ snipp ]
>Using my proposed API this example could be rewritten a bit
>cleaner, as follows:
>
>    from coro import coroutine, suspend, EarlyExit  # Provisional module name
>
>    def fringe(L):
>	for x in L:
>	    if type(x) is type(L):
>		fringe(x)
>	    else:
>		suspend(x)
>
>    def printinorder(L):
>	c = coroutine(f, L)
>	try:
>	    while 1:
>		print c(),
>	except EarlyExit:
>	    pass
>	print
>
>    x = [0, 1, [2, [3]], [4,5], [[[6]]]]
>    printinorder(x) # prints "0 1 2 3 4 5 6"
>
>This is about as simple as it gets.  It supports both generators (like
>the example here) and coroutines (like Tim's Demo/threads/squasher.py
>example).

Neat! (It's _very_ close to what I proposed myself, so how can I not like
it? ;-)

One question: I assume values can also be passed when explicitly resuming a
coroutine. However, the _first_ time a coroutine is "resumed" (started,
really), the stored init arguments get passed to the function. What to do
with a value passed to the resume call? Two possible solutions:
1) don't not allow passing a value the first time
2) add the value(s) to the init arguments

As code:

  def fribble(initarg):
     ...

  c = coroutine(fribble, initarg)
  c(x)  # <-- where does x go? or is it not allowed?

>Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
>propose a function current() which returns the current coroutine
>object.

Starts looking more and more like what I proposed myself...

>There should also be a way to kill a coroutine (or at least
>to send an exception).  When a coroutine falls through at its end,
>*some* other coroutine needs to be resumed.

Here's what I did in my own strawman:
- resume the guy that resumed me, if any (making it equivalent to suspend())
- if we were resumed by a suspend() call (which in my proposal means we
don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
resume the "main" coroutine, which is the (non-coro) code that started the
first coroutine.

Maybe coroutines should have a kill() method, which would post the
EarlyExit exception. Maybe even a postException() method (like uthreads
have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

>I believe this can be implemented with a much simplified stackless
>approach, that doesn't cater towards continuations (but still borrows
>a lot of wisdom from Christian's Stackless).  It can also be
>implemented using threads, which should give some hope for getting the
>same API supported in JPython, making it more attractive.  I am hoping
>to create an implementation on top of Stackless, just to experiment
>with the idiom.

Here's my own code, based on stackless 1.1:
   http://www.petr.com/just/corojust.tar.gz
or
   http://www.petr.com/just/corojust/
for the individual files.

Apart from the initialisation it's pretty much what you propose.

Just





From guido at python.org  Fri Nov 17 13:51:09 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 07:51:09 -0500
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."
             <l03102803b63a9e816862@[193.78.237.158]> 
References: <l03102803b63a9e816862@[193.78.237.158]> 
Message-ID: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>

(A typically brotherly dialog. :-)

[Guido]
> >This is about as simple as it gets.  It supports both generators (like
> >the example here) and coroutines (like Tim's Demo/threads/squasher.py
> >example).

[Just]
> Neat! (It's _very_ close to what I proposed myself, so how can I not like
> it? ;-)

I looked at what you did, and probably a dozen other examples.

> One question: I assume values can also be passed when explicitly resuming a
> coroutine. However, the _first_ time a coroutine is "resumed" (started,
> really), the stored init arguments get passed to the function. What to do
> with a value passed to the resume call? Two possible solutions:
> 1) don't not allow passing a value the first time
> 2) add the value(s) to the init arguments
> 
> As code:
> 
>   def fribble(initarg):
>      ...
> 
>   c = coroutine(fribble, initarg)
>   c(x)  # <-- where does x go? or is it not allowed?

According to Tim, it's a standard issue that every coroutine API has
to deal with.  I lay awake worrying about this for a long time.  I
read the thread about this in the python-coro list.  I even started
thinking about a very different API, that would disinguish between
get() and put() operations.  But there was nothing that satisfied all
constraints.  My current solution: for the initial transfer, *no
argument is allowed*.  (Except None. :-)  I've looked at several
examples (8 queens, squasher, fringe) and they all work fine under
this assumption.

> >Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> >propose a function current() which returns the current coroutine
> >object.
> 
> Starts looking more and more like what I proposed myself...

Great minds and all that... :-)

I've been thinking about names a bit more, and 'main' is a truly lousy
name.  I now propose a function initial() to get at the main
coroutine.  (A function out of symmetry with current(), but also
because in an environment that also supports OS-level threads, there
is a different main coroutine per OS thread.)

> >There should also be a way to kill a coroutine (or at least
> >to send an exception).  When a coroutine falls through at its end,
> >*some* other coroutine needs to be resumed.
> 
> Here's what I did in my own strawman:
> - resume the guy that resumed me, if any (making it equivalent to suspend())

But there could be a cycle here!  What if A resumed B resumed C
resumed A?

> - if we were resumed by a suspend() call (which in my proposal means we
> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
> resume the "main" coroutine, which is the (non-coro) code that started the
> first coroutine.

But the main coroutine may not be expecting this -- e.g. in the
fringe() example, the main coroutine is expecting the next fringe
item or an EarlyExit exception.

> Maybe coroutines should have a kill() method, which would post the
> EarlyExit exception. Maybe even a postException() method (like uthreads
> have), allowing you to post arbitrary exceptions to the coroutine. Dunno.

Indeed.  This needs a bit more thinking (which so far I've
successfully managed to avoid :-).

> >I believe this can be implemented with a much simplified stackless
> >approach, that doesn't cater towards continuations (but still borrows
> >a lot of wisdom from Christian's Stackless).  It can also be
> >implemented using threads, which should give some hope for getting the
> >same API supported in JPython, making it more attractive.  I am hoping
> >to create an implementation on top of Stackless, just to experiment
> >with the idiom.
> 
> Here's my own code, based on stackless 1.1:
>    http://www.petr.com/just/corojust.tar.gz
> or
>    http://www.petr.com/just/corojust/
> for the individual files.
> 
> Apart from the initialisation it's pretty much what you propose.

I saw that, and even peeked a bit at it.  It seems you must have
struggled a bit with the initialization sequence in the squasher
example, gmcm_coro.py (which BTW is due to Tim Peters in its current
-- Python -- form).  That's alright -- coroutine initialization *is* a
problem.

I've found that finalization is a problem, too: I'm not at all sure
that my version of that example correctly frees all the coroutines
when it is done, since several of them are still suspended (waiting
for more input) when the decision to exit is made.  Also, I had to
invent the 'main' API in order to make it work.  Tim's original
(Demo/threads/squasher.py in the CVS tree) has a kill operation that
makes this easier.  His kill() kills the entire *group* of related
coroutines -- a concept that we don't have and probably don't need: it
just makes using the API more complicated, and Tim has said he'd do
away with it if he had to do it again.

BTW, I've got an almost-working implementation of the coro API using
Stackless continuations!  Problem: I don't know how to raise an
exception for a continuation -- which I need to implement EarlyExit.
Can anyone help?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From just at letterror.com  Fri Nov 17 14:44:51 2000
From: just at letterror.com (Just van Rossum)
Date: Fri, 17 Nov 2000 14:44:51 +0100
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
In-Reply-To: <200011171251.HAA24664@cj20424-a.reston1.va.home.com>
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."            
 <l03102803b63a9e816862@[193.78.237.158]>
 <l03102803b63a9e816862@[193.78.237.158]>
Message-ID: <l03102807b63ae45cd9f0@[193.78.237.158]>

At 7:51 AM -0500 17-11-2000, Guido van Rossum wrote:
[ initialisation ]
>My current solution: for the initial transfer, *no
>argument is allowed*.

In retrospect, that indeed seems the best solution. I wonder why we (as in
the python-coro at egroups clan) didn't think of that before... Glad to have
you here ;-)


[ "main" coro ]
>I've been thinking about names a bit more, and 'main' is a truly lousy
>name.  I now propose a function initial() to get at the main
>coroutine.  (A function out of symmetry with current(), but also
>because in an environment that also supports OS-level threads, there
>is a different main coroutine per OS thread.)

Fine with me.

>> Here's what I did in my own strawman:
>> - resume the guy that resumed me, if any (making it equivalent to suspend())
>
>But there could be a cycle here!  What if A resumed B resumed C
>resumed A?

Here's what I did: if A does suspend(), it resumes C, but at the same time
A _loses_ it's "resumer". So if it does do a suspend() again, it'll resume
the "initial" coro.

>> - if we were resumed by a suspend() call (which in my proposal means we
>> don't have a "resumer", to avoid endless suspend()/suspend() bouncing),
>> resume the "main" coroutine, which is the (non-coro) code that started the
>> first coroutine.
>
>But the main coroutine may not be expecting this -- e.g. in the
>fringe() example, the main coroutine is expecting the next fringe
>item or an EarlyExit exception.

Sure, but you need to be able to go _somewhere_, and this seems more useful
than, say, terminating the app ;-)

[ my own implementiation ]
>> Apart from the initialisation it's pretty much what you propose.
>
>I saw that, and even peeked a bit at it.  It seems you must have
>struggled a bit with the initialization sequence in the squasher
>example, gmcm_coro.py (which BTW is due to Tim Peters in its current
>-- Python -- form).  That's alright -- coroutine initialization *is* a
>problem.

Still, the first-time-no-arguments-allowed rule solves it nicely.

>I've found that finalization is a problem, too: I'm not at all sure
>that my version of that example correctly frees all the coroutines
>when it is done, since several of them are still suspended (waiting
>for more input) when the decision to exit is made.

Are you talking of freeing all data associated with the coroutine, or about
properly finalizing it (so that any finally clauses get executed)? If the
former: don't even try, it's a nightmare with continuations. If the latter:
good question ;-).

>BTW, I've got an almost-working implementation of the coro API using
>Stackless continuations!  Problem: I don't know how to raise an
>exception for a continuation -- which I need to implement EarlyExit.
>Can anyone help?

def hellraiser():
    continuation.return_current()
    raise <whatever>

k = hellraiser()
k.caller = your_co
k()

Just





From tismer at tismer.com  Fri Nov 17 14:47:53 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 15:47:53 +0200
Subject: [Python-Dev] Re: [python-coro] coroutines and microthreads
References: Your message of "Fri, 17 Nov 2000 09:55:48 +0100."             <l03102803b63a9e816862@[193.78.237.158]> <l03102803b63a9e816862@[193.78.237.158]> <l03102807b63ae45cd9f0@[193.78.237.158]>
Message-ID: <3A153709.A56845AC@tismer.com>


Just van Rossum wrote:
...
> >BTW, I've got an almost-working implementation of the coro API using
> >Stackless continuations!  Problem: I don't know how to raise an
> >exception for a continuation -- which I need to implement EarlyExit.
> >Can anyone help?
> 
> def hellraiser():
>     continuation.return_current()
>     raise <whatever>
> 
> k = hellraiser()
> k.caller = your_co
> k()
> 
> Just

Hey, there is it again, the little trick. I love it :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From thomas.heller at ion-tof.com  Fri Nov 17 17:21:50 2000
From: thomas.heller at ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 17:21:50 +0100
Subject: [Python-Dev] imputil
Message-ID: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>

imputil, which is now an official part of
python, changes the semantics of sys.path
as soon as importers are installed.

On the other hand, a quick and dirty search
finds at least these files in the standard library
where it is assumed that sys.path is a list of strings:

linecache.py, profile.py, pdb.py, pyclbr.py

Should imputil be fixed to install the ImportManager
in a different way (not installing the importers
into sys.path), or should the library be fixed?

Thomas





From jeremy at alum.mit.edu  Fri Nov 17 05:31:23 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 16 Nov 2000 23:31:23 -0500 (EST)
Subject: [Python-Dev] screwup on the patches & bugs notifications
Message-ID: <14868.46235.22940.869048@bitdiddle.concentric.net>

I hurried to get notifications of outstanding patches and bugs last
night and botched both of them.  Sorry about that.  If you can a
message about patches, you can disregard it entirely.  It was made from
an old snapshot of the SF database.  The bug mail contains current
bugs, but the formatting of the mail headers was off.  I've fixed that
for the future.

Jeremy



From guido at python.org  Fri Nov 17 17:35:09 2000
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Nov 2000 11:35:09 -0500
Subject: [Python-Dev] imputil
In-Reply-To: Your message of "Fri, 17 Nov 2000 17:21:50 +0100."
             <038701c050b2$7dd00740$e000a8c0@thomasnotebook> 
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook> 
Message-ID: <200011171635.LAA26120@cj20424-a.reston1.va.home.com>

> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

Sorry to burst your bubble.  Imputil is still experimental.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Fri Nov 17 17:45:58 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 17:45:58 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook>
Message-ID: <3A1560C6.B872D948@lemburg.com>

Thomas Heller wrote:
> 
> imputil, which is now an official part of
> python, changes the semantics of sys.path
> as soon as importers are installed.
> 
> On the other hand, a quick and dirty search
> finds at least these files in the standard library
> where it is assumed that sys.path is a list of strings:
> 
> linecache.py, profile.py, pdb.py, pyclbr.py
> 
> Should imputil be fixed to install the ImportManager
> in a different way (not installing the importers
> into sys.path), or should the library be fixed?

My understanding was that Importers need to provide
a __str__ method which is then used... haven't looked
at imputil.py in ages though, so I can't really comment.

Perhaps imputil.py should leave sys.path alone (or maybe just
wipe it from unneeded entries) and use a new sys.importers
object for the import management ?!

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fdrake at acm.org  Fri Nov 17 17:58:47 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri, 17 Nov 2000 11:58:47 -0500 (EST)
Subject: [Python-Dev] forwarded message from nimoy
Message-ID: <14869.25543.871007.185471@cj42289-a.reston1.va.home.com>


  It sounds like people are happy with Python.  ;)  Just thought I'd
pass this bit of praise araound, since it seems to apply to everyone's
efforts.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations

-------------- next part --------------
An embedded message was scrubbed...
From: "nimoy" <jtnimoy at ucla.edu>
Subject: feedback
Date: Thu, 16 Nov 2000 21:26:26 -0800
Size: 2337
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001117/f3c6904b/attachment-0001.eml>

From thomas.heller at ion-tof.com  Fri Nov 17 18:24:17 2000
From: thomas.heller at ion-tof.com (Thomas Heller)
Date: Fri, 17 Nov 2000 18:24:17 +0100
Subject: [Python-Dev] imputil
References: <038701c050b2$7dd00740$e000a8c0@thomasnotebook> <3A1560C6.B872D948@lemburg.com>
Message-ID: <04a501c050bb$372df8c0$e000a8c0@thomasnotebook>

> Thomas Heller wrote:
> > 
> > imputil, which is now an official part of
> > python, changes the semantics of sys.path
> > as soon as importers are installed.
> > 
> > On the other hand, a quick and dirty search
> > finds at least these files in the standard library
> > where it is assumed that sys.path is a list of strings:
> > 
> > linecache.py, profile.py, pdb.py, pyclbr.py
> > 
> > Should imputil be fixed to install the ImportManager
> > in a different way (not installing the importers
> > into sys.path), or should the library be fixed?
[MAL]
> 
> My understanding was that Importers need to provide
> a __str__ method which is then used... haven't looked
> at imputil.py in ages though, so I can't really comment.
> 
No importer currently does define __str__, neither is it used
in the standard libary. The pattern is (in linecache):

        for dirname in sys.path:
            fullname = os.path.join(dirname, basename)

> Perhaps imputil.py should leave sys.path alone (or maybe just
> wipe it from unneeded entries) and use a new sys.importers
> object for the import management ?!
This sounds better.

Thomas





From mal at lemburg.com  Fri Nov 17 18:13:42 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 18:13:42 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
Message-ID: <3A156746.D2B59356@lemburg.com>

Hi,

sorry to bother you again with this, but the current state
of Python's community elements is in such a bad shape, that
I'm getting more and more frustrated...

First, starship is taken off the net due to a mailing problem.
After a few days starship resurfaces: but apparently using
some really old backup which didn't even contain all files
-- at least the .ssh dirs seem to be lost causing RSA logins
using SSH to fail (don't know about normal logins -- my old
passwords don't work anymore either, but that could have been
caused by the harddisk crash earlier this year).

As consequence, I moved all my Python Pages to a new site
which should provide more stability. Next, I tried to get this
message across to the comp.lang.python.* crowd: I posted
messages to the python.org mailing lists which should have
been gatewayed to the newsgroups -- nothing happened. The
messages seem to be stuck somewhere half-way across the 
Atlantic.

The result: even though I tried to do some good, by moving my
site, noone will notice and I still get complaints about
my extensions not working with 2.0 -- the announcement about
the new versions *did* make it to comp.lang.python and the
announcement list: but it still had the starship URL on it,
so now people find the announcement and there's no way
to point them to the new URL (I can't login to starship to
post a notice there...).

Help someone, please.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From akuchlin at mems-exchange.org  Fri Nov 17 18:45:07 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 12:45:07 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <3A156746.D2B59356@lemburg.com>; from mal@lemburg.com on Fri, Nov 17, 2000 at 06:13:42PM +0100
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <20001117124507.G25735@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 06:13:42PM +0100, M.-A. Lemburg wrote:
>As consequence, I moved all my Python Pages to a new site
>which should provide more stability. Next, I tried to get this

Frankly I think this is the right move, and my pages are now at amk.ca
for pretty much the same reason.  

>message across to the comp.lang.python.* crowd: I posted
>messages to the python.org mailing lists which should have
>been gatewayed to the newsgroups -- nothing happened. The

News/mail gatewaying seems fairly seriously broken at the moment; I've
noticed many followups to posts that I've never seen.  I read c.l.p on
news.cnri.reston.va.us, presumably the same machine used for NNTP
posting by python.org's Mailman, which would imply that Mailman is at
fault.  

--amk



From tismer at tismer.com  Fri Nov 17 17:59:57 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 18:59:57 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A15640D.C2412AB1@tismer.com>

Marc,

"M.-A. Lemburg" wrote:

[justified complaints about Starship]

I'd like to apologize for this mess, but I'd be happier
if I were guilty. Instead, I'm unable to do anything.
Since the Starship has begun to move
again, it got out of my control. I still don't have root
access to the new machine via ssh. This might be by chance,
at least I hope so!
This situation is absolutely what I do not want.
If things don't improve quickly, I might consider
to move the domain to a better server. Moving to
BeOpen should have been a good thing. Finally it
wasn't, but nobody could predict this. Then Starship
went into some Zombie state and was unaccessible
when the DC site was built. Meanwhile the old ship
re-appeared under a different IP, and Jim Tittsler
offered to do a transfer of the current data, but he
also seems to have no super user for the new site.

BTW, you might have ssh access via port 10022, at least
for me this works just fine:

ssh -v -l pirx -p 10022 starship.python.net

Marc, you are the first Starship user! If you leave,
I consider the project dead. Please don't.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From barry at digicool.com  Fri Nov 17 18:59:36 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 12:59:36 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
	<20001117124507.G25735@kronos.cnri.reston.va.us>
Message-ID: <14869.29192.697233.746198@anthem.concentric.net>

>>>>> "AK" == Andrew Kuchling <akuchlin at mems-exchange.org> writes:

    AK> News/mail gatewaying seems fairly seriously broken at the
    AK> moment; I've noticed many followups to posts that I've never
    AK> seen.  I read c.l.p on news.cnri.reston.va.us, presumably the
    AK> same machine used for NNTP posting by python.org's Mailman,
    AK> which would imply that Mailman is at fault.

Not necessarily.

First, there's certainly nothing in the Mailman log files indicating
any problems writing to nntp.  Second, Mailman on python.org actually
gates messages to ournews.cnri.reston.va.us which is an internal
machine on the CNRI network, and should be one hop downstream from
news.cnri.reston.va.us (a.k.a. news.alterdial.uu.net).

My prediction is that the news is piling up on ournews and there are
problems feeding those messages upstream to alterdial.  This is not a
rare occurance unfortunately.  A while back I contacted the CNRI
admins about this, but I don't remember hearing a positive resolution.

I'm reluctant to point python-list's newserver directly at the
alterdial machine because I remember there were worse problems with
that approach (like postings actually getting lost as opposed to just
sitting behind a log jam -- ournews never expires c.l.py.* postings).

Andrew, maybe you can ping the folks at CNRI more directly to at least
make sure that there are no problems between ournews<->news?

-Barry



From akuchlin at mems-exchange.org  Fri Nov 17 19:08:56 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 13:08:56 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
In-Reply-To: <14869.29192.697233.746198@anthem.concentric.net>; from barry@digicool.com on Fri, Nov 17, 2000 at 12:59:36PM -0500
References: <3A156746.D2B59356@lemburg.com> <20001117124507.G25735@kronos.cnri.reston.va.us> <14869.29192.697233.746198@anthem.concentric.net>
Message-ID: <20001117130856.C7763@kronos.cnri.reston.va.us>

On Fri, Nov 17, 2000 at 12:59:36PM -0500, Barry A. Warsaw wrote:
>My prediction is that the news is piling up on ournews and there are
>problems feeding those messages upstream to alterdial.  This is not a

And I believe you're correct; pointing a newsreader at ournews, I can
see postings (such as MAL's "ANN: New URL for the mx-Extensions") that
I never saw when reading news on uunet's server, so this is another
example of CNRI's incompetent sysadmining.  I've sent a note off
asking them to look into it.

--amk



From barry at digicool.com  Fri Nov 17 19:10:03 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Fri, 17 Nov 2000 13:10:03 -0500
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
	<3A15640D.C2412AB1@tismer.com>
Message-ID: <14869.29819.597320.836410@anthem.concentric.net>

>>>>> "CT" == Christian Tismer <tismer at tismer.com> writes:

    CT> I still don't have root access to the new machine via
    CT> ssh. This might be by chance, at least I hope so!

Gotta be, because /nobody/ had root access via ssh until moments ago
(I have root access to the underlying machine -- outside the chroot
jail).  I just cat'd my authorized_keys file, as well as pirx's and
jwt's to starship's root account, so Chris, Jim Tittsler and I at
least should be able to get in as root at starship.python.net.

This should let Jim do any necessary unpacking of transferred data.

-Barry



From tismer at tismer.com  Fri Nov 17 18:19:07 2000
From: tismer at tismer.com (Christian Tismer)
Date: Fri, 17 Nov 2000 19:19:07 +0200
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
		<3A15640D.C2412AB1@tismer.com> <14869.29819.597320.836410@anthem.concentric.net>
Message-ID: <3A15688B.B69C7878@tismer.com>


"Barry A. Warsaw" wrote:
> 
> >>>>> "CT" == Christian Tismer <tismer at tismer.com> writes:
> 
>     CT> I still don't have root access to the new machine via
>     CT> ssh. This might be by chance, at least I hope so!
> 
> Gotta be, because /nobody/ had root access via ssh until moments ago
> (I have root access to the underlying machine -- outside the chroot
> jail).  I just cat'd my authorized_keys file, as well as pirx's and
> jwt's to starship's root account, so Chris, Jim Tittsler and I at
> least should be able to get in as root at starship.python.net.
> 
> This should let Jim do any necessary unpacking of transferred data.

Ahhh, puuuh, that's great. :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From mal at lemburg.com  Fri Nov 17 20:55:13 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri, 17 Nov 2000 20:55:13 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com> <3A15640D.C2412AB1@tismer.com>
Message-ID: <3A158D21.134A44EE@lemburg.com>

Christian Tismer wrote:

[This trick got me started again... thanks.]
> BTW, you might have ssh access via port 10022, at least
> for me this works just fine:
> 
> ssh -v -l pirx -p 10022 starship.python.net
> 
> Marc, you are the first Starship user! If you leave,
> I consider the project dead. Please don't.

Sorry, Christian, but things got so much out of hand
lately, that I think a new site for my stuff will make
my users happier.

With the help of Andrew I have added redirects for all
my files on starship to the new site, so the move should
be painless (the old URLs will continue to work). I'm
sort of a virtual starship user now ;-)

Anyway, starship is not dead -- it has been a great
community site for the past three years and I hope that
it will now regain the status it had until early this
year when things started to go wrong due to the various
well-known problems.

Let's take this relaunch as a chance to get things moving
again. I'd suggest to restart providing everybody in the Python 
community should with an account on the server to host Python
software in return for the annual fee (what was the PSA-membership
fee until recently). This should by the sysadmins enough
pizza to keep the server running for a looong time ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Sat Nov 18 13:51:16 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 18 Nov 2000 13:51:16 +0100
Subject: [Python-Dev] Starship, Python.org, etc.
References: <3A156746.D2B59356@lemburg.com>
Message-ID: <3A167B44.4D53B796@lemburg.com>

"M.-A. Lemburg" wrote:
> 
> Help someone, please.

Thank you to everyone who jumped to the rescue :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From ping at lfw.org  Sat Nov 18 22:57:31 2000
From: ping at lfw.org (Ka-Ping Yee)
Date: Sat, 18 Nov 2000 13:57:31 -0800 (PST)
Subject: [Python-Dev] socket.error should be an IOError?
Message-ID: <Pine.LNX.4.10.10011181352250.504-100000@skuld.kingmanhall.org>

In socketmodule, socket.error is currently just a plain old error
(derived only from Exception).  Looks to me like it should be an IOError.
Do you agree?

 line 2379 of socketmodule.c:

???????d = PyModule_GetDict(m);
???????PySocket_Error = PyErr_NewException("socket.error", NULL, NULL);
???????if (PySocket_Error == NULL)

...just change that first NULL to PyExc_IOError.



-- ?!ng

"All models are wrong; some models are useful."
    -- George Box




From jack at oratrix.nl  Sun Nov 19 23:27:14 2000
From: jack at oratrix.nl (Jack Jansen)
Date: Sun, 19 Nov 2000 23:27:14 +0100
Subject: [Python-Dev] site.py, .pth files and unix-dependencies
Message-ID: <20001119222719.5980A1301E4@oratrix.oratrix.nl>

In the course of getting distutils to behave on the Mac I came across
.pth files for the first time (all packages I ever tried until now
used the __init__.py method of advertising themselves) and it turns
out they don't work on the Mac.

There's some funny code in site.py that triggers on "os.sep ==
'/'". I've added an "elif os.sep == ':'" for the mac-specific location 
where site-python is (sys.prefix + :lib:site-python), but checking for 
os.sep in stead of simply checking that sys.platform==posix makes me
wonder whether something infinitely more deep than my understanding
allows me to grasp is going on.

Can someone enlighten me? Is my fix okay and should I check it in?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen at oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++



From akuchlin at mems-exchange.org  Mon Nov 20 23:32:30 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Mon, 20 Nov 2000 17:32:30 -0500
Subject: [Python-Dev] Too much silence
Message-ID: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>

All the PEPs for 2.1 are supposed to be complete for Dec. 16, and some
of those PEPs are pretty complicated.  I'm a bit worried that it's
been so quiet on python-dev lately, especially after the previous two
weeks of lively discussion.  (And no comments on my Python/Distutils
PEP at all -- not even an assigned number... <sniffle>)

--amk



From nas at arctrix.com  Mon Nov 20 17:52:16 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 08:52:16 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001120085216.A30685@glacier.fnational.com>

No doubt.  I'm still waiting for some feedback on my coercion
patch.  I'd write a PEP but I don't think it would help.  Marc's
web page already explains things in detail.  Also, I don't want
to spend too much time going down a path that Guido and the other
developers are not going to accept.

The patch makes some pretty major changes to Python's internals
(for the better I think).  If something is going to be done for
2.1 then it should happen sooner rather than later.

  Neil



From jeremy at alum.mit.edu  Tue Nov 21 00:39:25 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 20 Nov 2000 18:39:25 -0500 (EST)
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120085216.A30685@glacier.fnational.com>
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
	<20001120085216.A30685@glacier.fnational.com>
Message-ID: <14873.46637.76321.845888@bitdiddle.concentric.net>

Should Marc's Web page be turned into the PEP then?  I don't have time
to read a patch, but I do have time to read a PEP.

Jeremy



From mal at lemburg.com  Tue Nov 21 00:58:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 00:58:10 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
		<20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net>
Message-ID: <3A19BA92.1A555299@lemburg.com>

Jeremy Hylton wrote:
> 
> Should Marc's Web page be turned into the PEP then?  I don't have time
> to read a patch, but I do have time to read a PEP.

If someone could do this, I'd appreciate it. I like Neil's
idea of using a type flag to signal "new style number", but apart
from that, I think that the web page pretty much makes things
clear.

The patch would need to be updated to Python 2.0, though -- don't
have time for this myself, so it's up to you.

I completely agree with Neil, that these things should be
done now rather than later: the current coercion mechanism is
a major bottleneck when it comes to implementing new types in C
that are supposed to interoperate with existing builtin types.

What the patch basically does is add Python's instance coercion
support to the C level -- in a backward compatible way.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From nas at arctrix.com  Mon Nov 20 18:54:45 2000
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 20 Nov 2000 09:54:45 -0800
Subject: [Python-Dev] Too much silence
In-Reply-To: <3A19BA92.1A555299@lemburg.com>; from mal@lemburg.com on Tue, Nov 21, 2000 at 12:58:10AM +0100
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com>
Message-ID: <20001120095445.A30774@glacier.fnational.com>

On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> Jeremy Hylton wrote:
> > Should Marc's Web page be turned into the PEP then?  I don't have time
> > to read a patch, but I do have time to read a PEP.

> The patch would need to be updated to Python 2.0, though -- don't
> have time for this myself, so it's up to you.

My patch _is_ up to date.  My original message also included a
brief description of what the patch does.  For more details look
at the URL to Marc's page that I provided.  There is also PEP 208
(which should probably be updated).

Marc and Moshe: who is currently championing PEP 207 and 208?

Barry or Jeremy: the formatting of 208 on sourceforge seems to be
screwy.  Part of the extended print PEP seems to be included.

I'll increase the traffic on this list yet. :)

  Neil



From tim.one at home.com  Tue Nov 21 02:21:15 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 20 Nov 2000 20:21:15 -0500
Subject: [Python-Dev] Too much silence
In-Reply-To: <20001120095445.A30774@glacier.fnational.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOFIAAA.tim.one@home.com>

[Neil Schemenauer]
> My patch _is_ up to date.  My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).

Yes; PEP 208 should be self-contained.

> Marc and Moshe: who is currently championing PEP 207 and 208?
>
> Barry or Jeremy: the formatting of 208 on sourceforge seems to be
> screwy.  Part of the extended print PEP seems to be included.

PEP 208 is in a severely broken state.  This is the full content of
pep-0208.txt:

====================================
PEP: 208
Title: Reworking the Coercion Model
Version: $Revision: 1.1 $
Owner: davida at activestate.com (David Ascher)
Python-Version: 2.1
Status: Incomplete



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
====================================

No change has been checked in since this original skeleton was added in the
middle of July.  So I guess somebody has been uploading damaged HTML for 208
without going thru CVS and the HTML generation process.  Nothing Barry or
Jeremy can do about that:  somebody has to update the .txt file.

> I'll increase the traffic on this list yet. :)

Guido will eventually chime in on the coercions patch, but is on vacation
until next Monday (however, AFAIK, he didn't release anything first <wink>).
I'll chime in too, but am almost in the same boat.

here-yet-absent-ly y'rs  - tim




From moshez at zadka.site.co.il  Tue Nov 21 14:06:31 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 15:06:31 +0200
Subject: [Python-Dev] Too much silence 
In-Reply-To: Message from Neil Schemenauer <nas@arctrix.com> 
   of "Mon, 20 Nov 2000 08:52:16 PST." <20001120085216.A30685@glacier.fnational.com> 
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>  <20001120085216.A30685@glacier.fnational.com> 
Message-ID: <E13yD84-0004hI-00@darjeeling.zadka.site.co.il>

> No doubt.  I'm still waiting for some feedback on my coercion
> patch.  I'd write a PEP but I don't think it would help.  Marc's
> web page already explains things in detail.  Also, I don't want
> to spend too much time going down a path that Guido and the other
> developers are not going to accept.

Neil, you can take the coercion patch from me. I thought I'd
have more time to deal with it then I do. Please let me know.
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From thomas at xs4all.net  Tue Nov 21 09:47:44 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 21 Nov 2000 09:47:44 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Nov 20, 2000 at 05:32:30PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
Message-ID: <20001121094744.D540@xs4all.nl>

On Mon, Nov 20, 2000 at 05:32:30PM -0500, Andrew Kuchling wrote:

> I'm a bit worried that it's been so quiet on python-dev lately, especially
> after the previous two weeks of lively discussion.

I'll appologize for my silence, but I can't speak for the rest of -dev ;)
I'm going to be silent for a few months more, though, as I'm swamped in work
and supposed to replace my boss during his 2-month vacation. I'll spend what
little time I have on catalog-sig rather than -dev issues, though I'll try
to keep up with the messages on -dev at least ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From mal at lemburg.com  Tue Nov 21 10:22:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 21 Nov 2000 10:22:56 +0100
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>
Message-ID: <3A1A3EF0.3584B425@lemburg.com>

Neil Schemenauer wrote:
> 
> On Tue, Nov 21, 2000 at 12:58:10AM +0100, M.-A. Lemburg wrote:
> > Jeremy Hylton wrote:
> > > Should Marc's Web page be turned into the PEP then?  I don't have time
> > > to read a patch, but I do have time to read a PEP.
> 
> > The patch would need to be updated to Python 2.0, though -- don't
> > have time for this myself, so it's up to you.
> 
> My patch _is_ up to date. 

Cool... sorry, but I didn't have time to look closely.

> My original message also included a
> brief description of what the patch does.  For more details look
> at the URL to Marc's page that I provided.  There is also PEP 208
> (which should probably be updated).
> 
> Marc and Moshe: who is currently championing PEP 207 and 208?

Not me -- got no time to spare.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Tue Nov 21 12:31:34 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 13:31:34 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
Message-ID: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>

I've looked at the people bugs can be assigned to, and Greg Stein is
missing. Is this purposeful or just an oversight?

Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
would look at it -- it's his code originally, so maybe he does not think
it's a bug.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From gstein at lyra.org  Tue Nov 21 13:34:44 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 04:34:44 -0800
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>; from moshez@math.huji.ac.il on Tue, Nov 21, 2000 at 01:31:34PM +0200
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
Message-ID: <20001121043444.K21426@lyra.org>

On Tue, Nov 21, 2000 at 01:31:34PM +0200, Moshe Zadka wrote:
> I've looked at the people bugs can be assigned to, and Greg Stein is
> missing. Is this purposeful or just an oversight?
> 
> Specifically, bug 123045 seems to be a two-second fix, but I'd rather Greg
> would look at it -- it's his code originally, so maybe he does not think
> it's a bug.

Probably an oversight. I've had bugs assigned before, but a month or so ago
I accidentally removed myself from the Python project (damn trash can on the
"my" page is located in a horrible place, with no conformation). Tim put me
back, but there is probably a magic switch somewhere to enable bug
assignments.

I'm going to mail Roy Fielding to see if servers or clients are allowed to
send lower-cased HTTP version strings. Looking at the Apache code, it will
assume HTTP/1.0 if a client sends a lower-cased version. I'm suspicious that
the RFC didn't clarify the case-sensitivity on that string (yes, the bug
submitter is correct that the RFC allows a lower-cased version, but that
just seems wrong/mis-intended).

In any case... pending the input from Roy, I'll deal with the bug.

But one of the SF Python Admins will need to enable my bug db privs to work
on that bug properly.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From fdrake at acm.org  Tue Nov 21 15:41:31 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 09:41:31 -0500 (EST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <20001121043444.K21426@lyra.org>
References: <Pine.GSO.4.10.10011211328450.14461-100000@sundial>
	<20001121043444.K21426@lyra.org>
Message-ID: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > Probably an oversight. I've had bugs assigned before, but a month or so ago
 > I accidentally removed myself from the Python project (damn trash can on the
 > "my" page is located in a horrible place, with no conformation). Tim put me
 > back, but there is probably a magic switch somewhere to enable bug
 > assignments.

  I've fixed the permissions; you're back!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at math.huji.ac.il  Tue Nov 21 22:35:49 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Tue, 21 Nov 2000 23:35:49 +0200 (IST)
Subject: [Python-Dev] Greg Stein in Bug Assignments?
In-Reply-To: <14874.35227.380150.19752@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011212334230.10616-100000@sundial>

On Tue, 21 Nov 2000, Fred L. Drake, Jr. wrote:

>   I've fixed [Greg Stein's] permissions; you're back!  ;-)

Well, the bright side is that it generated some noise on Python-Dev
<wink>.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Tue Nov 21 23:06:04 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:06:04 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121170604.B15925@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
>Update of /cvsroot/python/python/dist/src/Lib/xml/dom
>In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
>
>Modified Files:
>	minidom.py 
...

Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
has wrong semantics".

Does the patch check for illegal children (bug #116678).

--amk



From fdrake at acm.org  Tue Nov 21 23:16:52 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:16:52 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121170604.B15925@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
Message-ID: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Yay!  This checkin may fix Bug #116677: "minidom:Node.appendChild()
 > has wrong semantics".

  Sorry; that would have taken just enough more that I didn't want to
get into that today, but it shouldn't be hard.  See my comments on the
bug page (just added).

 > Does the patch check for illegal children (bug #116678).

  No.  There's still seriously little parameter checking in minidom,
and I'm not sure I want to add that.  One of the problems people had
with PyOM was the general weight of the code, and adding all the
checks contributes to that (though I suspect the proxies in PyDOM were
a far more serious contributor).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From akuchlin at mems-exchange.org  Tue Nov 21 23:22:33 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:22:33 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:16:52PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172233.A16224@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:16:52PM -0500, Fred L. Drake, Jr. wrote:
>and I'm not sure I want to add that.  One of the problems people had
>with PyDOM was the general weight of the code, and adding all the
>checks contributes to that (though I suspect the proxies in PyDOM were
>a far more serious contributor).

Indeed; I think the proxies really obfuscated the code.  Some simple
parameter checking, though, shouldn't add too much of a burden, and
will protect users from common mistakes that will result in invalid
trees.

--amk




From fdrake at acm.org  Tue Nov 21 23:20:47 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:20:47 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121172233.A16224@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > Indeed; I think the proxies really obfuscated the code.  Some simple
 > parameter checking, though, shouldn't add too much of a burden, and
 > will protect users from common mistakes that will result in invalid
 > trees.

  So when should we be expecting your patch?  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From akuchlin at mems-exchange.org  Tue Nov 21 23:25:08 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:25:08 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Tue, Nov 21, 2000 at 05:20:47PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <14874.62783.46263.857491@cj42289-a.reston1.va.home.com>
Message-ID: <20001121172508.A16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:20:47PM -0500, Fred L. Drake, Jr. wrote:
>  So when should we be expecting your patch?  ;-)

Expect it when you see it. 

--amk



From paulp at ActiveState.com  Tue Nov 21 23:34:29 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 14:34:29 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us>
Message-ID: <3A1AF875.4110F856@activestate.com>

Andrew Kuchling wrote:
> 
> ...
> 
> Indeed; I think the proxies really obfuscated the code.  Some simple
> parameter checking, though, shouldn't add too much of a burden, and
> will protect users from common mistakes that will result in invalid
> trees.

Those checks would slow down the original tree building unless we split
the interface into "internal" methods that we use ourself and "external
methods" that do the extra checking. Worth the effort and extra code
complexity? Maybe...maybe not.

 Paul Prescod



From jeremy at alum.mit.edu  Tue Nov 21 23:26:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:26:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <3A1AF875.4110F856@activestate.com>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
Message-ID: <14874.63110.889245.738951@bitdiddle.concentric.net>

>>>>> "PP" == Paul Prescod <paulp at ActiveState.com> writes:

  PP> Andrew Kuchling wrote:
  >>
  >> ...
  >>
  >> Indeed; I think the proxies really obfuscated the code.  Some
  >> simple parameter checking, though, shouldn't add too much of a
  >> burden, and will protect users from common mistakes that will
  >> result in invalid trees.

  PP> Those checks would slow down the original tree building unless
  PP> we split the interface into "internal" methods that we use
  PP> ourself and "external methods" that do the extra checking. Worth
  PP> the effort and extra code complexity? Maybe...maybe not.

Could those checks be implemented as assertions?  If so, people who
care about speed can use "python -O"

Jeremy




From akuchlin at mems-exchange.org  Tue Nov 21 23:43:50 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 21 Nov 2000 17:43:50 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121174350.B16259@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
>Could those checks be implemented as assertions?  If so, people who
>care about speed can use "python -O"

Making error checks optional and leaving it up to the user to avoid
problems...  did I get subscribed to the perl5-porters list by
mistake?

--amk



From fdrake at acm.org  Tue Nov 21 23:41:32 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2000 17:41:32 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
	<14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <14874.64028.658410.793457@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > Could those checks be implemented as assertions?  If so, people who
 > care about speed can use "python -O"

  Yes, but it is not clear that the checks are expensive.
  Another issue is compliance with the spec -- DOM level 1 states that
certain exceptions will be raised for various conditions, and using
assertions to check those would mean that exceptions would *not* be
raised in those cases.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From jeremy at alum.mit.edu  Tue Nov 21 23:38:14 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 21 Nov 2000 17:38:14 -0500 (EST)
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121174350.B16259@kronos.cnri.reston.va.us>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121170604.B15925@kronos.cnri.reston.va.us>
	<14874.62548.963722.34329@cj42289-a.reston1.va.home.com>
	<20001121172233.A16224@kronos.cnri.reston.va.us>
	<3A1AF875.4110F856@activestate.com>
	<14874.63110.889245.738951@bitdiddle.concentric.net>
	<20001121174350.B16259@kronos.cnri.reston.va.us>
Message-ID: <14874.63830.803853.153636@bitdiddle.concentric.net>

>>>>> "AMK" == Andrew Kuchling <akuchlin at mems-exchange.org> writes:

  AMK> On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
  >> Could those checks be implemented as assertions?  If so, people
  >> who care about speed can use "python -O"

  AMK> Making error checks optional and leaving it up to the user to
  AMK> avoid problems...  did I get subscribed to the perl5-porters
  AMK> list by mistake?

Not quite what I asked about: Enabling error checks by default, but
allowing users to turn them off in optmized mode.  If the checks are
expensive, which Fred subsequently said he wasn't sure about, this
might not be unreasonable.

Perhaps I'm just odd when it comes to -O.  I've never used it.

Jeremy




From gstein at lyra.org  Tue Nov 21 23:55:39 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:55:39 -0800
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <14874.63110.889245.738951@bitdiddle.concentric.net>; from jeremy@alum.mit.edu on Tue, Nov 21, 2000 at 05:26:14PM -0500
References: <200011212202.OAA16186@slayer.i.sourceforge.net> <20001121170604.B15925@kronos.cnri.reston.va.us> <14874.62548.963722.34329@cj42289-a.reston1.va.home.com> <20001121172233.A16224@kronos.cnri.reston.va.us> <3A1AF875.4110F856@activestate.com> <14874.63110.889245.738951@bitdiddle.concentric.net>
Message-ID: <20001121145539.B21426@lyra.org>

On Tue, Nov 21, 2000 at 05:26:14PM -0500, Jeremy Hylton wrote:
> >>>>> "PP" == Paul Prescod <paulp at ActiveState.com> writes:
>   PP> Andrew Kuchling wrote:
>   >> ...
>   >> Indeed; I think the proxies really obfuscated the code.  Some
>   >> simple parameter checking, though, shouldn't add too much of a
>   >> burden, and will protect users from common mistakes that will
>   >> result in invalid trees.
> 
>   PP> Those checks would slow down the original tree building unless
>   PP> we split the interface into "internal" methods that we use
>   PP> ourself and "external methods" that do the extra checking. Worth
>   PP> the effort and extra code complexity? Maybe...maybe not.
> 
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

+1 ... that would be the way to do it.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From gstein at lyra.org  Tue Nov 21 23:56:52 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 14:56:52 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <200011212202.OAA16186@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Nov 21, 2000 at 02:02:24PM -0800
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
Message-ID: <20001121145652.C21426@lyra.org>

On Tue, Nov 21, 2000 at 02:02:24PM -0800, Fred L. Drake wrote:
> Update of /cvsroot/python/python/dist/src/Lib/xml/dom
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv16177/Lib/xml/dom
> 
> Modified Files:
> 	minidom.py 
> Log Message:
>...
> *** minidom.py	2000/10/23 18:09:50	1.13
> --- minidom.py	2000/11/21 22:02:22	1.14
> ***************
> *** 15,22 ****
>   """
>   
> - import pulldom
>   import string

This "import string" plus numerous uses within minidom seem a bit extraneous
since this is part of the Python 2.0 library...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From tim.one at home.com  Wed Nov 22 00:09:39 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 18:09:39 -0500
Subject: [Python-Dev] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145539.B21426@lyra.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>

[Andrew Kuchling]
> Indeed; I think the proxies really obfuscated the code.  Some
> simple parameter checking, though, shouldn't add too much of a
> burden, and will protect users from common mistakes that will
> result in invalid trees.

[Paul Prescod]
> Those checks would slow down the original tree building unless
> we split the interface into "internal" methods that we use
> ourself and "external methods" that do the extra checking. Worth
> the effort and extra code complexity? Maybe...maybe not.

[Jeremy Hylton]
> Could those checks be implemented as assertions?  If so, people who
> care about speed can use "python -O"

[Greg Stein]
> +1 ... that would be the way to do it.

-1.  User input is never trustworthy.  Your and your users' software lives
will be a lot happier if you stick to the rule that an assertion failure
always (always!) announces a bug in the implementation -- assertion failure
is never a user's fault.  This makes assertions *possibly* suitable for
Paul's hypothesized "internal methods", but never for checking that
user-supplied arguments satisfy preconditions.

pinning-the-blame-is-90%-of-debugging-and-"assert"-should-pin-it-
    exactly-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 01:32:50 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 16:32:50 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCIEBFIBAA.tim.one@home.com>
Message-ID: <3A1B1432.E900C248@activestate.com>

Tim Peters wrote:
> 
> ...
> 
> -1.  User input is never trustworthy.  Your and your users' software lives
> will be a lot happier if you stick to the rule that an assertion failure
> always (always!) announces a bug in the implementation -- assertion failure
> is never a user's fault.  

So you prefer

if __debug__ and node.nodeType!=ELEMENT_TYPE: 
    raise TypeError

Unfortunately there's no way to turn that off at "compile time" so you
always incur the __debug__ lookup cost. That would send us back to two
versions of the methods.

Maybe testing would indicate that the performance implications are
minor. If so, I wouldn't mind having the type checks in there.

 Paul Prescod



From tim.one at home.com  Wed Nov 22 02:34:32 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 20:34:32 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B1432.E900C248@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>

[Tim, objects to abusing assertions]

[Paul Prescod]
> So you prefer
>
> if __debug__ and node.nodeType!=ELEMENT_TYPE:
>     raise TypeError

I personally prefer

if node.nodeType != ELEMENT_TYPE:
    raise TypeError

if that is in fact a correct test of whatever user-input precondition it is
you're verifying.  An assert would be appropriate if it were "impossible"
for the test to fail.

> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.

Actually, there is:

if __debug__:
    if node.nodeType != ELEMENT_TYPE:
        raise TypeError

Python produces no code for that block under -O (btw, this is the same
mechanism that makes asserts vanish under -O:  it's __debug__ that's magic,
not asserts).

As a user, though, I don't expect -O to turn off argument verification!
Same as the Python implementation in these respects:  public API functions
*always* check their arguments, while some private API functions check only
in Debug builds (and via the C library's assert() function, as it's a bug in
the implementation if a private API is misused).

do-s-right-thing-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 02:56:35 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 17:56:35 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCKEBKIBAA.tim.one@home.com>
Message-ID: <3A1B27D3.8ECDA690@activestate.com>

Tim Peters wrote:
> 
>...
> 
> As a user, though, I don't expect -O to turn off argument verification!
> Same as the Python implementation in these respects:  public API functions
> *always* check their arguments, while some private API functions check only
> in Debug builds (and via the C library's assert() function, as it's a bug in
> the implementation if a private API is misused).

As a user, I don't expect much argument verification from the Python
library at all! C-level verification makes sense because the alternative
is core dumps. That's not acceptable.

For the rare Python-coded function that DOES do argument verification, I
wouldn't have much of an expectation of the affect of "-O" on it
because, like Jeremy, I hardly ever use -O. So maybe the argument is not
worth having -- if nobody ever uses -O then we should always just
balance safety and performance rather than expecting the user to choose
one or the other.

 Paul Prescod



From tim.one at home.com  Wed Nov 22 04:12:44 2000
From: tim.one at home.com (Tim Peters)
Date: Tue, 21 Nov 2000 22:12:44 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B27D3.8ECDA690@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>

[Paul Prescod]
> As a user, I don't expect much argument verification from the Python
> library at all! C-level verification makes sense because the alternative
> is core dumps. That's not acceptable.

Why not?  I don't see any real difference between a core dump and an
uncaught & unexpected Python exception:  in either case the program didn't
get the job done, and left stuff in an unknown state.  Nothing
supernaturally evil about a core dump in that respect; if one is
unacceptable, so is the other.  In the good old days a core dump often
crashed the OS too, but that's rare even on Windows now.

> For the rare Python-coded function that DOES do argument verification, I
> wouldn't have much of an expectation of the affect of "-O" on it
> because, like Jeremy, I hardly ever use -O. So maybe the argument is not
> worth having -- if nobody ever uses -O then we should always just
> balance safety and performance rather than expecting the user to choose
> one or the other.

I don't care about argument verification except to the extent that it
validates preconditions.  If the preconditions for using a public function
aren't stated, the docs are inadequate (what's the user supposed to do then?
guess?).  If the preconditions aren't verified, then a user error may lead
to an incomprehensible error somewhere in the bowels.  Most library
functions have preconditions of the form "x is a sequence" or "y supports
.sort()", and for such builtin types & operations laziness is usually
tolerable because the specific exception raised by accident (in the absence
of checking and the presence of a bad argument) says "not a sequence" or
"can't be .sort()ed" more or less directly.  If you've got fancier
preconditions, an accidental exception likely makes no sense at all to the
user.

Andrew's claim was "some simple parameter checking, though, shouldn't add
too much of a burden, and will protect users from common mistakes that will
result in invalid trees".  I'm arguing both that validation is valuable to
the user and that "-O" is a rotten way to turn it off (even if people *did*
use it -- and I agree that few ever do).  Without any validation first,
though, an argument about how to turn it off-- or whether there's even a
need to --is at best premature.  If Andrew is wrong (that user mistakes
aren't common, or that simple checking couldn't protect users in a useful
way), I haven't heard anyone say so.

if-not-the-rest-is-just-a-question-of-biting-the-bullet-ly y'rs  - tim




From paulp at ActiveState.com  Wed Nov 22 05:13:22 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Tue, 21 Nov 2000 20:13:22 -0800
Subject: [Python-Dev] Assertions
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <3A1B47E2.E8E57AE0@activestate.com>

Tim Peters wrote:
> 
>...
>
> Why not?  I don't see any real difference between a core dump and an
> uncaught & unexpected Python exception:  in either case the program didn't
> get the job done, and left stuff in an unknown state.  

A core dump would kill Zope, PythonWin, Alice etc. An exception does
not. To me, that's a big difference. Also, Py_Object type checks are
extremely cheap in C code. And once we put in the parameter checks the
user will get an unexpected Python exception. Presumably they are not
building faulty XML trees on purpose!

Anyhow, I am won over despite your unpersuasive argument.

I note that minidom will not always give you an exception for a poorly
formed tree. That means that the programmer may not find her error until
the XML is "out of Python's hands." It should give an exception sooner
or later but not never.

 Paul Prescod



From fdrake at acm.org  Wed Nov 22 06:19:18 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:19:18 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/xml/dom minidom.py,1.13,1.14
In-Reply-To: <20001121145652.C21426@lyra.org>
References: <200011212202.OAA16186@slayer.i.sourceforge.net>
	<20001121145652.C21426@lyra.org>
Message-ID: <14875.22358.807220.890291@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > This "import string" plus numerous uses within minidom seem a bit extraneous
 > since this is part of the Python 2.0 library...

  An identical copy should be shippable as part of PyXML, which needs
to support Python 1.5.2 as well.  Otherwise, a number of the changes
would be slightly different.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 06:33:38 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:33:38 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
	<3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <14875.23218.734329.350940@cj42289-a.reston1.va.home.com>

Paul Prescod writes:
 > I note that minidom will not always give you an exception for a poorly
 > formed tree. That means that the programmer may not find her error until
 > the XML is "out of Python's hands." It should give an exception sooner
 > or later but not never.

  I'd like to mention again that there's also a matter of compliance
with the specification.  The DOM level 1 recommendation includes
specific documentation about the exceptions that are raised in certain
conditions.  Perhaps we should "map" these to more Pythonic
exceptions, and perhaps not, but I think the exceptions should be
raised when the API specification says they will be.  This is an
important aspect of compliance, and the XML community has demonstrated
substantially more interest in standards compliance than the HTML
community ever did; we should reap the benefits and not end up having
Python discarded because the standard implementation isn't compliant.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 06:35:19 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 00:35:19 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
References: <3A1B27D3.8ECDA690@activestate.com>
	<LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
Message-ID: <14875.23319.386269.278252@cj42289-a.reston1.va.home.com>

Tim Peters writes:
 > need to --is at best premature.  If Andrew is wrong (that user mistakes
 > aren't common, or that simple checking couldn't protect users in a useful
 > way), I haven't heard anyone say so.

  Frankly, I found it very helpful to get exceptions when working with
PyDOM.  Some of them annoyed me, but it made me write better code.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From uche.ogbuji at fourthought.com  Wed Nov 22 07:23:14 2000
From: uche.ogbuji at fourthought.com (uche.ogbuji at fourthought.com)
Date: Tue, 21 Nov 2000 23:23:14 -0700
Subject: [Python-Dev] Assertions 
In-Reply-To: Message from Paul Prescod <paulp@ActiveState.com> 
   of "Tue, 21 Nov 2000 16:32:50 PST." <3A1B1432.E900C248@activestate.com> 
Message-ID: <200011220623.XAA13573@localhost.localdomain>

> Tim Peters wrote:
> > 
> > ...
> > 
> > -1.  User input is never trustworthy.  Your and your users' software lives
> > will be a lot happier if you stick to the rule that an assertion failure
> > always (always!) announces a bug in the implementation -- assertion failure
> > is never a user's fault.  
> 
> So you prefer
> 
> if __debug__ and node.nodeType!=ELEMENT_TYPE: 
>     raise TypeError
> 
> Unfortunately there's no way to turn that off at "compile time" so you
> always incur the __debug__ lookup cost. That would send us back to two
> versions of the methods.
> 
> Maybe testing would indicate that the performance implications are
> minor. If so, I wouldn't mind having the type checks in there.

Of course, continuations would be perfect here, as one could set the 
continuation to resume after the check if he wished to bypass it.

Something tells me I should duck now...

not-very-seriously-mind-you'ly yrs

-- 
Uche Ogbuji                               Principal Consultant
uche.ogbuji at fourthought.com               +1 303 583 9900 x 101
Fourthought, Inc.                         http://Fourthought.com 
4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA
Software-engineering, knowledge-management, XML, CORBA, Linux, Python





From gstein at lyra.org  Wed Nov 22 07:33:03 2000
From: gstein at lyra.org (Greg Stein)
Date: Tue, 21 Nov 2000 22:33:03 -0800
Subject: [Python-Dev] Assertions
In-Reply-To: <3A1B47E2.E8E57AE0@activestate.com>; from paulp@ActiveState.com on Tue, Nov 21, 2000 at 08:13:22PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com>
Message-ID: <20001121223302.L21426@lyra.org>

By definition, isn't minidom supposed to be cheap/easy/quick? The quick
answer to a problem?

If somebody wants an exception-conforming DOM implementation with all the
bells and whistles, then they can go elsewhere. If minidom starts getting
loaded up, then it has kind of defeated its purpose, no?

Cheers,
-g

On Tue, Nov 21, 2000 at 08:13:22PM -0800, Paul Prescod wrote:
> Tim Peters wrote:
> > 
> >...
> >
> > Why not?  I don't see any real difference between a core dump and an
> > uncaught & unexpected Python exception:  in either case the program didn't
> > get the job done, and left stuff in an unknown state.  
> 
> A core dump would kill Zope, PythonWin, Alice etc. An exception does
> not. To me, that's a big difference. Also, Py_Object type checks are
> extremely cheap in C code. And once we put in the parameter checks the
> user will get an unexpected Python exception. Presumably they are not
> building faulty XML trees on purpose!
> 
> Anyhow, I am won over despite your unpersuasive argument.
> 
> I note that minidom will not always give you an exception for a poorly
> formed tree. That means that the programmer may not find her error until
> the XML is "out of Python's hands." It should give an exception sooner
> or later but not never.
> 
>  Paul Prescod
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/



From moshez at zadka.site.co.il  Wed Nov 22 18:33:27 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 19:33:27 +0200
Subject: [Python-Dev] PEP Process Inefficient?
Message-ID: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>

The PEP process, while *miles* better then anything I've seen in any
other development process I've ever seen, has some deficiencies.
I'll try to point them out, and to suggest some partial solution:

1. Users are not sure who to post PEP questions/remarks to: Python dev?
   the original author? The Python devver they know best?
2. It is the responsiblity of the PEP author to add open questions/pertinent
   remarks to the PEP.
3. Mail about the PEP which contains important discussion is lost.

Proposal: (still some shaky areas though):

Why not use Squishdot for PEPs? The PEP would be the "article", and
remarks could be comments. I think (but I'm not sure) Squishdot
has the ability to edit/moderate comments, and retroactively change the
article. If not, then my second best hope is some kind of Wiki system,
with self imposed article/comments-like structure.

Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
has inspired a lot of discussion, and very interesting comments. However
for someone who doesn't remember all the e-mails by heart, wading through
the archives is much too time consuming. In addition, personal mail sent to
me is not available for public discussion. 

Problems with proposed solution: no e-mail notification. I think Zwiki
does have an option for e-mail notification, though. Don't know about
Squishdot.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Wed Nov 22 10:56:23 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 10:56:23 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <3A1B9847.1E6205AC@lemburg.com>

Moshe Zadka wrote:
> 
> The PEP process, while *miles* better then anything I've seen in any
> other development process I've ever seen, has some deficiencies.
> I'll try to point them out, and to suggest some partial solution:
> 
> 1. Users are not sure who to post PEP questions/remarks to: Python dev?
>    the original author? The Python devver they know best?

Perhaps the PEP should specify this by e.g. using a header
Comments-To: email-address

The email address could then be made "usable" on the PEP web pages.

> 2. It is the responsiblity of the PEP author to add open questions/pertinent
>    remarks to the PEP.
> 3. Mail about the PEP which contains important discussion is lost.

The original idea was for PEP authors to include the important
parts of those discussions in summary in the PEP -- AFAIK, this
is not done :-(
 
> Proposal: (still some shaky areas though):
> 
> Why not use Squishdot for PEPs? The PEP would be the "article", and
> remarks could be comments. I think (but I'm not sure) Squishdot
> has the ability to edit/moderate comments, and retroactively change the
> article. If not, then my second best hope is some kind of Wiki system,
> with self imposed article/comments-like structure.
> 
> Immediate problem I'm trying to solve: PEP 228, the numeric model PEP,
> has inspired a lot of discussion, and very interesting comments. However
> for someone who doesn't remember all the e-mails by heart, wading through
> the archives is much too time consuming. In addition, personal mail sent to
> me is not available for public discussion.
> 
> Problems with proposed solution: no e-mail notification. I think Zwiki
> does have an option for e-mail notification, though. Don't know about
> Squishdot.

Hmm, wouldn't something like the SF bug manager be the ideal
platform ? Perhaps someone could get the SF guys to add something
like an "idea" manager with similar functionality but different
content to SF ?!

In the meantime, I think PEP discussions ought to taken to the
sig mailing lists using python-dev as fallback solution.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Wed Nov 22 11:27:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 12:27:10 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1B9847.1E6205AC@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>

On Wed, 22 Nov 2000, M.-A. Lemburg wrote:

> The original idea was for PEP authors to include the important
> parts of those discussions in summary in the PEP -- AFAIK, this
> is not done :-(

I'm trying to do this. However, summarising every e-mail sent to me
about this PEP is quite a burden.

> Hmm, wouldn't something like the SF bug manager be the ideal
> platform ? Perhaps someone could get the SF guys to add something
> like an "idea" manager with similar functionality but different
> content to SF ?!

Yeah, that would work to.

> In the meantime, I think PEP discussions ought to taken to the
> sig mailing lists using python-dev as fallback solution.

I think most of the PEPs are not in the charter of any new sig, and
python-dev is invitation only forum. Perhaps having pep-xxx-sig would
solve it too?

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Wed Nov 22 12:16:09 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed, 22 Nov 2000 12:16:09 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial>
Message-ID: <3A1BAAF8.231B3914@lemburg.com>

Moshe Zadka wrote:
> 
> On Wed, 22 Nov 2000, M.-A. Lemburg wrote:
> 
> > The original idea was for PEP authors to include the important
> > parts of those discussions in summary in the PEP -- AFAIK, this
> > is not done :-(
> 
> I'm trying to do this. However, summarising every e-mail sent to me
> about this PEP is quite a burden.
> 
> > Hmm, wouldn't something like the SF bug manager be the ideal
> > platform ? Perhaps someone could get the SF guys to add something
> > like an "idea" manager with similar functionality but different
> > content to SF ?!
> 
> Yeah, that would work to.
> 
> > In the meantime, I think PEP discussions ought to taken to the
> > sig mailing lists using python-dev as fallback solution.
> 
> I think most of the PEPs are not in the charter of any new sig, and
> python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> solve it too?

I don't think people would sign up for each and every PEP...
and you know how hard it is to even move a discussion from
one forum to another -- this either doesn't work out or 
kills discussion.

I think the simplest solution would be something like the
SF bug manager. I'm pretty sure that other projects on SF
have similar problems, so this might be worthwhile dicussing
with the SF folks. The technology and user interface are there,
it would only take a few new tables in the database.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fdrake at users.sourceforge.net  Wed Nov 22 13:51:00 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Wed, 22 Nov 2000 04:51:00 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011221251.EAA16696@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From jeremy at alum.mit.edu  Wed Nov 22 15:50:22 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 22 Nov 2000 09:50:22 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
References: <E13ydlw-00060e-00@darjeeling.zadka.site.co.il>
Message-ID: <14875.56622.102311.641555@bitdiddle.concentric.net>

>>>>> "MZ" == Moshe Zadka <moshez at zadka.site.co.il> writes:

  MZ> The PEP process, while *miles* better then anything I've seen in
  MZ> any other development process I've ever seen, has some

(Not that we've used the PEP process for much of anything yet.
Except, I guess, extended print, which I've grown quite fond of.)

  MZ> deficiencies.  I'll try to point them out, and to suggest some
  MZ> partial solution:

I don't like your partial solution, so I'll try to pick about the
problems with the process <0.8 wink>.  Put another way: I can't
envision Slashdot as a useful design forum, but am interested in
improving the PEP process.  

  MZ> 1. Users are not sure who to post PEP questions/remarks to:
  MZ>    Python dev?  the original author? The Python devver they know
  MZ>    best?

In the absence of a reason to post elsewhere, the comments ought to be
sent to the PEP author.  If the message is intended to provoke wider
discussion, then the user can post it on any relevent forum (and cc
the PEP author).  A Unicode PEP might be discussed on the i18n-sig; a
Web PEP on the python-web-modules list; a change to core Python on
python-dev. 

I don't think there will ever be a rule that says: "Comments on PEP
1812 must be posted to the pep-1812 web forum."  Discussion should
occur in the forum in which it is most relevent.  Perhaps we could all
send our comments to Andrew, and he could write a bi-weekly
pep-comment summary <wink>.

Is the problem that you don't know where to post comments on someone
else's PEP, or that you are having trouble keeping track of
discussions in multiple places?

  MZ> 2. It is the responsiblity of the PEP author to add open
  MZ>    questions/pertinent remarks to the PEP.

What is the deficiency with this approach?  The PEP is supposed to
present a coherent proposal and design for a new language feature.
It is the designer's responsibility to write a good design document;
several people can share responsibility for the design.

Are you saying it's a problem that the PEPs aren't open to
modification by anyone?  (The last thing we need is someone who
doesn't understand what a first-class function is messing with the
scoping PEP <0.2 wink>.)

The designer is responsible for discussing trade-offs and alternatives
to justify her design.  This is where the responsibility to address
questions and comments comes from.

  MZ> 3. Mail about the PEP which contains important discussion is
  MZ>    lost.

How?  I've got all the mail on the nested static scopes PEP.  Is the
problem just the personal burden of keeping track of lots of mail
messages discussing a PEP?

Any discussion that occurs on a mailing list will be preserved in the
mailing list archive.  That's not lost.  So the only problem would be
with discussion that occurs in private email, where everyone deletes
copies of the mail.

I wonder if the key problem is not having an authoritative mail
archive for the PEP.  One part of the Scheme RFI process that we did
not copy was having a mail archive for each document.  Perhaps this
would be sufficient to address your concerns.  If you, the PEP author,
receive a comment by private email, you could bounce it to the
archive.

the-biggest-problem-with-the-PEP-process-is-no-one-writing-PEPs-ly y'rs,
Jeremy



From moshez at math.huji.ac.il  Wed Nov 22 16:12:28 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:12:28 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.56622.102311.641555@bitdiddle.concentric.net>
Message-ID: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>

The key problem, as Jeremy noted, was that there is no archive of
discussion about the PEP. I had a much more long winded and roundabout
way of saying that. Also, a minor related problem is that when the
only address on the PEP points to me (for example), I get all the mail
about the PEP personally -- and I'm loath to bounce it to a public forum.
*Having* a public forum which is *indicated in the PEP* would solve that.

I could phrase that as a point-by-point answer to Jeremy (and originally
did) but I decided that that would be idiotic.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Wed Nov 22 16:21:01 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:21:01 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221657310.14321-100000@sundial>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011221657310.14321-100000@sundial>
Message-ID: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > only address on the PEP points to me (for example), I get all the mail
 > about the PEP personally -- and I'm loath to bounce it to a public forum.
 > *Having* a public forum which is *indicated in the PEP* would solve that.

  Having a public forum isn't quite what's needed (though useful);
much of the time, it's more important to have a public *archive*.  I'm
sure there's a way to set things up a mail handler that maintains an
mbox file with collected mails on a topic, and a pipermail browsable
version of that should be available, and each PEP should point to the
appropriate archive.  (It should also clearly state that relevant
emails may be added to the archive at the recipient's discretion!)
  Using a mail handler for this would allow the handler to be added to
the CC: on the emails and generally collect replies without any
additional work on the part of the PEP author.  The handler could be
CC'd when the PEP is posted so that replies to the posting go there as
well.
  Now we just need someone interested in writing a handler.  ;-{


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at math.huji.ac.il  Wed Nov 22 16:28:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 17:28:56 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>

On Wed, 22 Nov 2000, Fred L. Drake, Jr. wrote:

>   Using a mail handler for this would allow the handler to be added to
> the CC: on the emails and generally collect replies without any
> additional work on the part of the PEP author.  The handler could be
> CC'd when the PEP is posted so that replies to the posting go there as
> well.
>   Now we just need someone interested in writing a handler.  ;-{

Ummmmm....couldn't that be implemented as a mailman list with *no*
subscribers (subscription requests auto-bounce)
I probably missed something?
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From akuchlin at mems-exchange.org  Wed Nov 22 16:32:45 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:32:45 -0500
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>; from gstein@lyra.org on Tue, Nov 21, 2000 at 10:33:03PM -0800
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com> <3A1B47E2.E8E57AE0@activestate.com> <20001121223302.L21426@lyra.org>
Message-ID: <20001122103245.A17172@kronos.cnri.reston.va.us>

On Tue, Nov 21, 2000 at 10:33:03PM -0800, Greg Stein wrote:
>If somebody wants an exception-conforming DOM implementation with all the
>bells and whistles, then they can go elsewhere. If minidom starts getting
>loaded up, then it has kind of defeated its purpose, no?

Checking for the correct children should be quite fast; in PyDOM it
was basically the line "if newnode.type not in
self._LEGAL_CHILDREN_TYPES: raise ...".  I don't know about the other
minidom bug report, but will try to look into it before too long.

--amk

   Some compilers allow a check during execution that subscripts do not
   exceed array dimensions. This is a help, but not sufficient. First,
   many programmers do not use such compilers because "They're not
   efficient." (Presumably, this means that it is vital to get the wrong
   answers quickly.)
 
   Kernighan and Plauger, in _The Elements of Programming Style_




From fredrik at effbot.org  Wed Nov 22 16:49:38 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Wed, 22 Nov 2000 16:49:38 +0100
Subject: [Python-Dev] PEP Process Inefficient?
References: <14875.56622.102311.641555@bitdiddle.concentric.net><Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <012f01c0549b$d39cb320$3c6340d5@hagrid>

Fred L. Drake wrote:
>   Having a public forum isn't quite what's needed (though useful);
> much of the time, it's more important to have a public *archive*.  I'm
> sure there's a way to set things up a mail handler that maintains an
> mbox file with collected mails on a topic, and a pipermail browsable
> version of that should be available, and each PEP should point to the
> appropriate archive.  (It should also clearly state that relevant
> emails may be added to the archive at the recipient's discretion!)

http://www.lfw.org/ping/roundup.html

</F>




From akuchlin at mems-exchange.org  Wed Nov 22 16:34:36 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Nov 2000 10:34:36 -0500
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Wed, Nov 22, 2000 at 10:21:01AM -0500
References: <14875.56622.102311.641555@bitdiddle.concentric.net> <Pine.GSO.4.10.10011221657310.14321-100000@sundial> <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
Message-ID: <20001122103436.B17172@kronos.cnri.reston.va.us>

On Wed, Nov 22, 2000 at 10:21:01AM -0500, Fred L. Drake, Jr. wrote:
>  Using a mail handler for this would allow the handler to be added to
>the CC: on the emails and generally collect replies without any
>additional work on the part of the PEP author.  The handler could be
>CC'd when the PEP is posted so that replies to the posting go there as
>well.

This sounds suspiciously like the nosy lists in ?!ng's Roundup
bug-tracking tool.

--amk



From fdrake at acm.org  Wed Nov 22 16:33:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:33:04 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <Pine.GSO.4.10.10011221727460.14321-100000@sundial>
References: <14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011221727460.14321-100000@sundial>
Message-ID: <14875.59184.806912.900828@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Ummmmm....couldn't that be implemented as a mailman list with *no*
 > subscribers (subscription requests auto-bounce)
 > I probably missed something?

  Definately, but at that point, why disallow subscribers unless
there's already a forum for the PEP?
  I don't object to setting up a mailing list if there isn't one that
fits the bill, but I think Mailman is a fairly heavy way to do the
job, and server resources definately seem scarce these days, so we
don't want to overload what we have!
  (We'd probably also want to add checks that a message hasn't already
been sent along, and deal with forwards a little differently -- we
only want the forwarded message if someone is just archiving a message
received in private email.  Hmmm... this is starting to sound like an
AI problem!)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From fdrake at acm.org  Wed Nov 22 16:35:19 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 10:35:19 -0500 (EST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122103436.B17172@kronos.cnri.reston.va.us>
References: <14875.56622.102311.641555@bitdiddle.concentric.net>
	<Pine.GSO.4.10.10011221657310.14321-100000@sundial>
	<14875.58461.840036.573067@cj42289-a.reston1.va.home.com>
	<20001122103436.B17172@kronos.cnri.reston.va.us>
Message-ID: <14875.59319.351125.5491@cj42289-a.reston1.va.home.com>

Andrew Kuchling writes:
 > This sounds suspiciously like the nosy lists in ?!ng's Roundup
 > bug-tracking tool.

  Yes, this is very suspicious, indeed!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From trentm at ActiveState.com  Wed Nov 22 17:38:51 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:38:51 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <3A1BAAF8.231B3914@lemburg.com>; from mal@lemburg.com on Wed, Nov 22, 2000 at 12:16:09PM +0100
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com>
Message-ID: <20001122083851.H16729@ActiveState.com>

On Wed, Nov 22, 2000 at 12:16:09PM +0100, M . -A . Lemburg wrote:
> Moshe Zadka wrote:
> > > In the meantime, I think PEP discussions ought to taken to the
> > > sig mailing lists using python-dev as fallback solution.
> > 
> > I think most of the PEPs are not in the charter of any new sig, and
> > python-dev is invitation only forum. Perhaps having pep-xxx-sig would
> > solve it too?
> 
> I don't think people would sign up for each and every PEP...
> and you know how hard it is to even move a discussion from
> one forum to another -- this either doesn't work out or 
> kills discussion.
> 
> I think the simplest solution would be something like the
> SF bug manager. I'm pretty sure that other projects on SF
> have similar problems, so this might be worthwhile dicussing
> with the SF folks. The technology and user interface are there,
> it would only take a few new tables in the database.


A Roundup mailing list with a category for each PEP?

Trent

-- 
Trent Mick
TrentM at ActiveState.com



From trentm at ActiveState.com  Wed Nov 22 17:40:40 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 22 Nov 2000 08:40:40 -0800
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>; from trentm@ActiveState.com on Wed, Nov 22, 2000 at 08:38:51AM -0800
References: <Pine.GSO.4.10.10011221225020.28558-100000@sundial> <3A1BAAF8.231B3914@lemburg.com> <20001122083851.H16729@ActiveState.com>
Message-ID: <20001122084040.I16729@ActiveState.com>

On Wed, Nov 22, 2000 at 08:38:51AM -0800, Trent Mick wrote:
> 
> A Roundup mailing list with a category for each PEP?
> 

Or I should just read all the thread first! Y'all are way ahead of me.


Trent

-- 
Trent Mick
TrentM at ActiveState.com



From moshez at math.huji.ac.il  Wed Nov 22 17:41:50 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Wed, 22 Nov 2000 18:41:50 +0200 (IST)
Subject: [Python-Dev] PEP Process Inefficient?
In-Reply-To: <20001122083851.H16729@ActiveState.com>
Message-ID: <Pine.GSO.4.10.10011221840420.19237-100000@sundial>

On Wed, 22 Nov 2000, Trent Mick wrote:

> A Roundup mailing list with a category for each PEP?

OK, when we get over the make-Moshe-feel-stupid-for-not-proposing-roundup
part, just let me know <wink>. Yep, Roundup is perfect for this!

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From barry at digicool.com  Wed Nov 22 20:19:54 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Wed, 22 Nov 2000 14:19:54 -0500
Subject: [Python-Dev] Too much silence
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us>
	<20001120085216.A30685@glacier.fnational.com>
	<14873.46637.76321.845888@bitdiddle.concentric.net>
	<3A19BA92.1A555299@lemburg.com>
	<20001120095445.A30774@glacier.fnational.com>
Message-ID: <14876.7258.137276.784970@anthem.concentric.net>

>>>>> "NS" == Neil Schemenauer <nas at arctrix.com> writes:

    NS> Barry or Jeremy: the formatting of 208 on sourceforge seems to
    NS> be screwy.  Part of the extended print PEP seems to be
    NS> included.

Something's really screwed up here.  The pep-0208.txt file is almost
empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
that I see on the web page.  I just did a "cvs up" and a "cvs log" and
I don't see anything new checked in for pep 208.

Did somebody forget to commit their changes to CVS?
-Barry



From fdrake at acm.org  Wed Nov 22 21:44:59 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 22 Nov 2000 15:44:59 -0500 (EST)
Subject: [Python-Dev] Assertions
In-Reply-To: <20001121223302.L21426@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCEEBMIBAA.tim.one@home.com>
	<3A1B47E2.E8E57AE0@activestate.com>
	<20001121223302.L21426@lyra.org>
Message-ID: <14876.12363.340244.877950@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > By definition, isn't minidom supposed to be cheap/easy/quick? The quick
 > answer to a problem?
 > 
 > If somebody wants an exception-conforming DOM implementation with all the
 > bells and whistles, then they can go elsewhere. If minidom starts getting
 > loaded up, then it has kind of defeated its purpose, no?

  I've started a new discussion about this over in the XML SIG; if
you're interested, please join in on that list.  If you haven't been
following it, you might want to check the archives starting with
yesterday.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From cgw at fnal.gov  Wed Nov 22 22:20:05 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Wed, 22 Nov 2000 15:20:05 -0600 (CST)
Subject: [Python-Dev] more Sourceforge CVS screwiness
Message-ID: <14876.14469.534078.512149@buffalo.fnal.gov>

Haven't seen this for a while - I was unable to build the most current
version of Python from CVS and was about to squawk, when I realized
that the problem was a bogus "config.h" in my dist/src/Include
directory:

shell$ pwd
/usr/local/src/Python-CVS/python/dist/src/Include
shell$ cvs status config.h                                                                             
===================================================================
File: config.h          Status: Up-to-date

   Working revision:    2.1
   Repository revision: 2.1     /cvsroot/python/python/dist/src/Include/Attic/config.h,v
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      (none)


A fresh checkout of the whole tree does not contain this file.  I know
how to fix this (edit CVS/Entries) but it troubles me that old files
from the Attic keep re-appearing when I do updates (without any
sticky tags or anything, I always do "cvs update -dP")

Am I the only person who sees this?  Does Sourceforge just not like me
for some reason?




From moshez at zadka.site.co.il  Thu Nov 23 07:46:51 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Thu, 23 Nov 2000 08:46:51 +0200
Subject: [Python-Dev] Too much silence 
In-Reply-To: Message from barry@digicool.com (Barry A. Warsaw) 
   of "Wed, 22 Nov 2000 14:19:54 EST." <14876.7258.137276.784970@anthem.concentric.net> 
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com>  <14876.7258.137276.784970@anthem.concentric.net> 
Message-ID: <E13yq9k-0007Ci-00@darjeeling.zadka.site.co.il>

> Something's really screwed up here.  The pep-0208.txt file is almost
> empty; it doesn't seem to contain any of the text (MAL's I'm assuming)
> that I see on the web page.  I just did a "cvs up" and a "cvs log" and
> I don't see anything new checked in for pep 208.
> 
> Did somebody forget to commit their changes to CVS?

I did, when you define forget to include "didn't have the time yet
to translate the web page to PEP".
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From thomas at xs4all.net  Wed Nov 22 23:40:57 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 22 Nov 2000 23:40:57 +0100
Subject: [Python-Dev] Too much silence
In-Reply-To: <14876.7258.137276.784970@anthem.concentric.net>; from barry@digicool.com on Wed, Nov 22, 2000 at 02:19:54PM -0500
References: <E13xzUE-0005mS-00@kronos.cnri.reston.va.us> <20001120085216.A30685@glacier.fnational.com> <14873.46637.76321.845888@bitdiddle.concentric.net> <3A19BA92.1A555299@lemburg.com> <20001120095445.A30774@glacier.fnational.com> <14876.7258.137276.784970@anthem.concentric.net>
Message-ID: <20001122234057.H540@xs4all.nl>

On Wed, Nov 22, 2000 at 02:19:54PM -0500, Barry A. Warsaw wrote:

[ The HTML version of PEP 208 is screwed ]

> Did somebody forget to commit their changes to CVS?

-rw-rw-r--   1 nowonder python      25515 Nov 10 09:31 pep-0208.html

Peter-SK wrote these versions to disk, it seems, but it could have been
edited later. Maybe he can check his .txt version to see whether that's
garbled as well ?

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From akuchlin at mems-exchange.org  Thu Nov 23 17:37:22 2000
From: akuchlin at mems-exchange.org (A.M. Kuchling)
Date: Thu, 23 Nov 2000 11:37:22 -0500
Subject: [Python-Dev] Regression testing
Message-ID: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>

This morning I was looking at test_minidom.py, which contains its own
simplistic framework for running a series of regression tests.  Maybe
another 2.1 task could be selecting a standard unit testing framework;
this should mostly be a matter of selecting one, since there are
already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
and the unittest.py from Quixote, and have a vague feeling I'm
forgetting one other module.  No prizes for guessing which one I'd
vote for. :)

--amk




From trentm at ActiveState.com  Thu Nov 23 17:48:30 2000
From: trentm at ActiveState.com (Trent Mick)
Date: Thu, 23 Nov 2000 08:48:30 -0800
Subject: [Python-Dev] Regression testing
In-Reply-To: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>; from akuchlin@mems-exchange.org on Thu, Nov 23, 2000 at 11:37:22AM -0500
References: <200011231637.LAA00819@207-172-111-235.s235.tnt1.ann.va.dialup.rcn.com>
Message-ID: <20001123084830.I31441@ActiveState.com>

On Thu, Nov 23, 2000 at 11:37:22AM -0500, A.M. Kuchling wrote:
> This morning I was looking at test_minidom.py, which contains its own
> simplistic framework for running a series of regression tests.  Maybe
> another 2.1 task could be selecting a standard unit testing framework;
> this should mostly be a matter of selecting one, since there are
> already some candidates.  I'm aware of PyUnit (pyunit.sourceforge.net)
> and the unittest.py from Quixote, and have a vague feeling I'm
> forgetting one other module.  No prizes for guessing which one I'd
> vote for. :)

Sure, if they provide a clear benefit. I.e. who will want to translate
all the existing tests. As well, I have a patch (which, I admit, I haven't
been maintaining or pushing much) to allow direct testing of the C API). I
have no experience with either of the unit testing frameworks you mention.
Will they be able to do that? Presumably yes, using the same hack that I
proposed.

http://sourceforge.net/patch/?func=detailpatch&patch_id=101162&group_id=5470

Trent


-- 
Trent Mick
TrentM at ActiveState.com



From lutz at rmi.net  Sat Nov 25 16:12:56 2000
From: lutz at rmi.net (Mark Lutz)
Date: Sat, 25 Nov 2000 08:12:56 -0700
Subject: [Python-Dev] A house upon the sand
Message-ID: <004401c056f2$34a9ee00$dc73fea9@vaio>

Hi again Guido,

On a totally unrelated topic: has anyone pushed the idea of a
Python language standard lately?  Things seem to be changing too
fast for many developers to keep up, and it seems to me that a
formal standard doc might help ease a few fears I've seen out 
there.  Books used to be a sort of de facto standard, but even
they aren't reliable anymore; and the manuals aren't useful as a 
standard if they are open to arbitrary change every few months.

Frankly, some people in my classes are very concerned by the rapid 
pace of Python change, and I think their fear is justified.  I get 
burned a little almost every time a new Python release rolls out
too.  Most recently, some new book examples that worked in 1.5.2 
this summer no longer work under 2.0 this fall; I understand that
most changes are improvements (and minor), but this is not a great
story to tell.

A prime example: the string module, used in almost every Python 
program ever written by the half-million Python users out there,
has suddenly been marked as deprecated.  I expect that it won't 
really go away, but has anyone considered the impact of even the
suggestion of its deprecation on Python's acceptance?

If using Python requires that programmers invest lots of time 
tracking the whims of python-dev, then Python will become much 
less useful, imo.  Most developers just don't have the extra time
to spare.  A formal standard doc could give us at least a baseline
Python definition that developers could cling to.  Companies need 
to see a solid and reliable foundation.

Unfortunately, I don't have the time or interest in pushing this
idea through to fruition myself.  Do you have any ideas along these 
lines?  Maybe this task belongs in whatever body eventually takes 
over ownership.  I'm copying this to python-dev in the hopes that 
it might trigger some sort of discussion.

Cheers,
--Mark Lutz  (http://rmi.net/~lutz)




From mal at lemburg.com  Sat Nov 25 17:21:06 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 17:21:06 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A1FE6F2.6A612EF9@lemburg.com>

Mark Lutz wrote:
> 
> Frankly, some people in my classes are very concerned by the rapid
> pace of Python change, and I think their fear is justified.  I get
> burned a little almost every time a new Python release rolls out
> too.  Most recently, some new book examples that worked in 1.5.2
> this summer no longer work under 2.0 this fall; I understand that
> most changes are improvements (and minor), but this is not a great
> story to tell.
> 
> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

It is depreciated because string methods provide a better
model of extensibility for future versions. string.py simply
interfaces to these new methods. It won't go away, but using the
methods directly will provide better performance and an
overall better experience... this doesn't mean that Python
programmers can no longer use string.py, but it should hint
them towards using the string methods instead.

Next to come are number methods, AFAICT ;-)
 
> If using Python requires that programmers invest lots of time
> tracking the whims of python-dev, then Python will become much
> less useful, imo.  Most developers just don't have the extra time
> to spare.  A formal standard doc could give us at least a baseline
> Python definition that developers could cling to.  Companies need
> to see a solid and reliable foundation.

The only major incompatbile changes in 2.0 are the .append()
et al. changes and the str(1L) change. These have been
highlighted in the changes paper.
 
> Unfortunately, I don't have the time or interest in pushing this
> idea through to fruition myself.  Do you have any ideas along these
> lines?  Maybe this task belongs in whatever body eventually takes
> over ownership.  I'm copying this to python-dev in the hopes that
> it might trigger some sort of discussion.

I don't see how a Python standard would do any good. Standards
have version numbers as well and change at about the same rate
(e.g. take Unicode 2.0 vs. Unicode 3.0).

Besides, who forces anyone to use 2.0 instead of 1.5.x which
has been around for many years now ?

Sorry, but I sense bureaucracy kreeping into the house...
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Sat Nov 25 17:25:32 2000
From: tismer at tismer.com (Christian Tismer)
Date: Sat, 25 Nov 2000 18:25:32 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <3A1FE7FC.AF14EF85@tismer.com>

(Marc, your clock is wrong by 1 hour :)

"M.-A. Lemburg" wrote:
> 
> Mark Lutz wrote:
...
> > Unfortunately, I don't have the time or interest in pushing this
> > idea through to fruition myself.  Do you have any ideas along these
> > lines?  Maybe this task belongs in whatever body eventually takes
> > over ownership.  I'm copying this to python-dev in the hopes that
> > it might trigger some sort of discussion.
> 
> I don't see how a Python standard would do any good. Standards
> have version numbers as well and change at about the same rate
> (e.g. take Unicode 2.0 vs. Unicode 3.0).
> 
> Besides, who forces anyone to use 2.0 instead of 1.5.x which
> has been around for many years now ?
> 
> Sorry, but I sense bureaucracy kreeping into the house...

Saying things explicitly is Mark's point, IMHO.

I don't see the danger of bureaucracy so much. Instead, setting
a standard is a sign of maturity for a language. Python as it
is, and in former versions, was a per-se standard, set by
the implementation. The Python documentation explains how
things work, but this is a description of the current
implementation, not setting possible ranges, like a standard.
A standard would allow anybody to write a Python implementation
which conforms to it, without knowing all the
details of the given implementation. He would not have to
guess (or ask Guido) what design decisions are final and
considered a part of the language, or just by chance,
ease of implementation or whatever reason might have let to it.
There are many issues where we know how it works
in one implementation and the other, but it is undefined
which behavior is intended, by chance, enforced or forbidden.

Example: Python 2.0 has now a garbage collector. Not so long
ago, a garbage collector seemed out of reach, and before
Guido sketched a GC by himself, I even had the impression
that a gc was out of question. (should have borrowed the
time machine :)
JPython had garbage collection in the first place, inherited
from Java. So, with Python 1.5.2, 2.0, and JPython, we get
three different implementations, and three different
lifetimes of objects.
It would be helpful if a Python standard would define
how an implementation should do this: Do objects have
to be destructed at all, or which objects have to, or
is it completely up to the implementor to decide?
Even if so, then this would be stated, and people
who write scripts compliant to Python standard 2.0
would know that they can never rely on object destruction,
for instance.

Other Example:
There are certain design decisions in the object layout,
which are exposed to the user, like __dict__ for many
objects. It is not clear if this is a language feature
or an implementation detail. The current user practice
enforces us to support __dict__, I think, although
very different decision would be drawn if someone
was designing a true compiler for Python. Does it have
to exist, and if so, does it have to be a dictionary?
Statements like "it happens to be a dict, but anything
supported by getattr() would do, but it must be there"
would be an example statement, most helpful for alternative
implementations, which still fulfill the standard.

better-explicit-than-implicit - ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From mal at lemburg.com  Sat Nov 25 18:59:10 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sat, 25 Nov 2000 18:59:10 +0100
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com>
Message-ID: <3A1FFDEE.E2321B40@lemburg.com>

Christian Tismer wrote:
> 
> > Mark Lutz wrote:
> ...
> > > Unfortunately, I don't have the time or interest in pushing this
> > > idea through to fruition myself.  Do you have any ideas along these
> > > lines?  Maybe this task belongs in whatever body eventually takes
> > > over ownership.  I'm copying this to python-dev in the hopes that
> > > it might trigger some sort of discussion.
> >
> > I don't see how a Python standard would do any good. Standards
> > have version numbers as well and change at about the same rate
> > (e.g. take Unicode 2.0 vs. Unicode 3.0).
> >
> > Besides, who forces anyone to use 2.0 instead of 1.5.x which
> > has been around for many years now ?
> >
> > Sorry, but I sense bureaucracy kreeping into the house...
> 
> Saying things explicitly is Mark's point, IMHO.
>
> I don't see the danger of bureaucracy so much. Instead, setting
> a standard is a sign of maturity for a language. Python as it
> is, and in former versions, was a per-se standard, set by
> the implementation. The Python documentation explains how
> things work, but this is a description of the current
> implementation, not setting possible ranges, like a standard.
> A standard would allow anybody to write a Python implementation
> which conforms to it, without knowing all the
> details of the given implementation. He would not have to
> guess (or ask Guido) what design decisions are final and
> considered a part of the language, or just by chance,
> ease of implementation or whatever reason might have let to it.
> There are many issues where we know how it works
> in one implementation and the other, but it is undefined
> which behavior is intended, by chance, enforced or forbidden.
> 
> [GC Example, __dict__ example]

Ok. Maybe I misunderstood Mark... if what he wants is a language
definition like e.g. the one for Java and its VM then I agree.
This would be a good thing. OTOH, I don't see much of a point
in giving the user a feeling of fixed standards when we are 
constantly talking about ways to break code ;-) (e.g. take the
Py3K upgrade path PEP).

We could have a language definition for say the 2.0 version
series, but the definition would have to be extended for
the 3.0 series (possibly even within the 2.0 series). It would
gives us a feeling of what "Python" stands for and also
simplify the task of writing different Python implementations,
but nothing much else... certainly not the warm fuzzy feeling
which Mark probably had in mind.

I think the overall impression of Python 2.0 being a moving target
is a bit overrated: there aren't really all that many changes
which affect existing code, so a Python 1.5.2 introduction will
still remain mostly valid even when the students use Python 2.0.

It is true, that Python 2.0 offers a lot more features than
1.5.2 and that in some contexts (e.g. database interfacing
on Windows) these new features have made their way into third
party extensions rather fast (e.g. switching from strings to
Unicode). The visible core language hasn't changed much though...
and this is goodness.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From thomas at xs4all.net  Sat Nov 25 20:45:10 2000
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 25 Nov 2000 20:45:10 +0100
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <3A1FE6F2.6A612EF9@lemburg.com>; from mal@lemburg.com on Sat, Nov 25, 2000 at 05:21:06PM +0100
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com>
Message-ID: <20001125204509.I540@xs4all.nl>

On Sat, Nov 25, 2000 at 05:21:06PM +0100, M.-A. Lemburg wrote:
> Mark Lutz wrote:

> > A prime example: the string module, used in almost every Python
> > program ever written by the half-million Python users out there,
> > has suddenly been marked as deprecated.  I expect that it won't
> > really go away, but has anyone considered the impact of even the
> > suggestion of its deprecation on Python's acceptance?

> It is depreciated because string methods provide a better
> model of extensibility for future versions. string.py simply
> interfaces to these new methods. It won't go away, but using the
> methods directly will provide better performance and an
> overall better experience...

Unfortunately, the term 'deprecated' suggests to most people, including me,
that it will be removed in future versions. I'm with Mark here; marking it
deprecated causes a lot of confusions, IMHO.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!



From tismer at tismer.com  Sun Nov 26 12:35:12 2000
From: tismer at tismer.com (Christian Tismer)
Date: Sun, 26 Nov 2000 13:35:12 +0200
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio> <3A1FE6F2.6A612EF9@lemburg.com> <3A1FE7FC.AF14EF85@tismer.com> <3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <3A20F570.6FCD09F2@tismer.com>


"M.-A. Lemburg" wrote:
> 
> Christian Tismer wrote:
> >
[favorizing a standard]
> > > Mark Lutz wrote:
[standard, to avoid confusion]
...
> Ok. Maybe I misunderstood Mark... if what he wants is a language
> definition like e.g. the one for Java and its VM then I agree.
> This would be a good thing. OTOH, I don't see much of a point
> in giving the user a feeling of fixed standards when we are
> constantly talking about ways to break code ;-) (e.g. take the
> Py3K upgrade path PEP).

Whatever other feelings come into play, a standard would
be a good thing, IMHO.

Aside, the PERL world is thinking into this direction as well.
See this rosler article (also interesting concerning Python:)
http://www.perl.com/pub/2000/06/rosler.html

Since Python is so much cleaner and smaller (still?), this
is a chance to be the winder here.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From tim.one at home.com  Sun Nov 26 23:39:42 2000
From: tim.one at home.com (Tim Peters)
Date: Sun, 26 Nov 2000 17:39:42 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>

On the deprecation of the string module:  where did this idea come from?
I've never seen anything saying that the string module is deprecated.  The
2.0 docs say that the three specific functions string.{atoi, atol, atof} are
deprecated as of 2.0, but that's it.

On the meaning of deprecation:  yes, it means they'll go away someday.

On gc semantics:  Section 3.1 ("Objects, values and types") of the Lang Ref
sez:

    Objects are never explicitly destroyed; however, when they become
    unreachable they may be garbage-collected. An implementation is
    allowed to postpone garbage collection or omit it altogether -- it is
    a matter of implementation quality how garbage collection is
    implemented, as long as no objects are collected that are still
    reachable. (Implementation note: the current implementation uses a
    reference-counting scheme which collects most objects as soon as
    they become unreachable, but never collects garbage containing
    circular references.)

If Python had a standard, the only thing that would change is that the
obsolete parenthetical comment would be removed.

On formal standardization:  not likely.  An ISO/ANSI std takes a minimum of
person-decades of work, and Guido would be unable to make the time even to
review committee documents.  IOW, he won't cooperate, because he can't, and
that's that.  Propose changes to the Lang Ref instead (for example, Guido's
intent was that binary operators of equal precedence be evaluated strictly
left to right in the absence of parens, but the Lang Ref never got around to
saying so).

On the pace of change:  by all objective measures, Python change ended the
day 2.0 was released <0.9 wink>.

the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim




From martin at loewis.home.cs.tu-berlin.de  Sun Nov 26 23:53:54 2000
From: martin at loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 26 Nov 2000 23:53:54 +0100
Subject: [Python-Dev] A house upon the sand
Message-ID: <200011262253.XAA01869@loewis.home.cs.tu-berlin.de>

> the-lang-ref-is-the-only-std-you're-likely-to-get-ly y'rs  - tim

Since Java has been cited as an example of a language that has a
"language definition" - it is just a language reference book, either.
Sun did a number of attempts to make Java an Internation Standard, and
they withdrew when they finally understood what it means.

For Python, it would most likely mean that the only implementations
don't conform to the standard.

Regards,
Martin



From mwh21 at cam.ac.uk  Mon Nov 27 00:22:15 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 26 Nov 2000 23:22:15 +0000
Subject: [Python-Dev] A house upon the sand
In-Reply-To: "Tim Peters"'s message of "Sun, 26 Nov 2000 17:39:42 -0500"
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>
Message-ID: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>

"Tim Peters" <tim.one at home.com> writes:

> On the deprecation of the string module:  where did this idea come from?
> I've never seen anything saying that the string module is deprecated.

I thought this, and went looking.  I found on
http://www.python.org/1.6/, about four fifths of the way down:

Changed Modules

    string - most of this module is deprecated now that strings have
    methods. This no longer uses the built-in strop module, but takes
    advantage of the new string methods to provide transparent support
    for both Unicode and ordinary strings.

I hope (and believe) this is Wrong.

http://www.python.org/2.0/new-python.html says:

    The old string module is still around for backwards compatibility,
    but it mostly acts as a front-end to the new string methods.

which is IMHO better.

Cheers,
M.

-- 
  Every day I send overnight packages filled with rabid weasels to
  people who use frames for no good reason.
                             -- The Usenet Oracle, Oracularity #1017-1




From greg at cosc.canterbury.ac.nz  Mon Nov 27 00:56:22 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 27 Nov 2000 12:56:22 +1300 (NZDT)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>

Michael Hudson <mwh21 at cam.ac.uk>:

> http://www.python.org/2.0/new-python.html says:
>
>    The old string module is still around for backwards
>    compatibility,

This still suggests that continuing to use it is frowned
upon, though.

I think there are still legitimate reasons for using some
parts of the string module. For example, if you're one of
those stubborn people who refuse to accept that ",".join(foo)
is a better way of writing string.join(foo,",").

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fdrake at acm.org  Mon Nov 27 02:00:48 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 26 Nov 2000 20:00:48 -0500 (EST)
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>

Greg Ewing writes:
 > This still suggests that continuing to use it is frowned
 > upon, though.

  Backward compatible code is still being written, certainly, and not
everything is available as a method (just try ''.letters! ;).

 > I think there are still legitimate reasons for using some
 > parts of the string module. For example, if you're one of
 > those stubborn people who refuse to accept that ",".join(foo)
 > is a better way of writing string.join(foo,",").

  There will never be an excuse for that!  .join() should never have
been added as a method!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From cgw at fnal.gov  Mon Nov 27 02:08:15 2000
From: cgw at fnal.gov (Charles G Waldman)
Date: Sun, 26 Nov 2000 19:08:15 -0600 (CST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
	<14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
Message-ID: <14881.46079.317598.101585@buffalo.fnal.gov>

Fred L. Drake, Jr. writes:

 >   There will never be an excuse for that!  .join() should never have
 > been added as a method!

On the other hand it is not clear to me why "capwords" and "zfill" did
not become methods of string objects. 




From fredrik at effbot.org  Mon Nov 27 02:30:10 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 02:30:10 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov>
Message-ID: <005601c05811$95f391f0$3c6340d5@hagrid>

Charles wrote:
> On the other hand it is not clear to me why "capwords" and "zfill" did
> not become methods of string objects. 

fwiw, they were both present in my original unicode
implementation:

>>> from unicode import unicode
>>> a = unicode("hello world")
>>> a.capwords()
'Hello World'
>>> a.zfill(20)
'000000000hello world'

</F>




From fdrake at acm.org  Mon Nov 27 07:00:01 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 01:00:01 -0500 (EST)
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14881.46079.317598.101585@buffalo.fnal.gov>
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
	<14881.45632.183060.286728@cj42289-a.reston1.va.home.com>
	<14881.46079.317598.101585@buffalo.fnal.gov>
	<005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <14881.63585.563272.169607@cj42289-a.reston1.va.home.com>

Charles G Waldman writes:
 > On the other hand it is not clear to me why "capwords" and "zfill" did
 > not become methods of string objects. 

Fredrik Lundh writes:
 > fwiw, they were both present in my original unicode
 > implementation:

  Interesting.  I'd have expected capwords() on strings.  I wonder why
/F's implementation wasn't retained?  As for zfill(), I can imagine no
one thought it was sufficiently useful to keep around.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at zadka.site.co.il  Mon Nov 27 16:06:42 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 17:06:42 +0200
Subject: [Python-Dev] PEP-0218
Message-ID: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>

Well, another proof of the bugs in the PEP process -- my remarks were
lost, so I'll send them here.

Let me note that almost everything Greg Wilson wants to do can be done
via a Python class implementing a set using a dictionary mapping to None.
Almost?

* No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
* Convertors: if we want list/tuple to have a semblance of efficiency,
  we'll need to cache the element list as a list when accessed by
  index.
* Two different constructors: set() for building from sequence, Set()
  for building from elements. Might be confusing.
* Only possible elements of a set are immutables. OTOH, I'm not sure
  how Greg intends to implement his sets if these sets are allowed 
  to contain mutable elements. 


-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From esr at thyrsus.com  Mon Nov 27 08:30:33 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 02:30:33 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>; from moshez@zadka.site.co.il on Mon, Nov 27, 2000 at 05:06:42PM +0200
References: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <20001127023033.A7616@thyrsus.com>

Moshe Zadka <moshez at zadka.site.co.il>:
> Well, another proof of the bugs in the PEP process -- my remarks were
> lost, so I'll send them here.
> 
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.

I agree.  I wrote a full-featured set library long ago.  All it's
waiting for is rich comparisons.  Wait...are those in 2.0?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

It will be of little avail to the people, that the laws are made by
men of their own choice, if the laws be so voluminous that they cannot
be read, or so incoherent that they cannot be understood; if they be
repealed or revised before they are promulgated, or undergo such
incessant changes that no man, who knows what the law is to-day, can
guess what it will be to-morrow. Law is defined to be a rule of
action; but how can that be a rule, which is little known, and less
fixed?
	-- James Madison, Federalist Papers 62



From moshez at math.huji.ac.il  Mon Nov 27 08:40:47 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 09:40:47 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127023033.A7616@thyrsus.com>
Message-ID: <Pine.GSO.4.10.10011270938120.11335-100000@sundial>

On Mon, 27 Nov 2000, Eric S. Raymond wrote:

> I agree.  I wrote a full-featured set library long ago.  All it's
> waiting for is rich comparisons.  Wait...are those in 2.0?

Nope. I'm not even sure if they're going to be in 2.1. DA?
Anyway, can you at least give the URL? I wrote something like that
too (is this another Python ritual), and I'd be happy to try and
integrate them for 2.1.

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov 27 10:32:46 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 10:32:46 +0100
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>
Message-ID: <3A222A3E.D53D6D54@lemburg.com>

Fredrik Lundh wrote:
> 
> Charles wrote:
> > On the other hand it is not clear to me why "capwords" and "zfill" did
> > not become methods of string objects.
> 
> fwiw, they were both present in my original unicode
> implementation:
> 
> >>> from unicode import unicode
> >>> a = unicode("hello world")
> >>> a.capwords()
> 'Hello World'
> >>> a.zfill(20)
> '000000000hello world'

.zfill() is implemented for both strings and Unicode, .capwords()
only for Unicode. Both are disabled, though. I talked with
Guido about these methods and we decided to leave those two
methods disabled in the implementation. They just don't provide
much extra generally useful functionality.

s.capwords() can be emulated with ' '.join(s.capitalize().split()),
BTW.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gvwilson at nevex.com  Mon Nov 27 14:05:23 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:05:23 -0500 (EST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com>

(Note: PEP 0218 is very much a work in progress --- I just wanted to get
some preliminary thoughts down so that my conscience wouldn't nag me quite
so much... :-)

> Well, another proof of the bugs in the PEP process -- my remarks were lost,
> so I'll send them here.

Where did you send your comments?

Thanks,
Greg




From moshez at zadka.site.co.il  Mon Nov 27 22:24:42 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 23:24:42 +0200
Subject: [Python-Dev] PEP-0218 
In-Reply-To: Message from Greg Wilson <gvwilson@nevex.com> 
   of "Mon, 27 Nov 2000 08:05:23 EST." <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com> 
References: <Pine.LNX.4.10.10011270804330.24381-100000@akbar.nevex.com> 
Message-ID: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>

> > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > so I'll send them here.
> 
> Where did you send your comments?

To you personally?
-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From gvwilson at nevex.com  Mon Nov 27 14:30:13 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 08:30:13 -0500 (EST)
Subject: [Python-Dev] PEP-0218 
In-Reply-To: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
Message-ID: <Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>

> > > Well, another proof of the bugs in the PEP process -- my remarks were lost,
> > > so I'll send them here.
> > Where did you send your comments?
> To you personally?

Yes, I got those late last week.  Today's election day here in Canada, though,
so the weekend was a write-off.  I'll get to them as soon as I can...

Thanks,

Greg

p.s. emphasis on "day" --- as in, one 24 hour period... :-)




From barry at digicool.com  Mon Nov 27 14:46:14 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 08:46:14 -0500
Subject: [Python-Dev] PEP-0218 
References: <20001127212442.A1338A9BB@darjeeling.zadka.site.co.il>
	<Pine.LNX.4.10.10011270818300.24612-100000@akbar.nevex.com>
Message-ID: <14882.26022.860967.54489@anthem.concentric.net>

>>>>> "GW" == Greg Wilson <gvwilson at nevex.com> writes:

    GW> p.s. emphasis on "day" --- as in, one 24 hour period... :-)

Recount, recount!  But tell ya what.  If you have as much problems as
we had, we'll trade you Florida for a province or territory of your
choice.

-Barry



From barry at digicool.com  Mon Nov 27 15:46:40 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:46:40 -0500
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
	<3A1FE6F2.6A612EF9@lemburg.com>
	<3A1FE7FC.AF14EF85@tismer.com>
	<3A1FFDEE.E2321B40@lemburg.com>
Message-ID: <14882.29648.538582.454004@anthem.concentric.net>

>>>>> "M" == M  <mal at lemburg.com> writes:

    M> I think the overall impression of Python 2.0 being a moving
    M> target is a bit overrated: there aren't really all that many
    M> changes which affect existing code, so a Python 1.5.2
    M> introduction will still remain mostly valid even when the
    M> students use Python 2.0.

    M> It is true, that Python 2.0 offers a lot more features than
    M> 1.5.2 and that in some contexts (e.g. database interfacing on
    M> Windows) these new features have made their way into third
    M> party extensions rather fast (e.g. switching from strings to
    M> Unicode). The visible core language hasn't changed much
    M> though...  and this is goodness.

I agree completely.  What I think freaks people out is all the new
features that cropped up in 2.0.  People think they have to learn all
that new fancy stuff like string methods, list comprehensions, and
extended print.  For those who watch python-dev from the sidelines,
pie-in-the-sky discussions on things like continuations freak people
out even more.  There's too much brain-blowing new things being added
to my nice little language!

But it doesn't have to be that way.  As others have pointed out,
almost all the code that worked for 1.5.2 works for 2.0 and 1.5.2 was
released what? 18 months ago?  That's a lot of stability these days.

I understand that adding new stuff to the language has an impact on
slow-update media like books, and that should not be ignored.  But
people don't have to learn (or be taught) the new features to be just
as productive in Python 2.0 as they were in 1.5.2.  They can learn the
new stuff once they've grasped and are comfortable with the old.

2.0 introduced a lot of new stuff, as much because of social reasons
and pressures as for technical benefits.  On the flip side, we've had
messages in this forum decrying the recent dearth of messages.  I
suspect that's partly a reaction to all the changes in 2.0, letting
the new things settle in.  I can't imagine that 2.1 will have nearly
as many new features.

-Barry



From barry at digicool.com  Mon Nov 27 15:52:19 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 09:52:19 -0500
Subject: [Python-Dev] A house upon the sand
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk>
	<200011262356.MAA03151@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.29987.132557.43415@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg at cosc.canterbury.ac.nz> writes:

    GE> I think there are still legitimate reasons for using some
    GE> parts of the string module. For example, if you're one of
    GE> those stubborn people who refuse to accept that ",".join(foo)
    GE> is a better way of writing string.join(foo,",").

Actually, an even better way is

COMMA = ','
COMMA.join(foo)

To me, that is substantially easier to read than either of the above
two alternatives.

>>>>> "Fred" == Fred L Drake, Jr <fdrake at acm.org> writes:

    Fred>   There will never be an excuse for that!  .join() should
    Fred> never have been added as a method!

Of course, I completely disagree.  However, I have noticed that I
often define string constants like

SPACE = ' '
EMPTY = ''
NL = '\n'

just so I can write code like

NL.join(headers)
SPACE.join(names)
EMPTY.join(lines)

I doubt it's worth it, but maybe having a standard module called
stringconstants.py with some of the more obvious choices would make
things better?

toowtdi-my-foot-ly y'rs,
-Barry



From guido at python.org  Mon Nov 27 16:21:48 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:21:48 -0500
Subject: string methods (was Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 10:32:46 +0100."
             <3A222A3E.D53D6D54@lemburg.com> 
References: <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk><200011262356.MAA03151@s454.cosc.canterbury.ac.nz><14881.45632.183060.286728@cj42289-a.reston1.va.home.com> <14881.46079.317598.101585@buffalo.fnal.gov> <005601c05811$95f391f0$3c6340d5@hagrid>  
            <3A222A3E.D53D6D54@lemburg.com> 
Message-ID: <200011271521.KAA29437@cj20424-a.reston1.va.home.com>

> .zfill() is implemented for both strings and Unicode, .capwords()
> only for Unicode. Both are disabled, though. I talked with
> Guido about these methods and we decided to leave those two
> methods disabled in the implementation. They just don't provide
> much extra generally useful functionality.

Exactly.  There's a price to pay for adding methods, and I think these
two are below the threshold.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Mon Nov 27 16:31:33 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 10:31:33 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: Your message of "Mon, 27 Nov 2000 09:40:47 +0200."
             <Pine.GSO.4.10.10011270938120.11335-100000@sundial> 
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial> 
Message-ID: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>

[Moshe, on rich comparisons]
> Nope. I'm not even sure if they're going to be in 2.1. DA?

Yes, they should go into 2.1.  Moshe, you are listed as co-author of
PEP 207.  What's up?  Also, I thought you were going to co-author PEP
208 (Reworking the Coercion Model).  What's up with that?  Both are
empty!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gvwilson at nevex.com  Mon Nov 27 17:29:58 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Mon, 27 Nov 2000 11:29:58 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>

Hi, folks.  Is there a standard or usual way to provide version
information inside a Python module or class?  Special tokens in
the doc string, a method like "getVersion"...?

Thanks,
Greg



From fdrake at acm.org  Mon Nov 27 17:33:04 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 27 Nov 2000 11:33:04 -0500 (EST)
Subject: [Python-Dev] versioning standards?
In-Reply-To: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
References: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
	<NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <14882.36032.794185.285527@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Hi, folks.  Is there a standard or usual way to provide version
 > information inside a Python module or class?  Special tokens in
 > the doc string, a method like "getVersion"...?

  Modules sometimes offer a "__version__" attribute, which I've also
seen called "version" or "VERSION" in strange cases (xmllib and (I
think) PIL).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Mon Nov 27 17:42:02 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 17:42:02 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>
Message-ID: <3A228EDA.56050B52@lemburg.com>

Greg Wilson wrote:
> 
> Hi, folks.  Is there a standard or usual way to provide version
> information inside a Python module or class?  Special tokens in
> the doc string, a method like "getVersion"...?

AFAIK, __version__ with a string value is in common usage both
in modules and classes.

BTW, while we're at it: with the growing number of dependencies
between modules, packages and the Python lib version... how about
creating a standard to enable versioning in module/package imports ?

It would have to meet (at least) these requirements:

* import of a specific version

* alias of the unversioned name to the most recent version
  available

Thoughts ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Mon Nov 27 19:11:54 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 13:11:54 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: Your message of "Mon, 27 Nov 2000 17:42:02 +0100."
             <3A228EDA.56050B52@lemburg.com> 
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>  
            <3A228EDA.56050B52@lemburg.com> 
Message-ID: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>

> AFAIK, __version__ with a string value is in common usage both
> in modules and classes.

Correct.  This was agreed upon as a standard long ago.  It's probably
not documented anywhere as such.

> BTW, while we're at it: with the growing number of dependencies
> between modules, packages and the Python lib version... how about
> creating a standard to enable versioning in module/package imports ?
> 
> It would have to meet (at least) these requirements:
> 
> * import of a specific version
> 
> * alias of the unversioned name to the most recent version
>   available
> 
> Thoughts ?

This is a major rathole.  I don't know of any other language or system
that has solved this in a satisfactory way.  Most shared library
implementations have had to deal with this, but it's generally a very
painful manual process to ensure compatibility.  I would rather stay
out of trying to solve this for Python in a generic way -- when
specific package develop incompatible versions, it's up to the package
authors to come up with a backwards compatibility strategy.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Mon Nov 27 19:23:35 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 19:23:35 +0100
Subject: [Python-Dev] versioning standards?
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com>  
	            <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <3A22A6A7.B06F603E@lemburg.com>

Guido van Rossum wrote:
> 
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

Probably should be... along with some other __special_name__
which are in use, e.g. __copyright__ or __author__.

I think it would be a good idea to add these attribute names
as virtually "reserved" attribute names.
 
> > BTW, while we're at it: with the growing number of dependencies
> > between modules, packages and the Python lib version... how about
> > creating a standard to enable versioning in module/package imports ?
> >
> > It would have to meet (at least) these requirements:
> >
> > * import of a specific version
> >
> > * alias of the unversioned name to the most recent version
> >   available
> >
> > Thoughts ?
> 
> This is a major rathole.  I don't know of any other language or system
> that has solved this in a satisfactory way.  Most shared library
> implementations have had to deal with this, but it's generally a very
> painful manual process to ensure compatibility.  I would rather stay
> out of trying to solve this for Python in a generic way -- when
> specific package develop incompatible versions, it's up to the package
> authors to come up with a backwards compatibility strategy.

Well, I think it should at least be possible to install
multiple versions of the same packages in some way which
makes it possible for other programs to choose which version
to take.

With packages and the default import mechanism this is possible
using some caching and __path__ trickery -- I think Pmw does
this. The new "from ... import ... as" also provides ways
for this:

from mx import DateTime160 as DateTime

The problem with the latter approach is that pickles will use the
versioned name and thus won't port easily to later versions. But
you're probably right: only the package itself will know if it's
ok to use the newer version or not.

Would still be nice if we had some kind of documented naming
scheme for versioned modules and packages though, so that the
user will at least recognize these as being versioned import
names.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Mon Nov 27 19:50:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Mon, 27 Nov 2000 20:50:10 +0200 (IST)
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011272048300.10126-100000@sundial>

On Mon, 27 Nov 2000, Guido van Rossum wrote:

> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1.  Moshe, you are listed as co-author of
> PEP 207.  What's up?  Also, I thought you were going to co-author PEP
> 208 (Reworking the Coercion Model).  What's up with that?  Both are
> empty!

Ooops, the PEP-0207 thing is a mistake -- I meant to change 0208.
No time now, I'll change 0208 back to DA later tonight. (Sorry 
for being so israelo-centric <wink>). The 0208 PEP should get
done before the weekend. (Your weekend, not mine).

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Mon Nov 27 20:31:43 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 20:31:43 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
Message-ID: <3A22B69F.D3C78D5C@lemburg.com>

> Date: 2000-Nov-27 10:12
> By: mwh
> 
> Comment:
> I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...

Hehe, that was indeed a trivial patch. What was that about trees
in a forest...

I still think that the PySequence_Fast_GETITEM() macro should at
least include a fall-back type check which causes some exception in
case the used object was not "fixed" using PySequence_Fast() (which
I only noticed in string_join() just now).

Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
used more often :-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From paulp at ActiveState.com  Mon Nov 27 21:14:06 2000
From: paulp at ActiveState.com (Paul Prescod)
Date: Mon, 27 Nov 2000 12:14:06 -0800
Subject: [Python-Dev] A house upon the sand
References: <004401c056f2$34a9ee00$dc73fea9@vaio>
Message-ID: <3A22C08E.614A1A0@ActiveState.com>

Mark Lutz wrote:
> 

I read several different issues into your message:

 * standards doc
 * speed of change
 * deprecations
 * backwards incompatibilities

Standards doc: we have one but it is out of date. I think that it would
be great if someone has the bandwidth to improve it but I'm not going to
push Guido very hard to do that because there's a lot of more high
priority stuff I'd like to see him work on.

Speed of change: it wasn't really so radical if you think in terms of
how long these suggestions have been "queued". I would not like it if
Python kept up the same pace of language additions but I don't think
that's going to happen.

Deprecation: deprecations are a good thing. They are the only way that
Python will ever drop old features. If Python never drops old features
then it can only ever grow. If it only ever grows then it will only
become more and more difficult to learn. I don't think I have to justify
the importance of learnability...

Backwards Incompatibilities: I feel strongly that we need to be more
conservative about code breakage. Python 2 was not backwards compatible
enough. It should have deprecated the code and used warnings instead.
Here's my suggested solution for the future:

  http://python.sourceforge.net/peps/pep-0005.html

> A prime example: the string module, used in almost every Python
> program ever written by the half-million Python users out there,
> has suddenly been marked as deprecated.  I expect that it won't
> really go away, but has anyone considered the impact of even the
> suggestion of its deprecation on Python's acceptance?

Sometimes we make mistakes. Making mistakes is part of learning. String
functions should have been methods. We had to correct that mistake. What
other mechanism do we have for correcting mistakes other than
deprecation?

 Paul Prescod



From tim.one at home.com  Mon Nov 27 21:30:04 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 27 Nov 2000 15:30:04 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>

[Barry A. Warsaw]
> ...
> I have noticed that I often define string constants like
>
> SPACE = ' '
> EMPTY = ''
> NL = '\n'
>
> just so I can write code like
>
> NL.join(headers)
> SPACE.join(names)
> EMPTY.join(lines)
> 
> I doubt it's worth it, but maybe having a standard module called
> stringconstants.py with some of the more obvious choices would make
> things better?

-0.  Then I'd expect to see

    from stringconstants import *

at least once, which is at least once too often.  Sick trick:

SPACE, TAB, NL = " \t\n"

sequence-unpacking-is-more-general-than-you-think<wink>-ly y'rs
    - tim




From mwh21 at cam.ac.uk  Mon Nov 27 21:30:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:30:31 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 20:31:43 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com>
Message-ID: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> > Date: 2000-Nov-27 10:12
> > By: mwh
> > 
> > Comment:
> > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> 
> Hehe, that was indeed a trivial patch. What was that about trees
> in a forest...

The way I found it was perhaps instructive.  I was looking at the
function, and thought "that's a bit complicated" so I rewrote it (My
rewrite also seems to be bit quicker so I'll upload it as soon as make
test has finished[*]).  In the course of rewriting it, I saw the line
my patch touched and went "duh!".

> I still think that the PySequence_Fast_GETITEM() macro should at
> least include a fall-back type check which causes some exception in
> case the used object was not "fixed" using PySequence_Fast() (which
> I only noticed in string_join() just now).

It's hard to see how; you're not going to check each invocation of
PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
that PySequence_Fast should raise an exception on being passed a
string or Unicode object... but probably not.

> Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> used more often :-)

Yes.  But they're pretty new, aren't they?  I find them a bit
unsatisfactory that it's not possible to hoist the type check out of
the inner loop.  Still, it seems my PII's branch predictor nails that
one... (i.e. changing it so that it didn't check inside the loop made
naff-all difference to the running time).

Cheers,
M.

[*] which reminds me: test_unicodedata is failing for me at the
    moment.  Anyone else seeing this?  It looks like a python
    regrtest.py -g is all that's needed...

-- 
7. It is easier to write an incorrect program than understand a
   correct one.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From mwh21 at cam.ac.uk  Mon Nov 27 21:38:35 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 27 Nov 2000 20:38:35 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: Michael Hudson's message of "27 Nov 2000 20:30:31 +0000"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>

Michael Hudson <mwh21 at cam.ac.uk> writes:

> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > > 
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > 
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).

It's at

    https://sourceforge.net/patch/index.php?func=detailpatch&patch_id=102549&group_id=5470

I want a way of submitting patches by email.  Is roundup in a usable
state yet?

Cheers,
M.

-- 
  "The future" has arrived but they forgot to update the docs. 
                                        -- R. David Murray, 9 May 2000




From mal at lemburg.com  Mon Nov 27 22:08:41 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon, 27 Nov 2000 22:08:41 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in 
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A22CD59.BC091CAD@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Date: 2000-Nov-27 10:12
> > > By: mwh
> > >
> > > Comment:
> > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> >
> > Hehe, that was indeed a trivial patch. What was that about trees
> > in a forest...
> 
> The way I found it was perhaps instructive.  I was looking at the
> function, and thought "that's a bit complicated" so I rewrote it (My
> rewrite also seems to be bit quicker so I'll upload it as soon as make
> test has finished[*]).  In the course of rewriting it, I saw the line
> my patch touched and went "duh!".

Yeah. The bug must have sneaked in there when the function was
updated to the PySequence_Fast_* implementation.

BTW, could you also add a patch for the test_string.py and
test_unicode.py tests ?

> > I still think that the PySequence_Fast_GETITEM() macro should at
> > least include a fall-back type check which causes some exception in
> > case the used object was not "fixed" using PySequence_Fast() (which
> > I only noticed in string_join() just now).
> 
> It's hard to see how; you're not going to check each invocation of
> PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> that PySequence_Fast should raise an exception on being passed a
> string or Unicode object... but probably not.

Since not using PySequence_Fast() to initialize the protocol,
I'd suggest doing a Py_FatalError() with some explanatory
text which gets printed to stderr -- still better than a
segfault at some later point due to some dangling pointers...

> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they? 

Yep. Fredrik added them quite late in the 2.0 release process.

> I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.  Still, it seems my PII's branch predictor nails that
> one... (i.e. changing it so that it didn't check inside the loop made
> naff-all difference to the running time).

I think Fredrik speculated on having the compiler optimizers
taking care of the check... hmm, it would probably also help
to somehow declare PyTypeObject slots "const" -- is this possible
in a struct definition ?
 
> Cheers,
> M.
> 
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

Is that for the CVS version or the release version ?
 
-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at effbot.org  Mon Nov 27 22:29:25 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:29:25 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <000f01c058b9$1fc03c60$3c6340d5@hagrid>

Michael Hudson wrote:
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).

your patch changes the behaviour of join on non-standard sequences.

if you use join on a long synthetic sequence, your new version may run
out of memory where the old one worked just fine.

not sure that's a good idea.

-0

</F>




From tim.one at home.com  Mon Nov 27 22:18:35 2000
From: tim.one at home.com (Tim Peters)
Date: Mon, 27 Nov 2000 16:18:35 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEIDIBAA.tim.one@home.com>

[Michael Hudson]
> ...
> [*] which reminds me: test_unicodedata is failing for me at the
>     moment.  Anyone else seeing this?  It looks like a python
>     regrtest.py -g is all that's needed...

All tests pass for me under current CVS, Windows.  More info?




From fredrik at effbot.org  Mon Nov 27 22:35:08 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:35:08 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
Message-ID: <000901c058b9$ec78aee0$3c6340d5@hagrid>

> your patch changes the behaviour of join on non-standard sequences.
> 
> if you use join on a long synthetic sequence, your new version may run
> out of memory where the old one worked just fine.

looking again, this was obviously only true for the unicode version
(which uses PySequence_GetItem), not for the 8-bit string version
(which uses the PySequence_Fast API).

maybe we should change the 8-bit implementation instead?

if not, +1.

</F>




From barry at digicool.com  Mon Nov 27 22:30:06 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:30:06 -0500
Subject: [Python-Dev] A house upon the sand
References: <14882.29987.132557.43415@anthem.concentric.net>
	<LNBBLJKPBEHFEDALKOLCIEHOIBAA.tim.one@home.com>
Message-ID: <14882.53854.350972.738517@anthem.concentric.net>

>>>>> "TP" == Tim Peters <tim.one at home.com> writes:

    TP> at least once, which is at least once too often.  Sick trick:

    TP> SPACE, TAB, NL = " \t\n"

Oh, that is perversely delicious!  I love it.

super-cali-fragi-listic-expi-ali-chomp-chomp-ly y'rs,
-Barry



From fredrik at effbot.org  Mon Nov 27 22:41:17 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Mon, 27 Nov 2000 22:41:17 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <002701c058ba$c99ced40$3c6340d5@hagrid>

Michael Hudson wrote:
> > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > used more often :-)
> 
> Yes.  But they're pretty new, aren't they?  I find them a bit
> unsatisfactory that it's not possible to hoist the type check out of
> the inner loop.

if you can figure out a way to do that with C macros,
I'd love to see it...

on the other hand, PySequence_Fast guarantees that the
returned object is either a list or a tuple, so you can easily
move the test outside the loop yourself:

    if (PyTuple_Check(seq))
        for (...)
            item = PyTuple_GET_ITEM(seq, i)
            ...
    else
        for (...)
            item = PyList_GET_ITEM(seq, i)
            ...

> Still, it seems my PII's branch predictor nails that one... (i.e. changing
> it so that it didn't check inside the loop made naff-all difference to the
> running time).

yeah, the type isn't likely to change inside the loop...

</F>




From barry at digicool.com  Mon Nov 27 22:32:05 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 16:32:05 -0500
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
	<3A22B69F.D3C78D5C@lemburg.com>
	<m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
	<m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <14882.53973.591033.946156@anthem.concentric.net>

>>>>> "MH" == Michael Hudson <mwh21 at cam.ac.uk> writes:

    MH> I want a way of submitting patches by email.

I want a way of submitting bugs by email too.
-Barry



From greg at cosc.canterbury.ac.nz  Tue Nov 28 00:32:55 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 12:32:55 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <14882.29987.132557.43415@anthem.concentric.net>
Message-ID: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>

Anyone with an interest in the string functions vs.
string methods debate should read this article, which
was referenced in comp.lang.python recently:

    How Non-Member Functions Improve Encapsulation
    by Scott Meyers
    http://www.cuj.com/archive/1802/feature.html

Mr. Meyers presents some very well-reasoned arguments
against the everything-should-be-a-method mentality.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From mal at lemburg.com  Tue Nov 28 00:45:44 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 00:45:44 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <3A22F228.810F7B29@lemburg.com>

Greg Ewing wrote:
> 
> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Note that the motivation for turning to string methods was
that of migrating from strings to Unicode. Adding Unicode
support to the strop C module would have caused very complicated
code -- methods helped by enabling polymorphic code which is
one of the great advantages of writing software for an interface
rather than an implementation.

Note that functions can make very good use of methods and
thus implement polymorphic functionality -- this is not
about methods vs. functions it's about methods to enable
polymorphic functions.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Tue Nov 28 00:58:52 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 18:58:52 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 12:32:55 +1300."
             <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>

> Anyone with an interest in the string functions vs.
> string methods debate should read this article, which
> was referenced in comp.lang.python recently:
> 
>     How Non-Member Functions Improve Encapsulation
>     by Scott Meyers
>     http://www.cuj.com/archive/1802/feature.html
> 
> Mr. Meyers presents some very well-reasoned arguments
> against the everything-should-be-a-method mentality.

Just skimmed it -- seems to be a good argument.  (Also for why
capwords and zfill aren't worth adding as methods. :-)

It would be easy to propose join() as a built-in, and this looks
attractive, if it weren't for the existence of os.path.join().  Some
people probably write ``from os.path import join'' and once join() is
a built-in function, this may be confusing for readers who miss that a
different join is imported.  (I'm not saying that it breaks existing
code -- just that it's confusing to have two joins.)  But maybe this
is a moot argument since string.join() is already a function.

I wonder what Java enthusiasts would say after reading Meyers'
article: there *are* no non-member non-friend functions in Java.  The
best approximation is static methods, but they still live in a
class...

--Guido van Rossum (home page: http://www.python.org/~guido/)




From barry at digicool.com  Tue Nov 28 01:10:56 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:10:56 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
	<200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
Message-ID: <14882.63504.970688.304915@anthem.concentric.net>

>>>>> "GE" == Greg Ewing <greg at cosc.canterbury.ac.nz> writes:

    GE> Anyone with an interest in the string functions vs.
    GE> string methods debate should read this article, which
    GE> was referenced in comp.lang.python recently:

    |     How Non-Member Functions Improve Encapsulation
    |     by Scott Meyers
    |     http://www.cuj.com/archive/1802/feature.html

    GE> Mr. Meyers presents some very well-reasoned arguments
    GE> against the everything-should-be-a-method mentality.

Remember that Meyers is talking about C++ here, where there's a
distinction in the `friendliness' of a function.  He himself says that
the choice is between non-member non-friend functions and member
functions (i.e. methods).  Friend functions have the same dependency
on implementation changes as methods.

In Python, there is no such friend distinction, although you can come
close to faking it with private names.  So the argument based on
degrees of encapsulation doesn't hold water in Python.

in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,
-Barry



From guido at python.org  Tue Nov 28 01:07:38 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:07:38 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 00:45:44 +0100."
             <3A22F228.810F7B29@lemburg.com> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
            <3A22F228.810F7B29@lemburg.com> 
Message-ID: <200011280007.TAA30005@cj20424-a.reston1.va.home.com>

> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

Of course.  Meyers starts by saying that if it needs implementation
details it must be a method.  This is true (if only for efficiency
reasons) for most string methods.

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

Meyers also says that if it needs to be virtual it needs to be a
method.  Polymorphism is roughly equivalent to virtual in this
context, and this alone justifies the move to methods.

But join() is special: it is polymorphic in two arguments, and making
it a method of the separator argument doesn't help.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fredrik at effbot.org  Tue Nov 28 01:24:42 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 01:24:42 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com>
Message-ID: <042401c058d1$9b61f030$3c6340d5@hagrid>

mal wrote:
> > Mr. Meyers presents some very well-reasoned arguments
> > against the everything-should-be-a-method mentality.
> 
> Note that the motivation for turning to string methods was
> that of migrating from strings to Unicode. Adding Unicode
> support to the strop C module would have caused very complicated
> code -- methods helped by enabling polymorphic code which is
> one of the great advantages of writing software for an interface
> rather than an implementation.

as Alex pointed out on comp.lang.python, the first if-clause
in Meyers algorithm is:

    if (f needs to be virtual)
       make f a member function of C;

which probably covers the join case pretty well...

> Note that functions can make very good use of methods and
> thus implement polymorphic functionality -- this is not
> about methods vs. functions it's about methods to enable
> polymorphic functions.

a bit further down, the algorithm goes:

    else if (f can be implemented via C's
             public interface)
       make f a non-member function;

which covers all remaining non-trivial functions in the string
module...

(okay, you could get around this by renaming the method to
something a bit more annoying; let's say the "public interface"
contains a virtual method called "use_myself_to_join_sequence",
in which "join" could be a non-member convenience function
using nothing but the public interface.  but that's just silly...)

:::

fwiw, I still think that Meyers article cannot be fully applied
to Python, since he's ignoring the case:

    else if (f can be implemented via C's
             public interface, but may have
             to be overridden by a subclass
             to C)
       make f a member function;

this isn't a problem in a language that supports function over-
loading (see the interfaces/packages section in meyers article),
but as we all know, Python doesn't...

</F>




From barry at digicool.com  Tue Nov 28 01:17:32 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:17:32 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
	<3A22F228.810F7B29@lemburg.com>
	<200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <14882.63900.127012.804027@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> But join() is special: it is polymorphic in two arguments,
    GvR> and making it a method of the separator argument doesn't
    GvR> help.

What about calling the built-in strjoin().  That avoids the conflict
with os.path.join() and strengthens the connection with the intended
behavior -- that the return value is a string and the elements are
str()-ified.

or-use-sjoin()-ly y'rs,
-Barry



From guido at python.org  Tue Nov 28 01:12:01 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 19:12:01 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Mon, 27 Nov 2000 19:10:56 EST."
             <14882.63504.970688.304915@anthem.concentric.net> 
References: <14882.29987.132557.43415@anthem.concentric.net> <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
            <14882.63504.970688.304915@anthem.concentric.net> 
Message-ID: <200011280012.TAA30062@cj20424-a.reston1.va.home.com>

> Remember that Meyers is talking about C++ here, where there's a
> distinction in the `friendliness' of a function.  He himself says that
> the choice is between non-member non-friend functions and member
> functions (i.e. methods).  Friend functions have the same dependency
> on implementation changes as methods.
> 
> In Python, there is no such friend distinction, although you can come
> close to faking it with private names.  So the argument based on
> degrees of encapsulation doesn't hold water in Python.
> 
> in-python-everyone's-your-friend-and-thank-guido-for-that-ly y'rs,

I disagree.  While non-methods *can* look inside an implementation,
it's still implied that they break encapsulation if they do.  (This
may explain why you like to use private names -- they make the
encapsulation explicit.  For me, it exists even when it is implicit.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Tue Nov 28 01:26:25 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Mon, 27 Nov 2000 19:26:25 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <14882.29987.132557.43415@anthem.concentric.net>
	<200011272332.MAA03271@s454.cosc.canterbury.ac.nz>
	<14882.63504.970688.304915@anthem.concentric.net>
	<200011280012.TAA30062@cj20424-a.reston1.va.home.com>
Message-ID: <14882.64433.691041.119209@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> I disagree.  While non-methods *can* look inside an
    GvR> implementation, it's still implied that they break
    GvR> encapsulation if they do.  (This may explain why you like to
    GvR> use private names -- they make the encapsulation explicit.
    GvR> For me, it exists even when it is implicit.)

True, but it still means that changing the implementation has an
unknown effect on existing code, because there /could/ be a lot of
breakage.  That can be (and has been! :) written off as the fault of
the 3rd party author, but still, it isn't enforcible like it is in
C++, where, if it ain't a friend, it's not going to break.  In C++ you
can count the number of functions that will break to determine the
degree of encapsulation, but you can't in Python.

Witness the recent change to cgi.FieldStorage.  We removed the
self.lines attribute because it was a memory bloat, and we /thought/
that it wouldn't affect existing code, but there was no way to really
know for sure because the intent of the cgi module author wasn't at
all clear.  Was self.lines intended to be public but just accidentally
undocumented?  Was it a debugging tool?

As an aside, this argument's not really true with built-in types such
as strings because the internal implementation isn't publically
available to the Python programmer.

-Barry



From greg at cosc.canterbury.ac.nz  Tue Nov 28 01:24:07 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:24:07 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <200011272358.SAA29910@cj20424-a.reston1.va.home.com>
Message-ID: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz>

Guido said:

> It would be easy to propose join() as a built-in, and this looks
> attractive, if it weren't for the existence of os.path.join().

and

> join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

How about an operator, then?

We already have % rather than a string.format or some such.
Presumably this is because Guido thought it was such a handy
thing that it should be instantly available at our fingertips.
I think a similar argument could be made for string.join, and
also its inverse string.split.

So, how about:

   x & s   == string.join(x, s)

   s1 | s2 == string.split(s1, s2)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From greg at cosc.canterbury.ac.nz  Tue Nov 28 01:31:35 2000
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2000 13:31:35 +1300 (NZDT)
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: <042401c058d1$9b61f030$3c6340d5@hagrid>
Message-ID: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>

Fredrik Lundh <fredrik at effbot.org>:

> he's ignoring the case:
>
>    else if (f can be implemented via C's
>             public interface, but may have
>             to be overridden by a subclass
>             to C)

Isn't that the same as "f needs to be virtual"?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg at cosc.canterbury.ac.nz	   +--------------------------------------+



From fredrik at effbot.org  Tue Nov 28 02:11:48 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 02:11:48 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011280031.NAA03281@s454.cosc.canterbury.ac.nz>
Message-ID: <044a01c058d8$3144b320$3c6340d5@hagrid>

Greg Ewing wrote:
> > he's ignoring the case:
> >
> >    else if (f can be implemented via C's
> >             public interface, but may have
> >             to be overridden by a subclass
> >             to C)
> 
> Isn't that the same as "f needs to be virtual"?

in Python, sure.

in C++, it depends.  thanks to function overloading, you can have
polymorphism without having to make everything virtual...

</F>




From guido at python.org  Tue Nov 28 02:22:07 2000
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2000 20:22:07 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 13:24:07 +1300."
             <200011280024.NAA03278@s454.cosc.canterbury.ac.nz> 
References: <200011280024.NAA03278@s454.cosc.canterbury.ac.nz> 
Message-ID: <200011280122.UAA30268@cj20424-a.reston1.va.home.com>

> How about an operator, then?
> 
> We already have % rather than a string.format or some such.
> Presumably this is because Guido thought it was such a handy
> thing that it should be instantly available at our fingertips.
> I think a similar argument could be made for string.join, and
> also its inverse string.split.
> 
> So, how about:
> 
>    x & s   == string.join(x, s)
> 
>    s1 | s2 == string.split(s1, s2)

I don't see the mnemonic.  (For %, the mnemonic is clear: %s, %d etc.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From esr at thyrsus.com  Tue Nov 28 02:25:03 2000
From: esr at thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Nov 2000 20:25:03 -0500
Subject: [Python-Dev] Rich Comparisons and New Coercions
In-Reply-To: <200011271531.KAA29648@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 10:31:33AM -0500
References: <Pine.GSO.4.10.10011270938120.11335-100000@sundial> <200011271531.KAA29648@cj20424-a.reston1.va.home.com>
Message-ID: <20001127202503.A1013@thyrsus.com>

Guido van Rossum <guido at python.org>:
> [Moshe, on rich comparisons]
> > Nope. I'm not even sure if they're going to be in 2.1. DA?
> 
> Yes, they should go into 2.1. 

Good.  Within a few hours after I see that feature in the beta I'll
have a very rich Set class for the standard library.  It includes all
the standard boolean operations, symmetric difference, powerset, and
useful overloads for most of the standard operators.  As the header
comment says:

# A set-algebra module for Python
#
# The functions work on any sequence type and return lists.
# The set methods can take a set or any sequence type as an argument.

Most of the code is tested already.  All it's waiting on is rich comparisons
so I can handle partial-ordering of sets properly.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

When only cops have guns, it's called a "police state".
        -- Claire Wolfe, "101 Things To Do Until The Revolution" 



From jeremy at alum.mit.edu  Tue Nov 28 03:10:17 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 27 Nov 2000 21:10:17 -0500 (EST)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <000f01c058b9$1fc03c60$3c6340d5@hagrid>
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com>
	<3A22B69F.D3C78D5C@lemburg.com>
	<m3bsv16tug.fsf@atrus.jesus.cam.ac.uk>
	<m38zq56th0.fsf@atrus.jesus.cam.ac.uk>
	<000f01c058b9$1fc03c60$3c6340d5@hagrid>
Message-ID: <14883.5129.304362.407763@bitdiddle.concentric.net>

test_unicodedata is failing for me on Linux.  Fredrik, you made a
change on Nov. 3 that changed test/output/test_unicodedata but not
test/test_unicodedata.pu.  Did you verify that the test suite worked
then?

Jeremy



From mwh21 at cam.ac.uk  Tue Nov 28 09:52:00 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 28 Nov 2000 08:52:00 +0000
Subject: [Python-Dev] Re: [Bug #121013] Bug in   <stringobject>.join(<unicodestring>)
In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 27 Nov 2000 22:08:41 +0100"
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> Michael Hudson wrote:
> > 
> > "M.-A. Lemburg" <mal at lemburg.com> writes:
> > 
> > > > Date: 2000-Nov-27 10:12
> > > > By: mwh
> > > >
> > > > Comment:
> > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > >
> > > Hehe, that was indeed a trivial patch. What was that about trees
> > > in a forest...
> > 
> > The way I found it was perhaps instructive.  I was looking at the
> > function, and thought "that's a bit complicated" so I rewrote it (My
> > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > test has finished[*]).  In the course of rewriting it, I saw the line
> > my patch touched and went "duh!".
> 
> Yeah. The bug must have sneaked in there when the function was
> updated to the PySequence_Fast_* implementation.
> 
> BTW, could you also add a patch for the test_string.py and
> test_unicode.py tests ?

Here's an effort, but it's a memory scribbling bug.  If I change
test_unicode.py thus:

Index: test_unicode.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
retrieving revision 1.22
diff -c -r1.22 test_unicode.py
*** test_unicode.py     2000/10/23 17:22:08     1.22
--- test_unicode.py     2000/11/28 08:49:11
***************
*** 70,76 ****
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = 'wxyz'
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
--- 70,76 ----
  
  # join now works with any sequence type
  class Sequence:
!     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
      def __len__(self): return len(self.seq)
      def __getitem__(self, i): return self.seq[i]
  
***************
*** 78,83 ****
--- 78,87 ----
  test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
  test('join', u' ', u'w x y z', Sequence())
  test('join', u' ', TypeError, 7)
+ test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
+ test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
+ test('join', ' ', u'w x y z', Sequence())
+ test('join', ' ', TypeError, 7)
  
  class BadSeq(Sequence):
      def __init__(self): self.seq = [7, u'hello', 123L]

and back out the fix for the join bug, this happens:

...
...
Testing Unicode formatting strings... done.
Testing builtin codecs...
Traceback (most recent call last):
  File "test_unicode.py", line 378, in ?
    assert unicode('hello','utf8') == u'hello'
  File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
 ?
    import codecs,aliases
SystemError: compile.c:185: bad argument to internal function
Segmentation fault

i.e. it crashes miles away from the problem.

I'll reply to the other stuff later - no time now.

Cheers,
M.

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                         and quoted by Aahz Maruch on comp.lang.python




From mal at lemburg.com  Tue Nov 28 10:17:29 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:17:29 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in   
 <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com> <3A22B69F.D3C78D5C@lemburg.com> <m3bsv16tug.fsf@atrus.jesus.cam.ac.uk> <3A22CD59.BC091CAD@lemburg.com> <m33dgc7a33.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A237829.EA13903D@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > Michael Hudson wrote:
> > >
> > > "M.-A. Lemburg" <mal at lemburg.com> writes:
> > >
> > > > > Date: 2000-Nov-27 10:12
> > > > > By: mwh
> > > > >
> > > > > Comment:
> > > > > I hope you're all suitably embarrassed - please see patch #102548 for the trivial fix...
> > > >
> > > > Hehe, that was indeed a trivial patch. What was that about trees
> > > > in a forest...
> > >
> > > The way I found it was perhaps instructive.  I was looking at the
> > > function, and thought "that's a bit complicated" so I rewrote it (My
> > > rewrite also seems to be bit quicker so I'll upload it as soon as make
> > > test has finished[*]).  In the course of rewriting it, I saw the line
> > > my patch touched and went "duh!".
> >
> > Yeah. The bug must have sneaked in there when the function was
> > updated to the PySequence_Fast_* implementation.
> >
> > BTW, could you also add a patch for the test_string.py and
> > test_unicode.py tests ?
> 
> Here's an effort, but it's a memory scribbling bug.  If I change
> test_unicode.py thus:
> 
> Index: test_unicode.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/test/test_unicode.py,v
> retrieving revision 1.22
> diff -c -r1.22 test_unicode.py
> *** test_unicode.py     2000/10/23 17:22:08     1.22
> --- test_unicode.py     2000/11/28 08:49:11
> ***************
> *** 70,76 ****
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = 'wxyz'
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> --- 70,76 ----
> 
>   # join now works with any sequence type
>   class Sequence:
> !     def __init__(self): self.seq = [u'w',u'x',u'y',u'z']
>       def __len__(self): return len(self.seq)
>       def __getitem__(self, i): return self.seq[i]
> 
> ***************
> *** 78,83 ****
> --- 78,87 ----
>   test('join', u'', u'abcd', (u'a', u'b', u'c', u'd'))
>   test('join', u' ', u'w x y z', Sequence())
>   test('join', u' ', TypeError, 7)
> + test('join', ' ', u'a b c d', [u'a', u'b', u'c', u'd'])
> + test('join', '', u'abcd', (u'a', u'b', u'c', u'd'))
> + test('join', ' ', u'w x y z', Sequence())
> + test('join', ' ', TypeError, 7)
> 
>   class BadSeq(Sequence):
>       def __init__(self): self.seq = [7, u'hello', 123L]
> 
> and back out the fix for the join bug, this happens:
> 
> ...
> ...
> Testing Unicode formatting strings... done.
> Testing builtin codecs...
> Traceback (most recent call last):
>   File "test_unicode.py", line 378, in ?
>     assert unicode('hello','utf8') == u'hello'
>   File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 30, in
>  ?
>     import codecs,aliases
> SystemError: compile.c:185: bad argument to internal function
> Segmentation fault
> 
> i.e. it crashes miles away from the problem.

The test is only supposed to assure that we don't trip again. It's
not intended to work in some way *before* applying your patch.

I always try to integrate tests for bugs into the test suites
for my mx stuff and AFAICTL this also seems to be the Python
dev style.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Tue Nov 28 10:21:30 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 10:21:30 +0100
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz>  
	            <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>
Message-ID: <3A23791A.9C5ACD60@lemburg.com>

Guido van Rossum wrote:
> 
> But join() is special: it is polymorphic in two arguments, and making
> it a method of the separator argument doesn't help.

join() is special indeed, but what about the semantics we talked
about last year (?)...

join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]

This should fit all uses of join() (accept maybe os.path.join).

How about naming the beast concat() with sep defaulting to '' to
avoid the problems with os.path.join() ?!


-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at effbot.org  Tue Nov 28 10:54:31 2000
From: fredrik at effbot.org (Fredrik Lundh)
Date: Tue, 28 Nov 2000 10:54:31 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
References: <200011271854.KAA15307@sf-web3.vaspecialprojects.com><3A22B69F.D3C78D5C@lemburg.com><m3bsv16tug.fsf@atrus.jesus.cam.ac.uk><m38zq56th0.fsf@atrus.jesus.cam.ac.uk><000f01c058b9$1fc03c60$3c6340d5@hagrid> <14883.5129.304362.407763@bitdiddle.concentric.net>
Message-ID: <00a401c05923$73ca2950$3c6340d5@hagrid>

Jeremy wrote:

> test_unicodedata is failing for me on Linux.  Fredrik, you made a
> change on Nov. 3 that changed test/output/test_unicodedata but not
> test/test_unicodedata.pu.  Did you verify that the test suite worked
> then?

I added 38,000 missing characters to the unicode database.
that didn't change the test, only the test output...

the november 3 checkin included the following files:

    Lib/test/output/test_unicodedata
    Modules/unicodedata_db.h
    Objects/unicodetype_db.h
    Tools/unicode/makeunicodedata.py

:::

hmm.  looks like the makefile dependencies haven't been
updated (is this done by hand on Unix???)

in Modules/Makefile.pre.in

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h

should be

    unicodedatabase.o: unicodedatabase.c unicodedatabase.h unicodedata_db.h

and in Objects/Makefile.in

    unicodectype.o: unicodectype.c

should be:

    unicodectype.o: unicodectype.c unicodectype_db.h
    
</F>




From mwh21 at cam.ac.uk  Tue Nov 28 11:19:14 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:19:14 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
In-Reply-To: <3A22CD59.BC091CAD@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>

On Mon, 27 Nov 2000, M.-A. Lemburg wrote:

> Michael Hudson wrote:
> 
> > > I still think that the PySequence_Fast_GETITEM() macro should at
> > > least include a fall-back type check which causes some exception in
> > > case the used object was not "fixed" using PySequence_Fast() (which
> > > I only noticed in string_join() just now).
> > 
> > It's hard to see how; you're not going to check each invocation of
> > PySequence_Fast_GETITEM for a NULL return, are you?  It's possible
> > that PySequence_Fast should raise an exception on being passed a
> > string or Unicode object... but probably not.
> 
> Since not using PySequence_Fast() to initialize the protocol,
> I'd suggest doing a Py_FatalError() with some explanatory
> text which gets printed to stderr -- still better than a
> segfault at some later point due to some dangling pointers...

So you'd want PySequence_Fast_GETITEM to look like

#define PySequence_Fast_GETITEM(s,i) \
    (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
          (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
                  Py_FatalError("muffin!") ))

?  That'd probably be fair - you'd want to check any performance hit, but
I wouldn't be surprised if it was neglible.

> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they? 
> 
> Yep. Fredrik added them quite late in the 2.0 release process.
> 
> > I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.  Still, it seems my PII's branch predictor nails that
> > one... (i.e. changing it so that it didn't check inside the loop made
> > naff-all difference to the running time).
> 
> I think Fredrik speculated on having the compiler optimizers
> taking care of the check... 

gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like

.stabn 68,0,825,.LM312-string_join
.LM312:
        movl PyList_Type at GOT(%ebx),%eax
        cmpl %eax,4(%edi)
        jne .L875
        movl 12(%edi),%eax
        jmp .L898
        .p2align 4,,7
.L875:
        leal 12(%edi),%eax
.L898:
        movl -32(%ebp),%edx
        movl (%eax,%edx,4),%esi

... but I'm no expert in assembly reading.

> hmm, it would probably also help
> to somehow declare PyTypeObject slots "const" -- is this possible
> in a struct definition ?

I have no idea!

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:35:57 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:35:57 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <002701c058ba$c99ced40$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281025490.11335-100000@localhost.localdomain>

On Mon, 27 Nov 2000, Fredrik Lundh wrote:

> Michael Hudson wrote:
> > > Fredrik's PySequence_Fast_* APIs look interesting, BTW. Should be
> > > used more often :-)
> > 
> > Yes.  But they're pretty new, aren't they?  I find them a bit
> > unsatisfactory that it's not possible to hoist the type check out of
> > the inner loop.
> 
> if you can figure out a way to do that with C macros,
> I'd love to see it...

No.  You can consider my above comment as a round-about way of saying "I
hate C" :-)

> on the other hand, PySequence_Fast guarantees that the
> returned object is either a list or a tuple, so you can easily
> move the test outside the loop yourself:
> 
>     if (PyTuple_Check(seq))
>         for (...)
>             item = PyTuple_GET_ITEM(seq, i)
>             ...
>     else
>         for (...)
>             item = PyList_GET_ITEM(seq, i)
>             ...

But doing that would slightly eliminate the point of the PySequenceFast*
API wouldn't it?  Rhetorical question; it doesn't really seem to matter.

Slightly less rhetorical question: string_join currently just assumes a
list of strings.  But it is conceivable that one day it might be changed
to call __str__ on instances it finds, and then you'd have the issue of
__str__ methods changing the length of the list you're iterating over
(this might more plausibly apply in other situations where you might want
to use PySequence_Fast but I haven't gone looking for them). So would it
be feasible to change PySequence_Fast to smash the type of a list it's
given with the "immutable_list_type" from listobject.c and then have a
PySequence_Fast_End function that would decref the sequence and un-smash
the type as appropriate?

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:42:10 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:42:10 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in <stringobject>.join(<unicodestring>)
In-Reply-To: <00a401c05923$73ca2950$3c6340d5@hagrid>
Message-ID: <Pine.LNX.4.10.10011281036320.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, Fredrik Lundh wrote:

> Jeremy wrote:
> 
> > test_unicodedata is failing for me on Linux.  Fredrik, you made a
> > change on Nov. 3 that changed test/output/test_unicodedata but not
> > test/test_unicodedata.pu.  Did you verify that the test suite worked
> > then?
> 
> I added 38,000 missing characters to the unicode database.
> that didn't change the test, only the test output...
> 
> the november 3 checkin included the following files:
> 
>     Lib/test/output/test_unicodedata
>     Modules/unicodedata_db.h
>     Objects/unicodetype_db.h
>     Tools/unicode/makeunicodedata.py
> 
> :::
> 
> hmm.  looks like the makefile dependencies haven't been
> updated (is this done by hand on Unix???)

I think so.  I've done a complete rebuild and the test now passes (I was
sure I'd done one of those more recently than Nov 3...).  So, false alarm,
sorry.

Cheers,
M.




From mwh21 at cam.ac.uk  Tue Nov 28 11:43:49 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Tue, 28 Nov 2000 10:43:49 +0000 (GMT)
Subject: [Python-Dev] Re: [Bug #121013] Bug in    <stringobject>.join(<unicodestring>)
In-Reply-To: <3A237829.EA13903D@lemburg.com>
Message-ID: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>

On Tue, 28 Nov 2000, M.-A. Lemburg wrote:

> The test is only supposed to assure that we don't trip again. It's
> not intended to work in some way *before* applying your patch.

Yes.  I don't know what I was thinking earlier.  Too many beers last
night, perhaps.

> I always try to integrate tests for bugs into the test suites
> for my mx stuff and AFAICTL this also seems to be the Python
> dev style.

Do you want to check it in then?  I can't.

Cheers,
M.




From mal at lemburg.com  Tue Nov 28 12:07:22 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:07:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in    
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281042170.11335-100000@localhost.localdomain>
Message-ID: <3A2391EA.133A2CEF@lemburg.com>

Michael Hudson wrote:
> 
> On Tue, 28 Nov 2000, M.-A. Lemburg wrote:
> 
> > The test is only supposed to assure that we don't trip again. It's
> > not intended to work in some way *before* applying your patch.
> 
> Yes.  I don't know what I was thinking earlier.  Too many beers last
> night, perhaps.
> 
> > I always try to integrate tests for bugs into the test suites
> > for my mx stuff and AFAICTL this also seems to be the Python
> > dev style.
> 
> Do you want to check it in then?  I can't.

Best is to put it up on SF as patch. Someone will then pick it up and
check it in sooner or later. In any case that'll prevent it
getting lost.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mal at lemburg.com  Tue Nov 28 12:12:56 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 12:12:56 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in  
 <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <3A239338.800E84E8@lemburg.com>

Michael Hudson wrote:
> 
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair - you'd want to check any performance hit, but
> I wouldn't be surprised if it was neglible.

Yes.
 
> [Fredrik's PySequence_Fast_* APIs]
> >
> > I think Fredrik speculated on having the compiler optimizers
> > taking care of the check...
> 
> gcc 2.95.1 doesn't seem to - even with -O9 you get stuff like
> 
> .stabn 68,0,825,.LM312-string_join
> .LM312:
>         movl PyList_Type at GOT(%ebx),%eax
>         cmpl %eax,4(%edi)
>         jne .L875
>         movl 12(%edi),%eax
>         jmp .L898
>         .p2align 4,,7
> .L875:
>         leal 12(%edi),%eax
> .L898:
>         movl -32(%ebp),%edx
>         movl (%eax,%edx,4),%esi
> 
> ... but I'm no expert in assembly reading.
> 
> > hmm, it would probably also help
> > to somehow declare PyTypeObject slots "const" -- is this possible
> > in a struct definition ?
> 
> I have no idea!

Me neither -- it could help the compiler when moving type
checks out of inner loops: the type slot of PyObject normally
doesn't change (except on Windows where you have to patch it
in order make VC++ happy).

Anyone got a working idea in this direction ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Tue Nov 28 12:32:22 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2000 12:32:22 +0100
Subject: [Python-Dev] Re: [Bug #121013] Bug in  <stringobject>.join(<unicodestring>)
References: <Pine.LNX.4.10.10011281009130.11209-100000@localhost.localdomain>
Message-ID: <012c01c0592e$e0b463e0$0900a8c0@SPIFF>

michael wrote:
> > Since not using PySequence_Fast() to initialize the protocol,
> > I'd suggest doing a Py_FatalError() with some explanatory
> > text which gets printed to stderr -- still better than a
> > segfault at some later point due to some dangling pointers...
> 
> So you'd want PySequence_Fast_GETITEM to look like
> 
> #define PySequence_Fast_GETITEM(s,i) \
>     (PyList_Check((s)) ? PyList_GET_ITEM((s),(i)) : \
>           (PyTuple_Check((s))? PyTuple_GET_ITEM((s),(i)) : \
>                   Py_FatalError("muffin!") ))
> 
> ?  That'd probably be fair

sorry, but that's just silly -- the function is called GET_ITEM,
not GetItem.

it's no less safe than PyList_GET_ITEM or PyTuple_GET_ITEM or
any other "trade speed for safety" macro.

</F>




From guido at python.org  Tue Nov 28 13:13:57 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 07:13:57 -0500
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
In-Reply-To: Your message of "Tue, 28 Nov 2000 10:21:30 +0100."
             <3A23791A.9C5ACD60@lemburg.com> 
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>  
            <3A23791A.9C5ACD60@lemburg.com> 
Message-ID: <200011281213.HAA02449@cj20424-a.reston1.va.home.com>

> join() is special indeed, but what about the semantics we talked
> about last year (?)...
> 
> join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> 
> This should fit all uses of join() (accept maybe os.path.join).

This is much more general than the current definition --
e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
the definition this much.

> How about naming the beast concat() with sep defaulting to '' to
> avoid the problems with os.path.join() ?!

Hm... if we can stick to the string semantics this would be okay.  But
we'd lose the symmetry of split/join.  Note that string.join has a
default separator of ' ' to roughly match the default behavoir of
split.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Tue Nov 28 16:10:51 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 10:10:51 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: Your message of "26 Nov 2000 23:22:15 GMT."
             <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> 
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>  
            <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>

----------
My feelings about this topic have been expressed adequately by others.

I'm not sure what to say about the code that worked in 1.5.2 but broke
in 2.0 -- if these were book examples, I suspect that the book was
probably using undocumented (if commonly seen) features, line
multi-argument append() or connect().

As someone else said:  Change happens.  Get over it. :-)

> "Tim Peters" <tim.one at home.com> writes:
> 
> > On the deprecation of the string module:  where did this idea come from?
> > I've never seen anything saying that the string module is deprecated.

Michael Hudson:

> I thought this, and went looking.  I found on
> http://www.python.org/1.6/, about four fifths of the way down:
> 
> Changed Modules
> 
>     string - most of this module is deprecated now that strings have
>     methods. This no longer uses the built-in strop module, but takes
>     advantage of the new string methods to provide transparent support
>     for both Unicode and ordinary strings.
> 
> I hope (and believe) this is Wrong.

Because of its importance, the deprecation time of the string module
will be longer than that of most deprecated modules.  I expect it
won't be removed until Python 3000.

> http://www.python.org/2.0/new-python.html says:
> 
>     The old string module is still around for backwards compatibility,
>     but it mostly acts as a front-end to the new string methods.
> 
> which is IMHO better.

Yes.

And Greg Ewing:

> I think there are still legitimate reasons for using some
> parts of the string module. For example, if you're one of
> those stubborn people who refuse to accept that ",".join(foo)
> is a better way of writing string.join(foo,",").

This has been discussed -- jut note that continuing to use the string
module *is* frowned upon, and such stubborn code will get its just
desserts when Py3K arrives.

I suggest adding the following to the string module's documentation
(rather than marking it as explicitly deprecated):

    This module exists for backwards compatibility only.  It will
    eventually be deprecated and its use should be avoided in new code.

I also suggest that someone go through the standard library and get
rid of all uses of the string module.  (What to do with string.letters
c.s.?)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Tue Nov 28 16:22:18 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue, 28 Nov 2000 16:22:18 +0100
Subject: [Python-Dev] A house upon the sand
References: <LNBBLJKPBEHFEDALKOLCGEGAIBAA.tim.one@home.com>  
	            <m3u28u71zs.fsf@atrus.jesus.cam.ac.uk> <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <3A23CDAA.F14E0011@lemburg.com>

Guido van Rossum wrote:
> 
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
> 
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new code.
> 
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

We will need some place to put constants and other non-method
based string related functions (much like the Unicode 
database module which serves a similar purpose)... 
the ideal complement would be a stringdata module which provides
the same interfaces as the unicodedata module if reasonably 
possible (we'd get into encoding problems again...) !

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From tismer at tismer.com  Tue Nov 28 15:33:38 2000
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 28 Nov 2000 16:33:38 +0200
Subject: Methods vs. Functions (Re: [Python-Dev] A house upon the sand)
References: <200011272332.MAA03271@s454.cosc.canterbury.ac.nz> <3A22F228.810F7B29@lemburg.com> <200011280007.TAA30005@cj20424-a.reston1.va.home.com>  
	            <3A23791A.9C5ACD60@lemburg.com> <200011281213.HAA02449@cj20424-a.reston1.va.home.com>
Message-ID: <3A23C242.C24BE5D9@tismer.com>


Guido van Rossum wrote:
> 
> > join() is special indeed, but what about the semantics we talked
> > about last year (?)...
> >
> > join(seq, sep) := seq[0] + sep + seq[1] + sep + ... + seq[n]
> >
> > This should fit all uses of join() (accept maybe os.path.join).
> 
> This is much more general than the current definition --
> e.g. join(range(5), 0) would yield 10.  I'm not too keen on widening
> the definition this much.

No, if n is the length of the sequence, the above definition
would calculate 10 and then raise IndexError :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     where do you want to jump today?   http://www.stackless.com



From akuchlin at mems-exchange.org  Tue Nov 28 23:17:35 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 17:17:35 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282213.OAA31146@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Tue, Nov 28, 2000 at 02:13:47PM -0800
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
Message-ID: <20001128171735.A21996@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
>229
>is not sufficiently worked out and doesn't seem a priority.  (This can
>still make it into 2.1 if the author does the work.)

I was waiting for further comments on the idea before actually doing
the work.  Anyone see gaping holes in the basic idea?  (I *can* just
charge ahead and work on it anyway...)

--amk



From guido at python.org  Wed Nov 29 00:01:38 2000
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2000 18:01:38 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: Your message of "Tue, 28 Nov 2000 17:17:35 EST."
             <20001128171735.A21996@kronos.cnri.reston.va.us> 
References: <200011282213.OAA31146@slayer.i.sourceforge.net>  
            <20001128171735.A21996@kronos.cnri.reston.va.us> 
Message-ID: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>

> On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote:
> >229
> >is not sufficiently worked out and doesn't seem a priority.  (This can
> >still make it into 2.1 if the author does the work.)

Andrew:
> I was waiting for further comments on the idea before actually doing
> the work.  Anyone see gaping holes in the basic idea?  (I *can* just
> charge ahead and work on it anyway...)

I'd say charge ahead, but check where you are every once in a while.
Some things I'm not sure about include:

- Always shared libs.  What about Unixish systems that don't have
  shared libs?  What if you just want something to be hardcoded as
  statically linked, e.g. for security reasons?  (On the other hand
  there are some reasons why defaulting to shared libs is a good
  idea -- e.g. it makes it easier to upgrade a single extension module
  in an installation, e.g. for a critical fix.)

- Doesn't distutils currently always look for an existing installed
  Python installation to get its Makefile info?  How to force it to
  use the info from the build tree?

BTW off-topic: I didn't demote your PEP 222 (Web Library Enhancements)
but I'm pretty neutral on this -- except that it's probably a good
idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
than making the existing cgi.py even more complicated.  (Also, it's
very fragile -- you never know who uses what mis-feature.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From fdrake at acm.org  Wed Nov 29 00:21:44 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 28 Nov 2000 18:21:44 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
References: <200011282213.OAA31146@slayer.i.sourceforge.net>
	<20001128171735.A21996@kronos.cnri.reston.va.us>
	<200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <14884.15880.264559.125483@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > idea to start from scratch with a new cgi-ish module (cgi2.py?) rather

  Or it could be called "cgilib"...


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From jeff at ollie.clive.ia.us  Wed Nov 29 00:42:27 2000
From: jeff at ollie.clive.ia.us (Jeffrey C. Ollie)
Date: Tue, 28 Nov 2000 17:42:27 -0600
Subject: [Python-Dev] Python embedded in Exim
Message-ID: <20001128174227.A24208@ollie.clive.ia.us>

Well, I had an epiphany over the US Thanksgiving holiday weekend (or
maybe it was a brain aneurism) so I've spent some late nights and some
time stolen from work and come up with a first pass at embedding
Python in Exim.

I imagine that embedding Python in Exim will be interesting to those
folks writing virus scanners or for VERY tight integration of Mailman
with Exim.  You can also do all of those things that you can do with
the Perl string expansions.

Note: this code is VERY lightly tested. It compiles on my home
hacked-up RedHat Linux system and works on a couple of simple tests
that I've done, but no real e-mails have passed through this code yet.
I'm tossing this out for public comment on the design and
implementation...  It's been a while since I've done a lot of C
coding.

Unfortunately, you must embed Python 2.0.  This is because previous
versions of Python used a hacked copy of PCRE as it's regular
expression engine that conflicts with the copy of PCRE in Exim.

Starting in Python 2.0 the default regular expression is a completely
new one called SRE that supports Unicode.  The PCRE-based regular
expression engine is still included in Python 2.0 but I get around
that by creating a private copy of the Python library and delete the
offending object modules.  You could do that in versions of Python
prior to 2.0 but a lot of the standard library depends on regular
expressions I didn't think that there was much point in trying to make
the embedding work with 1.5.2 or 1.6.

Well, on to the patch itself.  The patch is made against Exim v3.20.
After you've patched the source code, you need to set four variables
in the Local/Makefile:

EXIM_PYTHON=python.o
PYTHON_INC=-I/usr/local/include/python2.0
PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a
PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil

Then build Exim as usual.

There are three runtime directives that control the embedding, all of which
are optional:

python_at_start		boolean - force startup of Python interpreter
python_module_paths	colon separated list of paths to append to sys.path
python_initial_import   colon separated list of modules to import at
                        interpreter initialization time

There are also two new command line switches -ys and -yd that will
force an immediate startup of Python or delay startup, overriding
python_at_start.

Then you use it:

${python{<module>.<function>}{<arg1>}...}

You must specify the module name and the function name.  The named
module will be imported automatically.  And currently you must specify
a function that returns a string (I'll look into adding accessing
attributes, methods of classes, and converting non-strings into
strings).  There can be up to eight arguments.  Each argument will be
expanded by Exim before it's passed to the Python interpreter.

Your python code has access to a module called "exim", which currently
defines two things:

exim.expand_string(s) which calls back to Exim to expand a string and
exim.error            which is an exception object used by expand_string

One simple example:

${python{string.upper}{$local_part}}

Error reporting isn't the best right now, I'm going to look into
formatting exceptions and tracebacks for more meaningful error
messages.

Well, I think that's the gist of it.  I'll work on patches to the
documentation when this thing becomes more stable.  As I said before
I've only lightly tested this... beware!

Comments, criticisms, suggestions, flames welcome...

Jeff

-------------- next part --------------
Index: exim/OS/Makefile-Base
diff -u exim/OS/Makefile-Base:1.1.1.1 exim/OS/Makefile-Base:1.1.1.1.2.3
--- exim/OS/Makefile-Base:1.1.1.1	Mon Nov 27 08:18:15 2000
+++ exim/OS/Makefile-Base	Tue Nov 28 16:49:25 2000
@@ -192,6 +192,16 @@
 	@chmod a+x ../util/convert4r3
 	@echo ">>> convert4r3 script built in util directory"; echo ""
 
+libpython.a: $(PYTHON_LIB)
+	if [ -n $(PYTHON_LIB) ] ;\
+	then \
+		cp $(PYTHON_LIB) libpython.a ;\
+		ar d libpython.a pcremodule.o pypcre.o ;\
+		ranlib libpython.a ;\
+	else \
+		ar cq libpython.a ;\
+		ranlib libpython.a ;\
+	fi
 
 # Targets for final binaries; the main one has a build number which is
 # updated each time. We don't bother with that for the auxiliaries.
@@ -201,11 +211,12 @@
         header.o host.o log.o match.o moan.o os.o parse.o queue.o \
         readconf.o retry.o rewrite.o \
         route.o search.o smtp_in.o smtp_out.o spool_in.o spool_out.o \
-        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PERL)
+        store.o string.o tls.o tod.o transport.o tree.o verify.o $(EXIM_PERL) \
+	$(EXIM_PYTHON)
 
 exim:   libident/libident.a pcre/libpcre.a lookups/lookups.a auths/auths.a \
         directors/directors.a routers/routers.a transports/transports.a \
-        $(OBJ_EXIM) version.c
+        $(OBJ_EXIM) version.c libpython.a
 	awk '{ print ($$1+1) }' cnumber.h > cnumber.temp
 	/bin/rm -f cnumber.h; mv cnumber.temp cnumber.h
 	$(CC) -c $(CFLAGS) $(INCLUDE) $(IPV6_INCLUDE) $(TLS_INCLUDE) version.c
@@ -215,7 +226,8 @@
 	  routers/routers.a transports/transports.a lookups/lookups.a \
 	  auths/auths.a \
 	  $(LIBS) $(LIBS_EXIM) $(IPV6_LIBS) $(EXTRALIBS) $(EXTRALIBS_EXIM) \
-	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) $(TLS_LIBS)
+	  $(DBMLIB) $(LIBRESOLV) $(LOOKUP_LIBS) $(PERL_LIBS) libpython.a $(PYTHON_EXTRA_LIBS) \
+	  $(TLS_LIBS)
 	$(EXIM_CHMOD)
 	@echo " "
 	@echo ">>> exim binary built"
@@ -316,6 +328,11 @@
 
 perl.o:          $(HDRS) perl.c
 	$(PERL_CC) $(PERL_CCOPTS) $(CFLAGS) $(INCLUDE) -c perl.c
+
+# Build instructions for python.o for when EXIM_PYTHON is set
+
+python.o:          $(HDRS) python.c
+	$(CC) -c $(CFLAGS) $(INCLUDE) $(PYTHON_INC) -I. python.c
 
 # Dependencies for the "ordinary" exim modules
 
Index: exim/scripts/MakeLinks
diff -u exim/scripts/MakeLinks:1.1.1.1 exim/scripts/MakeLinks:1.1.1.1.2.1
--- exim/scripts/MakeLinks:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/scripts/MakeLinks	Mon Nov 27 08:27:46 2000
@@ -189,6 +189,7 @@
 ln -s ../src/moan.c            moan.c
 ln -s ../src/parse.c           parse.c
 ln -s ../src/perl.c            perl.c
+ln -s ../src/python.c          python.c
 ln -s ../src/queue.c           queue.c
 ln -s ../src/readconf.c        readconf.c
 ln -s ../src/retry.c           retry.c
Index: exim/src/EDITME
diff -u exim/src/EDITME:1.1.1.2 exim/src/EDITME:1.1.1.2.2.4
--- exim/src/EDITME:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/EDITME	Tue Nov 28 16:20:19 2000
@@ -260,6 +260,10 @@
 # PERL_CCOPTS=
 # PERL_LIBS=
 
+# EXIM_PYTHON=python.o
+# PYTHON_INC=-I/usr/local/include/python2.0
+# PYTHON_LIB=/usr/local/lib/python2.0/config/libpython2.0.a
+# PYTHON_EXTRA_LIBS=-lpthread -ldl -lutil
 
 # This parameter sets the maximum length of the header portion of a message
 # that Exim is prepared to process. The default setting is one megabyte. There
Index: exim/src/config.h.defaults
diff -u exim/src/config.h.defaults:1.1.1.1 exim/src/config.h.defaults:1.1.1.1.2.1
--- exim/src/config.h.defaults:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/config.h.defaults	Mon Nov 27 08:43:04 2000
@@ -36,6 +36,8 @@
 
 #define EXIM_PERL
 
+#define EXIM_PYTHON
+
 #define HAVE_SA_LEN
 #define HEADER_MAXSIZE  (1024*1024)
 
Index: exim/src/exim.c
diff -u exim/src/exim.c:1.1.1.2 exim/src/exim.c:1.1.1.2.2.4
--- exim/src/exim.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/exim.c	Tue Nov 28 00:46:52 2000
@@ -351,6 +351,9 @@
 #ifdef EXIM_PERL
 int  perl_start_option = 0;
 #endif
+#ifdef EXIM_PYTHON
+int  python_start_option = 0;
+#endif
 int  recipients_arg = argc;
 int  sender_address_domain = 0;
 int  test_retry_arg = -1;
@@ -583,6 +586,17 @@
 simply recorded for checking and handling afterwards. Do a high-level switch
 on the second character (the one after '-'), to save some effort. */
 
+#ifdef EXIM_PYTHON
+opt_python_original_argc = argc;
+opt_python_original_argv = store_get(sizeof(char *) * (argc + 1));
+opt_python_original_argv[argc] = NULL;
+
+for (i = 0; i < argc; i++)
+  {
+  opt_python_original_argv[i] = string_copy(argv[i]);
+  }
+#endif
+
 for (i = 1; i < argc; i++)
   {
   BOOL badarg = FALSE;
@@ -1521,6 +1535,17 @@
     break;
     #endif
 
+    /* -ys: force Python startup; -yd force delayed Python startup */
+
+    #ifdef EXIM_PYTHON
+    case 'y':
+    if (*argrest == 's' && argrest[1] == 0) python_start_option = 1;
+    else
+      if (*argrest == 'd' && argrest[1] == 0) python_start_option = -1;
+    else badarg = TRUE;
+    break;
+    #endif
+
 
     case 'q':
 
@@ -1967,6 +1992,27 @@
   opt_perl_started = TRUE;
   }
 #endif /* EXIM_PERL */
+
+/* Start up Python interpreter if Python support is configured and
+there is a and the configuration or the command line specifies
+initializing starting. Note that the global variables are actually
+called opt_python_xxx. */
+
+#ifdef EXIM_PYTHON
+if (python_start_option != 0)
+  opt_python_at_start = (python_start_option > 0);
+if (opt_python_at_start)
+  {
+  char *errstr;
+  DEBUG(9) debug_printf("Starting Python interpreter\n");
+  errstr = init_python();
+  if (errstr != NULL)
+    {
+    fprintf(stderr, "exim: error during Python startup: %s\n", errstr);
+    return EXIT_FAILURE;
+    }
+  }
+#endif /* EXIM_PYTHON */
 
 /* Log the arguments of the call if the configuration file said so. This is
 a debugging feature for finding out what arguments certain MUAs actually use.
Index: exim/src/expand.c
diff -u exim/src/expand.c:1.1.1.2 exim/src/expand.c:1.1.1.2.2.3
--- exim/src/expand.c:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/expand.c	Tue Nov 28 00:23:10 2000
@@ -1318,7 +1318,7 @@
 
 Operators:
    domain                  extracts domain from an address
-   escape                  escapes non-printing characters
+   escape                  escapes non-printing characters 
    expand                  expands the string one more time
    hash_<n>_<m>            hash the string, making one that is of length n,
                              using m (default 26) characters from hashcodes
@@ -1865,6 +1865,100 @@
     continue;
     }
   #endif /* EXIM_PERL */
+
+  /* If Perl support is configured, handle calling embedded perl subroutines,
+  unless locked out at this time. Syntax is ${perl{sub}} or ${perl{sub}{arg}}
+  or ${perl{sub}{arg1}{arg2}} or up to a maximum of EXIM_PERL_MAX_ARGS
+  arguments (defined below). */
+
+  #ifdef EXIM_PYTHON
+  #define EXIM_PYTHON_MAX_ARGS 8
+
+  if (strcmp(name, "python") == 0)
+    {
+    int i = 0;
+    char *sub_name;
+    char *sub_arg[EXIM_PYTHON_MAX_ARGS + 1];
+    char *new_yield;
+
+    if (expand_forbid_python)
+      {
+      expand_string_message = "Python calls are not permitted";
+      goto EXPAND_FAILED;
+      }
+
+    while (isspace((uschar)*s)) s++;
+    if (*s != '{') goto EXPAND_FAILED_CURLY;
+    sub_name = expand_string_internal(s+1, TRUE, &s, FALSE);
+    if (sub_name == NULL) goto EXPAND_FAILED;
+    while (isspace((uschar)*s)) s++;
+    if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+
+    while (isspace((uschar)*s)) s++;
+
+    while (*s == '{')
+      {
+      if (i == EXIM_PYTHON_MAX_ARGS)
+        {
+        expand_string_message =
+          string_sprintf("Too many arguments passed to Python subroutine \"%s\" "
+            "(max is %d)", sub_name, EXIM_PYTHON_MAX_ARGS);
+        goto EXPAND_FAILED;
+        }
+      sub_arg[i] = expand_string_internal(s+1, TRUE, &s, FALSE);
+      if (sub_arg[i++] == NULL) goto EXPAND_FAILED;
+      while (isspace((uschar)*s)) s++;
+      if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+      while (isspace((uschar)*s)) s++;
+      }
+
+    if (*s++ != '}') goto EXPAND_FAILED_CURLY;
+    sub_arg[i] = 0;
+
+    /* Start the interpreter if necessary */
+
+    if (!opt_python_started)
+      {
+      char *initerror;
+      DEBUG(9) debug_printf("Starting Python interpreter\n");
+      initerror = init_python();
+      if (initerror != NULL)
+        {
+        expand_string_message =
+          string_sprintf("error during Python startup: %s\n", initerror);
+        goto EXPAND_FAILED;
+        }
+      }
+
+    /* Call the function */
+
+    new_yield = call_python_cat(yield, &size, &ptr, &expand_string_message,
+      sub_name, sub_arg);
+
+    /* NULL yield indicates failure; if the message pointer has been set to
+    NULL, the yield was undef, indicating a forced failure. Otherwise the
+    message will indicate some kind of Perl error. */
+
+    if (new_yield == NULL)
+      {
+      if (expand_string_message == NULL)
+        {
+        expand_string_message =
+          string_sprintf("Python subroutine \"%s\" returned undef to force "
+            "failure", sub_name);
+        expand_string_forcedfail = TRUE;
+        }
+      goto EXPAND_FAILED;
+      }
+
+    /* Yield succeeded. Ensure forcedfail is unset, just in case it got
+    set during a callback from Python. */
+
+    expand_string_forcedfail = FALSE;
+    yield = new_yield;
+    continue;
+    }
+  #endif /* EXIM_PYTHON */
 
   /* Handle character translation for "tr" */
 
Index: exim/src/functions.h
diff -u exim/src/functions.h:1.1.1.2 exim/src/functions.h:1.1.1.2.2.1
--- exim/src/functions.h:1.1.1.2	Mon Nov 27 08:18:54 2000
+++ exim/src/functions.h	Mon Nov 27 10:13:25 2000
@@ -16,6 +16,12 @@
 extern char *init_perl(char *);
 #endif
 
+#ifdef EXIM_PYTHON
+extern char *call_python_cat(char *, int *, int *, char **, char *, char **);
+extern void  cleanup_python(void);
+extern char *init_python(void);
+#endif
+
 #ifdef SUPPORT_TLS
 extern BOOL  tls_client_start(int, host_item *, address_item *, char *,
                char *, char *, char *, char *, int);
Index: exim/src/globals.c
diff -u exim/src/globals.c:1.1.1.1 exim/src/globals.c:1.1.1.1.2.2
--- exim/src/globals.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.c	Mon Nov 27 11:10:15 2000
@@ -35,6 +35,15 @@
 BOOL   opt_perl_started       = FALSE;
 #endif
 
+#ifdef EXIM_PYTHON
+BOOL   opt_python_at_start       = FALSE;
+char  *opt_python_module_paths   = NULL;
+char  *opt_python_initial_import = NULL;
+BOOL   opt_python_started        = FALSE;
+int    opt_python_original_argc  = 0;
+char **opt_python_original_argv  = NULL;
+#endif
+
 #ifdef HAVE_AUTH
 BOOL   auth_always_advertise  = TRUE;
 char  *auth_hosts             = NULL;
@@ -310,6 +319,7 @@
 BOOL   expand_forbid_exists   = FALSE;
 BOOL   expand_forbid_lookup   = FALSE;
 BOOL   expand_forbid_perl     = FALSE;
+BOOL   expand_forbid_python   = FALSE;
 int    expand_nlength[EXPAND_MAXN+1];
 int    expand_nmax            = -1;
 char  *expand_nstring[EXPAND_MAXN+1];
Index: exim/src/globals.h
diff -u exim/src/globals.h:1.1.1.1 exim/src/globals.h:1.1.1.1.2.2
--- exim/src/globals.h:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/globals.h	Mon Nov 27 11:10:15 2000
@@ -21,6 +21,15 @@
 extern BOOL   opt_perl_started;       /* Set once interpreter started */
 #endif
 
+#ifdef EXIM_PYTHON
+extern BOOL   opt_python_at_start;       /* start Python at startup */
+extern char  *opt_python_module_paths;   /* list of paths to append to sys.path */
+extern char  *opt_python_initial_import; /* list of modules to import at interpreter initialization time */
+extern BOOL   opt_python_started;        /* set once Python interpreter started */
+extern int    opt_python_original_argc;
+extern char **opt_python_original_argv;
+#endif
+
 #ifdef HAVE_AUTH
 extern BOOL   auth_always_advertise;  /* If FALSE, advertise only when needed */
 extern char  *auth_hosts;             /* These must authenticate */
@@ -208,6 +217,7 @@
 extern BOOL   expand_forbid_exists;   /* Lock out exists checking */
 extern BOOL   expand_forbid_lookup;   /* Lock out lookups */
 extern BOOL   expand_forbid_perl;     /* Lock out Perl calls */
+extern BOOL   expand_forbid_python;   /* Lock out Python calls */
 extern int    expand_nlength[];       /* Lengths of numbered strings */
 extern int    expand_nmax;            /* Max numerical value */
 extern char  *expand_nstring[];       /* Numbered strings */
Index: exim/src/python.c
diff -u /dev/null exim/src/python.c:1.1.2.11
--- /dev/null	Tue Nov 28 16:50:15 2000
+++ exim/src/python.c	Tue Nov 28 16:22:03 2000
@@ -0,0 +1,291 @@
+#include "exim.h"
+
+#include "Python.h"
+
+static void init_exim_module(void);
+
+char *init_python(void)
+{
+   if (opt_python_started)
+   {
+      return NULL;
+   }
+
+   Py_SetProgramName(opt_python_original_argv[0]);
+   
+   Py_Initialize();
+   
+   PySys_SetArgv(opt_python_original_argc, opt_python_original_argv);
+
+   init_exim_module();
+
+   if (opt_python_module_paths)
+   {
+      char *listptr = opt_python_module_paths;
+      int separator = 0;
+      char buffer[256];
+      PyObject *path;
+      PyObject *sys_module;
+      PyObject *sys_dict;
+      PyObject *sys_path;
+      PyObject *name;
+
+      name = PyString_FromString("sys");
+      if (name == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem creating string \"sys\"";
+      }
+
+      sys_module = PyImport_Import(name);
+      Py_DECREF(name);
+
+      if (sys_module == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to import sys";
+      }
+
+      sys_dict = PyModule_GetDict(sys_module);
+      if (sys_dict == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying get dictionary from module sys";
+      }
+
+      sys_path = PyDict_GetItemString(sys_dict, "path");
+      if (sys_path == NULL)
+      {
+	 PyErr_Clear();
+	 return "problem trying to get sys.path";
+      }
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)))
+      {
+	 path = PyString_FromString(buffer);
+	 if (path == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while allocating Python string for \"%s\"", buffer);
+	 }
+
+	 if(!PyList_Append(sys_path, path))
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem while insering string \"%s\"into sys.path", buffer);
+	 }
+      }
+   }
+
+   if (opt_python_initial_import)
+   {
+      char *listptr = opt_python_initial_import;
+      int separator = 0;
+      char buffer[256];
+      PyObject *module;
+      PyObject *name;
+
+      while(string_nextinlist(&listptr, &separator, buffer, sizeof(buffer)))
+      {
+	 name = PyString_FromString(buffer);
+	 if (name == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem creating string \"%s\"", buffer);
+	 }
+
+	 module = PyImport_Import(name);
+	 Py_DECREF(name);
+
+	 if (module == NULL)
+	 {
+	    PyErr_Clear();
+	    return string_sprintf("problem importing module %s", buffer);
+	 }
+      }
+   }
+
+      
+   opt_python_started = TRUE;
+   return NULL;
+}
+
+void cleanup_python(void)
+{
+  Py_Finalize();
+}
+
+static PyObject *find_function(char *name, char **errstrp)
+{
+   PyObject *module_name;
+   PyObject *function_name;
+   PyObject *module;
+   PyObject *dictionary;
+   PyObject *result;
+
+   char *ptr;
+
+   ptr = strrchr(name, '.');
+   
+   if (ptr == NULL)
+   {
+      *errstrp = string_sprintf("function name must include module: <module>.<name>");
+      return NULL;
+   }
+
+   function_name = PyString_FromString(ptr + 1);
+
+   if (function_name == NULL)
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("error trying to allocate function name");
+      return NULL;
+   }
+
+   module_name = PyString_FromStringAndSize(name, ptr - name);
+
+   if (function_name == NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp = string_sprintf("error trying to allocate module name");
+      return NULL;
+   }
+
+   module = PyImport_Import(module_name);
+
+   Py_DECREF(module_name);
+
+   if (module == NULL)
+   {
+      PyErr_Clear();
+      Py_DECREF(function_name);
+      *errstrp = string_sprintf("error trying to import module while trying to find %s", name);
+      return NULL;
+   }
+
+   dictionary = PyModule_GetDict(module);
+
+   result = PyDict_GetItem(dictionary, function_name);
+
+   Py_DECREF(function_name);
+
+   if (!PyCallable_Check(result))
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("%s is not a callable object", name);
+      return NULL;
+   }
+
+   return result;
+}
+
+char *call_python_cat(char *yield, int *sizep, int *ptrp, char **errstrp,
+		      char *name, char **arg)
+{
+   PyObject *function;
+   PyObject *arguments;
+   PyObject *result;
+   int index;
+   int count=0;
+   char **p;
+   char *buffer;
+   int length;
+
+   function = find_function(name, errstrp);
+
+   if (function == NULL)
+      return NULL;
+
+   p = arg;
+
+   while(*p)
+   {
+      count++;
+      p++;
+   }
+
+   arguments = PyTuple_New(count);
+
+   index = 0;
+   while(index < count)
+   {
+
+      PyTuple_SetItem(arguments, index, PyString_FromString(arg[index]));
+      index++;
+   }
+
+   result = PyObject_CallObject(function, arguments);
+
+   Py_DECREF(arguments);
+
+   if (result == NULL)
+   {
+      PyErr_Clear();
+      *errstrp = string_sprintf("error in Python function %s", name);
+      return NULL;
+   }
+
+   if(!PyString_Check(result))
+   {
+      Py_DECREF(result);
+      *errstrp = string_sprintf("Python function %s returned non-string", name);
+      return NULL;
+   }
+
+   if(PyString_AsStringAndSize(result, &buffer, &length))
+   {
+      PyErr_Clear();
+      Py_DECREF(result);
+      *errstrp = string_sprintf("could not extract the results of Python function %s", name);
+      return NULL;
+   }
+
+   yield = string_cat(yield, sizep, ptrp, buffer, length);
+
+   Py_DECREF(result);
+
+   return yield;
+}
+
+static PyObject *exim_error = NULL;
+
+static PyObject *exim_expand_string(PyObject *self, PyObject *args)
+{
+   char *i;
+   char *o;
+
+   if(!PyArg_ParseTuple(args, "s", i))
+      return NULL;
+   
+   o = expand_string(i);
+   
+   if (o == NULL)
+   {
+      PyErr_SetString(exim_error, expand_string_message);
+      return NULL;
+   }
+  
+   return PyString_FromString(o);
+}
+
+static PyMethodDef exim_methods[] = {
+   {"expand_string", exim_expand_string, 1},
+   {NULL,            NULL} /* sentinel */
+};
+
+static void init_exim_module(void)
+{
+   PyObject *m;
+   PyObject *d;
+   PyObject *o;
+
+   m = PyImport_AddModule("exim");
+   d = PyModule_GetDict(m);
+
+   Py_InitModule("exim", exim_methods);
+
+   exim_error = PyErr_NewException("exim.error", NULL, NULL);
+   PyDict_SetItemString(d, "error", exim_error);
+}
+
Index: exim/src/readconf.c
diff -u exim/src/readconf.c:1.1.1.1 exim/src/readconf.c:1.1.1.1.2.2
--- exim/src/readconf.c:1.1.1.1	Mon Nov 27 08:18:16 2000
+++ exim/src/readconf.c	Mon Nov 27 10:14:17 2000
@@ -160,6 +160,11 @@
   { "perl_at_start",            opt_bool,        &opt_perl_at_start },
   { "perl_startup",             opt_stringptr,   &opt_perl_startup },
 #endif
+#ifdef EXIM_PYTHON
+  { "python_at_start",          opt_bool,        &opt_python_at_start },
+  { "python_module_paths",      opt_stringptr,   &opt_python_module_paths },
+  { "python_initial_import",    opt_stringptr,   &opt_python_initial_import },
+#endif
 #ifdef LOOKUP_PGSQL
   { "pgsql_servers",            opt_stringptr,   &pgsql_servers },
 #endif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20001128/ab04e5bc/attachment-0001.pgp>

From gward at mems-exchange.org  Wed Nov 29 00:51:47 2000
From: gward at mems-exchange.org (Greg Ward)
Date: Tue, 28 Nov 2000 18:51:47 -0500
Subject: [Python-Dev] versioning standards?
In-Reply-To: <200011271811.NAA30901@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Nov 27, 2000 at 01:11:54PM -0500
References: <NEBBIACCCGNFLECLCLMHMELGCAAA.gvwilson@nevex.com> <3A228EDA.56050B52@lemburg.com> <200011271811.NAA30901@cj20424-a.reston1.va.home.com>
Message-ID: <20001128185146.A19116@ludwig.cnri.reston.va.us>

On 27 November 2000, Guido van Rossum said:
> > AFAIK, __version__ with a string value is in common usage both
> > in modules and classes.
> 
> Correct.  This was agreed upon as a standard long ago.  It's probably
> not documented anywhere as such.

I think that the "Distributing Python Modules" manual should have a
"Recommended Practices" section to cover things like this.  My personal
convention is that every single source file has a __revision__ variable,
and the "root" module [1] of a module distribution has __version__.

[1] the "root module" is the highest-level __init__.py in a package-ized
    distribution, or the module itself in a single-file distribution.
    Most non-packageized multi-module distributions have a fairly
    obvious place to put __version__.

        Greg



From akuchlin at mems-exchange.org  Wed Nov 29 03:57:48 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Tue, 28 Nov 2000 21:57:48 -0500
Subject: [Python-Dev] PEP 229 and 222
In-Reply-To: <200011282301.SAA03304@cj20424-a.reston1.va.home.com>; from guido@python.org on Tue, Nov 28, 2000 at 06:01:38PM -0500
References: <200011282213.OAA31146@slayer.i.sourceforge.net> <20001128171735.A21996@kronos.cnri.reston.va.us> <200011282301.SAA03304@cj20424-a.reston1.va.home.com>
Message-ID: <20001128215748.A22105@kronos.cnri.reston.va.us>

On Tue, Nov 28, 2000 at 06:01:38PM -0500, Guido van Rossum wrote:
>- Always shared libs.  What about Unixish systems that don't have
>  shared libs?  What if you just want something to be hardcoded as
>  statically linked, e.g. for security reasons?  (On the other hand

Beats me.  I'm not even sure if the Distutils offers a way to compile
a static Python binary.  (GPW: well, does it?)

>idea to start from scratch with a new cgi-ish module (cgi2.py?) rather
>than making the existing cgi.py even more complicated.  (Also, it's
>very fragile -- you never know who uses what mis-feature.)

Indeed.  My inclination is to design two new Request and Response
classes; haven't done that yet, though...

--amk




From fdrake at users.sourceforge.net  Wed Nov 29 07:20:31 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Tue, 28 Nov 2000 22:20:31 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011290620.WAA09888@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/



From tim.one at home.com  Wed Nov 29 07:23:10 2000
From: tim.one at home.com (Tim Peters)
Date: Wed, 29 Nov 2000 01:23:10 -0500
Subject: [Python-Dev] A house upon the sand
In-Reply-To: <200011281510.KAA03475@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMDIBAA.tim.one@home.com>

[Guido]
> ...
> Because of its importance, the deprecation time of the string module
> will be longer than that of most deprecated modules.  I expect it
> won't be removed until Python 3000.

I see nothing in the 2.0 docs, code, or "what's new" web pages saying that
it's deprecated.  So I don't think you can even start the clock on this one
before 2.1 (a fuzzy stmt on the web page for the unused 1.6 release doesn't
count ...).

> ...
> This has been discussed -- jut note that continuing to use the string
> module *is* frowned upon, and such stubborn code will get its just
> desserts when Py3K arrives.
>
> I suggest adding the following to the string module's documentation
> (rather than marking it as explicitly deprecated):
>
>     This module exists for backwards compatibility only.  It will
>     eventually be deprecated and its use should be avoided in new
>     code.
>
> I also suggest that someone go through the standard library and get
> rid of all uses of the string module.  (What to do with string.letters
> c.s.?)

They have to be in a module.  They've always been in the string module.
Common sense thus dictates "leave them in the string module" <0.3 wink>.

change-for-the-sake-of-irritation-is-arguably-irritating-ly y'rs  - tim




From bckfnn at worldonline.dk  Wed Nov 29 22:37:38 2000
From: bckfnn at worldonline.dk (Finn Bock)
Date: Wed, 29 Nov 2000 21:37:38 GMT
Subject: [Python-Dev] long formatting
Message-ID: <3a2576da.50607279@smtp.worldonline.dk>

Hi,

While adding support to Jython for formatting of longs in "%[duxXo]", I
inadvertently introduced another difference between Python and Jython:

Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
000000000000000000037777777766
>>>

Jython 2.0alpha1 on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> print '%030o' % -10L
-00000000000000000000000000012
>>>

The CPython behaviour looks as if the number have been through a C long
before it is formatted. It that intended? Would jython be wrong if it
used the result above?

regards,
finn



From fdrake at acm.org  Wed Nov 29 22:49:53 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 29 Nov 2000 16:49:53 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
References: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012
...
 > The CPython behaviour looks as if the number have been through a C long
 > before it is formatted. It that intended? Would jython be wrong if it
 > used the result above?

  It doesn't look like either of them is doing the right thing!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From tim_one at email.msn.com  Wed Nov 29 23:10:34 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:10:34 -0500
Subject: [Python-Dev] PEP-0218
In-Reply-To: <20001127150642.95E2AA9BB@darjeeling.zadka.site.co.il>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com>

[Moshe Zadka]
> ...
> Let me note that almost everything Greg Wilson wants to do can be done
> via a Python class implementing a set using a dictionary mapping to None.
> Almost?
>
> * No builitin syntax: import Set;Set(1,2,3) instead of {1,2,3}
> * Convertors: if we want list/tuple to have a semblance of efficiency,
>   we'll need to cache the element list as a list when accessed by
>   index.

Like everyone else, I have a Set class too.  Its __getitem__ is

    # so "for x in set:" works
    # see also set.tolist()
    def __getitem__(self, i):
        if i == 0:
            self.keys = self.d.keys()
        return self.keys[i]

Of course I don't *want* set[i] to mean anything at all; this is just a hack
to work with the current iteration protocol (i.e., Set.__getitem__ makes no
sense except in that it must be implemented so that "for x in set" works
today).  The one thing I could never get used to in Aaron's kjbuckets
implementation of sets is that for/in did not work as expected.

> * Two different constructors: set() for building from sequence, Set()
>   for building from elements. Might be confusing.

My Set.__init__:

    def __init__(self, seq=[]):
        self.d = d = {}
        for x in seq:
            d[x] = 1

That is, the constructor takes a sequence.  Same thing in Icon, which added
a set type late in the game.  No problem in practice.

> * Only possible elements of a set are immutables. OTOH, I'm not sure
>   how Greg intends to implement his sets if these sets are allowed
>   to contain mutable elements.

This needs to be addressed, but several approaches are possible.  For
example, my Set class allows for Sets of Sets (& so on), because I needed
them, and Sets are certainly mutable.

Now immutable objects are required because dicts require immutable objects
as keys.  However, a class (like Set!) can fool dicts, by supplying a
__hash__ and a __cmp__ "that work".  Dicts can be compared directly, so
__cmp__ is no problem.  The difficulty comes with the __hash__.  My solution
was to "freeze" a Set the instant __hash__ was invoked:  this allows
mutation up until the point a hash is captured, and disallows it thereafter.
Concretely,

    def __hash__(self):
        if self.frozen:
            hashcode = self.hashcode
        else:
            # The hash code must not depend on the order of the
            # keys.
            self.frozen = 1
            hashcode = 0
            _hash = hash
            for x in self.d.keys():
                hashcode = hashcode ^ _hash(x)
            self.hashcode = hashcode
        return hashcode

and all the mutating methods check self.frozen, raising ValueError if the
target set is currently frozen.  For example,

    # updating union
    def unionu(self, other):
        if self.frozen:
            raise ValueError(_frozenmsg)
        self.d.update(other.d)

Oddly enough, I almost never got a "frozen" error in practice; it seems that
by the time I was ready to build a set of sets, the constituent sets were at
their final values; and the few times I did get a frozen error, it was
actually a logic bug (I was mutating the set in error).  It's hard to guess
whether that's unique to me <0.5 wink>.

Icon has a different approach entirely:  an Icon set acts like a Python dict
*would* act if you inserted the id()s of the keys rather than the keys
themselves; that is, Icon's sets (and dicts) are based on object identity,
not object value.  Since object identity is invariant regardless of whether
the object is mutable, a hash table implementation has no difficulties.

I find identity semantics for sets less useful than value semantics, though.

[Eric Raymond]
> ...
> Within a few hours after I see [rich comparisons] in the beta I'll
> have a very rich Set class for the standard library.  It includes
> all the standard boolean operations, symmetric difference, powerset,
> and useful overloads for most of the standard operators.  As the
> header comment says:
>
> # A set-algebra module for Python
> #
> # The functions work on any sequence type and return lists.
> # The set methods can take a set or any sequence type as an argument.
> ...

I'm afraid this is a good example of why a set type is unlikely to make it
into the std distribution:  whenever a data structure is added to Python,
the agitation for variations is endless.  For example, returning a list
makes your flavor of sets unsuitable (because inefficient) for many
applications (e.g., there's no efficient way to test a list for element
membership).  "Almost all" set implementations I've seen for Python use
dicts for that reason.

Another set of complaints will be due to some people wanting functional
versions of the set operations, while others want mutating versions.  My own
Set class supplies both, because both are really needed in practice; e.g.,
the updating union was shown above; the functional union builds on that:

    # functional union
    def union(self, other):
        answer = self.__copy__()
        answer.unionu(other)
        return answer

(subtlety:  the functional union has no problem with "frozen" sets;
Set.__copy__ always returns a melted set).

Since it's Greg's PEP, it's up to him to make everyone happy <wink> -- but
since there's so much variation possible, I'm not sure Guido will ever bless
any Set class as "the" Set class.

BTW, one lesson to take from SETL:  a vital set operation in practice is a
mutating "pick a 'random' element E from the set, remove it from the set,
and return it".  An enormous number of set algorithms are of the form

    while not S.empty():
        pick some element from S
        deal with it, possibly mutating S

This operation can't be done efficiently in Python code if the set is
represented by a dict (the best you can do is materialize the full list of
keys first, and pick one of those).  That means my Set class often takes
quadratic time for what *should* be linear-time algorithms.

if-the-set-type-is-a-toy-there's-no-need-to-put-it-in-the-
    distribution-but-if-it's-not-a-toy-it-can't-be-written-
    in-python-today-ly y'rs  - tim





From tim_one at email.msn.com  Wed Nov 29 23:32:29 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Wed, 29 Nov 2000 17:32:29 -0500
Subject: [Python-Dev] long formatting
In-Reply-To: <3a2576da.50607279@smtp.worldonline.dk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENHIBAA.tim_one@email.msn.com>

[Finn Bock]
> While adding support to Jython for formatting of longs in "%[duxXo]", I
> inadvertently introduced another difference between Python and Jython:
>
> Python 2.0 (#8, Oct 16 2000, 17:27:58) [MSC 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> 000000000000000000037777777766
> >>>
>
> Jython 2.0alpha1 on java1.3.0 (JIT: null)
> Type "copyright", "credits" or "license" for more information.
> >>> print '%030o' % -10L
> -00000000000000000000000000012
> >>>
>
> The CPython behaviour looks as if the number have been through a C long
> before it is formatted. It that intended?

Hard to say, but it is what the code does <wink>.  In 1.5.2, "true longs"
raised an error in this context, so the code first looked to see whether the
putative Python long actually fit in a C long first; if so, it did not raise
an error, and ran the value thru C long formatting instead (in
stringobject.c):

				if (PyLong_Check(v) && PyLong_AsLong(v) == -1
				    && PyErr_Occurred()) {
					/* Too big for a C long. */

That test survived in 2.0; the difference is that it no longer raises an
error if the value doesn't fit in a C long.

> Would jython be wrong if it used the result above?

Hard to say!  Since the size of a Python int varies across platforms, not
even CPython will display the same thing for all Python longs across all
platforms.  I'm in favor of changing the test above to

				if (PyLong_Check(v)) {

Then CPython will match JPython here, and CPython will produce the same
result across all platforms when given a Python long.  The downside is that
it will not produce the same result as 2.0 (or earlier).

Guido, you can Pronounce now <wink>.





From guido at python.org  Thu Nov 30 04:14:47 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:14:47 -0500
Subject: [Python-Dev] Mail to guido@python.org lost or delayed
Message-ID: <200011300314.WAA18820@cj20424-a.reston1.va.home.com>

Folks, mail to guido at python.org may have been bouncing or dropped in a
black hole, since approximately 12:40 pm EST today -- due to a mail
server problem at my ISP (@home.com -- fast reliable cable bandwidth
but lousy mail and DNS service).  I found out about this and *thought*
I had fixed it by switching to guido at digicool.com, but in fact there
was still a forward to @home.com in effect.  I just finally fixed that
(around 10:10 pm EST).

You don't need to change the email address you have for me:
guido at python.org is still the preferred address.  However if you sent
me mail between the approximate times above, I probably haven't seen
it yet.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 04:17:44 2000
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2000 22:17:44 -0500
Subject: [Python-Dev] Re: long formatting
Message-ID: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>

I just skimmed the pipermail python-dev archives and found one item
to pronounce on: what "%o" should do with negative longs.  I agree
with Tim.  Also for "%x".  Reason: this matches oct() and hex().

>>> oct(-10L)
'-012L'
>>> oct(-10)
'037777777766'
>>> hex(-10L)
'-0xAL'
>>> hex(-10)
'0xfffffff6'
>>> 

Compatibility be damned (in this case).

(One could argue that for plain ints, the same result are desirable;
however when these are used as bitmasks, I as human reader prefer to
see '0xfffffff6' rather than '-0xA'.  Ah the insanity of consistency!)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From tim_one at email.msn.com  Thu Nov 30 06:41:03 2000
From: tim_one at email.msn.com (Tim Peters)
Date: Thu, 30 Nov 2000 00:41:03 -0500
Subject: [Python-Dev] Re: long formatting
In-Reply-To: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>

Alas, Guido and I have the same ISP (@home), and its mail server has been
dead all day.  Conveniently <snort>, my other ISP (MSN) coincidentally
stopped letting me send email over my cable connection too.

[Guido]
> I just skimmed the pipermail python-dev archives and found one item
> to pronounce on: what "%o" should do with negative longs.  I agree
> with Tim.  Also for "%x".  Reason: this matches oct() and hex().

Good point!

> >>> oct(-10L)
> '-012L'
> >>> oct(-10)
> '037777777766'

Note that this string varies across platforms.

> >>> hex(-10L)
> '-0xAL'
> >>> hex(-10)
> '0xfffffff6'

Ditto.

> >>>
>
> Compatibility be damned (in this case).

This became SF bug 123859, and has been closed.  Current CVS does:

>>> "%#o" % -10L
'-012'
>>> "%#X" % -10L
'-0XA'
>>>

> (One could argue that for plain ints, the same result are
> desirable; however when these are used as bitmasks, I as human
> reader prefer to see '0xfffffff6' rather than '-0xA'.  Ah the
> insanity of consistency!)

I doubt that we're done with this forever:  we're moving in the direction of
erasing user-visible distinctions between ints and longs.  For bitmask
output, we should probably introduce a new format code.

Fred, I see that you complained about Finn's output examples, but didn't say
why (just "they both look incorrect").  Were you hallucinating, or do you
have a legit complaint?





From fdrake at acm.org  Thu Nov 30 07:22:02 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 01:22:02 -0500 (EST)
Subject: [Python-Dev] long formatting
In-Reply-To: <14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
References: <200011300317.WAA18829@cj20424-a.reston1.va.home.com>
	<LNBBLJKPBEHFEDALKOLCIENNIBAA.tim_one@email.msn.com>
	<3a2576da.50607279@smtp.worldonline.dk>
	<14885.31233.710760.311556@cj42289-a.reston1.va.home.com>
Message-ID: <14885.61962.901996.857623@cj42289-a.reston1.va.home.com>

Finn Bock writes:
 > >>> print '%030o' % -10L
 > 000000000000000000037777777766
...
 > -00000000000000000000000000012

Fred L. Drake, Jr. writes:
 >   It doesn't look like either of them is doing the right thing!

Tim Peters writes:
 > Fred, I see that you complained about Finn's output examples, but didn't say
 > why (just "they both look incorrect").  Were you hallucinating, or do you
 > have a legit complaint?

[Fred scrutinizes everything very carefully again... blushes when he
 notices that this was the 'o' format code, not 'd', and quietly
 slinks off to walk the dog...]


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From moshez at zadka.site.co.il  Thu Nov 30 15:52:59 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 16:52:59 +0200
Subject: [Python-Dev] PEP-0218 
In-Reply-To: Message from "Tim Peters" <tim_one@email.msn.com> 
   of "Wed, 29 Nov 2000 17:10:34 EST." <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> 
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> 
Message-ID: <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>

> BTW, one lesson to take from SETL:  a vital set operation in practice is a
> mutating "pick a 'random' element E from the set, remove it from the set,
> and return it".  An enormous number of set algorithms are of the form
> 
>     while not S.empty():
>         pick some element from S
>         deal with it, possibly mutating S
> 
> This operation can't be done efficiently in Python code if the set is
> represented by a dict (the best you can do is materialize the full list of
> keys first, and pick one of those).  That means my Set class often takes
> quadratic time for what *should* be linear-time algorithms.

Hmmm...actually, I've been wanting a method .key() for dictionaries 
a long time. So if we give dictionaries this one small method, then
we *can* do this in Python.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From fdrake at users.sourceforge.net  Thu Nov 30 08:33:55 2000
From: fdrake at users.sourceforge.net (Fred L. Drake)
Date: Wed, 29 Nov 2000 23:33:55 -0800
Subject: [Python-Dev] [development doc updates]
Message-ID: <200011300733.XAA12728@orbital.p.sourceforge.net>

The development version of the documentation has been updated:

	http://python.sourceforge.net/devel-docs/

Today's changes are primarily in the xml.dom API documentation:

	http://python.sourceforge.net/devel-docs/lib/markup.html



From mal at lemburg.com  Thu Nov 30 09:30:22 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 09:30:22 +0100
Subject: [Python-Dev] PEP-0218
References: <LNBBLJKPBEHFEDALKOLCEENGIBAA.tim_one@email.msn.com> <20001130145259.3F3B3A9BB@darjeeling.zadka.site.co.il>
Message-ID: <3A26101E.7FF2467B@lemburg.com>

Moshe Zadka wrote:
> 
> > BTW, one lesson to take from SETL:  a vital set operation in practice is a
> > mutating "pick a 'random' element E from the set, remove it from the set,
> > and return it".  An enormous number of set algorithms are of the form
> >
> >     while not S.empty():
> >         pick some element from S
> >         deal with it, possibly mutating S
> >
> > This operation can't be done efficiently in Python code if the set is
> > represented by a dict (the best you can do is materialize the full list of
> > keys first, and pick one of those).  That means my Set class often takes
> > quadratic time for what *should* be linear-time algorithms.
> 
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

Shouldn't be hard to do... the C API for this is already in place:
PyDict_Next(). I'd prefer a method .getitem(), though, which then
returns a tuple (key, value).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Thu Nov 30 09:44:38 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 10:44:38 +0200 (IST)
Subject: [Python-Dev] PEP-0218
In-Reply-To: <3A26101E.7FF2467B@lemburg.com>
Message-ID: <Pine.GSO.4.10.10011301043050.24575-100000@sundial>

[Me]
> Hmmm...actually, I've been wanting a method .key() for dictionaries
> a long time. So if we give dictionaries this one small method, then
> we *can* do this in Python.

[MAL]
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

Well, I'm sure the C API allows it. And if we do it, we might as well
do it right: add .key(), .value() and .item().

--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From pf at artcom-gmbh.de  Thu Nov 30 10:25:37 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 10:25:37 +0100 (MET)
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: <3A26101E.7FF2467B@lemburg.com> from "M.-A. Lemburg" at "Nov 30, 2000  9:30:22 am"
Message-ID: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>

M.-A. Lemburg:
> Shouldn't be hard to do... the C API for this is already in place:
> PyDict_Next(). I'd prefer a method .getitem(), though, which then
> returns a tuple (key, value).

IMO '.pickitem()' would be a better name, since many people would
think, that 'getitem()' would take some kind of index as parameter.
Nevertheless I think this is a nice idea, though.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal at lemburg.com  Thu Nov 30 11:55:33 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 11:55:33 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <3A263225.E2FFDADB@lemburg.com>

Peter Funk wrote:
> 
> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Point taken. How about .fetchitem() -- this doesn't imply any
ordering and can't get confused with __getitem__() et al.

BTW, Moshe's .key() and .value() would break the strong binding
between dictionary keys and values -- I don't see much use in them.
Besides, .fetchitem() provides the same implied functionality.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From guido at python.org  Thu Nov 30 12:10:33 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:10:33 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:25:37 +0100."
             <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> 
Message-ID: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>

> M.-A. Lemburg:
> > Shouldn't be hard to do... the C API for this is already in place:
> > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > returns a tuple (key, value).
> 
> IMO '.pickitem()' would be a better name, since many people would
> think, that 'getitem()' would take some kind of index as parameter.
> Nevertheless I think this is a nice idea, though.

Pronouncement:

It is only efficient to get the *first* item, so let's make that
explicit.  The method names will be:

 .firstkey()
 .firstvalue()
 .firstitem()

Moshe will check in a patch.

Thinking aloud:

Would it be useful to also implement popkey(), popvalue(), popitem(),
which would remove the first item and then return the relevant part of
it?

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mal at lemburg.com  Thu Nov 30 12:30:09 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 12:30:09 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <3A263A41.AFEFFB53@lemburg.com>

Guido van Rossum wrote:
> 
> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> >
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()

-1

>  .firstvalue()

-1

>  .firstitem()

+1

I really think that .firstitem() is both more intuitive and 
efficient -- .firstvalue() and .firstkey() are simply derivatives
of this method or how would you define the return value of
those two methods ?
 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

+1

Same comment as above... .popitem() suffices.
 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From pf at artcom-gmbh.de  Thu Nov 30 12:33:30 2000
From: pf at artcom-gmbh.de (Peter Funk)
Date: Thu, 30 Nov 2000 12:33:30 +0100 (MET)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> from Guido van Rossum at "Nov 30, 2000  6:10:33 am"
Message-ID: <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>

Guido van Rossum:
[...]
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

Since [].pop() takes in optional index as parameter which defaults
to the *LAST* element, it would be better to make the name explicit:
.popfirstitem()

This is however some what long.  DEJAVU: the {}.setdefault() discussion.
May it is better to leave these methods out for the sake of simplicity, 
although they look useful.

Regards, Peter



From guido at python.org  Thu Nov 30 12:52:10 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 06:52:10 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:30:09 +0100."
             <3A263A41.AFEFFB53@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>  
            <3A263A41.AFEFFB53@lemburg.com> 
Message-ID: <200011301152.GAA26000@cj20424-a.reston1.va.home.com>

> > Pronouncement:
> > 
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> > 
> >  .firstkey()
> 
> -1
> 
> >  .firstvalue()
> 
> -1
> 
> >  .firstitem()
> 
> +1
> 
> I really think that .firstitem() is both more intuitive and 
> efficient -- .firstvalue() and .firstkey() are simply derivatives
> of this method or how would you define the return value of
> those two methods ?

Yes, firstvalue() and firstkey() return just the value or just the
key.  I can see that the first value is not very useful (since you
don't have the key, you can't mutate the dict).  firstkey() might
be just the ticket e.g. for set implementations.  OTOH you may be
right that it's best to add the minimal number of new methods.

> > Moshe will check in a patch.
> > 
> > Thinking aloud:
> > 
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> +1
> 
> Same comment as above... .popitem() suffices.

And for minimality reasons I'm at most -0 on it.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 13:09:03 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:09:03 -0500
Subject: [Python-Dev] Python track--O'Reilly Open Source Convention
Message-ID: <200011301209.HAA26079@cj20424-a.reston1.va.home.com>

Laura Lewin is looking for chairs for two Python tracks (or a Python
track and a Zope track) at the O'Reilly conference.  Anybody
interested in volunteering?  Some background info:

> We actually have two tracks, and there's room for strong differentiation 
> between the two.  Each track has a room for two days. We'll also want 
> tutorials on the Monday and Tuesday.
> 
> Like O'Reilly's books, we're not really aimed at the novice. We want to 
> give people who already use the technology more information. There should 
> be at least one introductory tutorial, but we don't want to focus 
> exclusively on that. However, if you, as Chair, have a vision for what you 
> want O'Reilly's conference to be, we'd be very happy to work with you to 
> realize that vision.
> 
> This year we're trying to raise the bar on our Python and Zope offerings. 
> We'd like to have a lot of editorial involvement from a program committee, 
> so that potential speakers have a chance to get their talks reviewed and 
> commented-upon by others.
> 
> The program chair would really be the point person for the conference, to 
> drum up papers and help publicize the conference to the Python community. 
> The chair would appoint a committee, receive submissions, select 
> presentations, seek out talks from people who
> didn't submit, and generally put together the material.
> 
> We can handle any combination of program committees and chairs: one person 
> overseeing two committees, two chairs two committees, whatever gets it all 
> done.
> 
> Does this sound like a job you're interested in? It doesn't pay, but it 
> would be great for the Python community, as I'm sure you know.

For more info, please write or call Laura Lewin LLewin at oreilly.com
(631) 262-9278.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido at python.org  Thu Nov 30 13:40:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 07:40:06 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 04:31:07 PST."
             <200011301231.EAA22228@slayer.i.sourceforge.net> 
References: <200011301231.EAA22228@slayer.i.sourceforge.net> 
Message-ID: <200011301240.HAA26261@cj20424-a.reston1.va.home.com>

Aargh!  I thought I told you to submit a patch (to SF), not to check
it in.

The code looks okay but we'll have to discuss whether we really want
all three (as opposed to only firstitem()).  I'm also not sure whether
KeyError might be a better exception.

Don't touch it for now, but let's think this over a bit.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From mwh21 at cam.ac.uk  Thu Nov 30 14:16:44 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 13:16:44 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Guido van Rossum's message of "Thu, 30 Nov 2000 06:10:33 -0500"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
Message-ID: <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>

Guido van Rossum <guido at python.org> writes:

> > M.-A. Lemburg:
> > > Shouldn't be hard to do... the C API for this is already in place:
> > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > returns a tuple (key, value).
> > 
> > IMO '.pickitem()' would be a better name, since many people would
> > think, that 'getitem()' would take some kind of index as parameter.
> > Nevertheless I think this is a nice idea, though.
> 
> Pronouncement:
> 
> It is only efficient to get the *first* item, so let's make that
> explicit.  The method names will be:
> 
>  .firstkey()
>  .firstvalue()
>  .firstitem()
> 
> Moshe will check in a patch.
> 
> Thinking aloud:
> 
> Would it be useful to also implement popkey(), popvalue(), popitem(),
> which would remove the first item and then return the relevant part of
> it?

I must admit I can't think of a situation where .firstkey would be
more useful than .popkey; I thought that this discussion was
discussion was inspired by the wish to write code like:

while not set.empty():
    x = set.popfirstkey()
    ...

Maybe I'm jsut being dense, though.

FWIW, I think 

    dict.firstkey()

reads better than

    dict.firstitem()[0]

if the former's what you're actually trying to say.  Why not write

    for k,_ in dict.items():
        ...

? (Overstating the case slightly for effect, but I hope the point is
clear).

Cheers,
M.

-- 
  Arrrrgh, the braindamage!  It's not unlike the massively
  non-brilliant decision to use the period in abbreviations 
  as well as a sentence terminator.  Had these people no 
  imagination at _all_?                 -- Erik Naggum, comp.lang.lisp




From fdrake at acm.org  Thu Nov 30 14:34:07 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:34:07 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43
In-Reply-To: <200011301231.EAA22229@slayer.i.sourceforge.net>
References: <200011301231.EAA22229@slayer.i.sourceforge.net>
Message-ID: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > +   \lineiii{\var{a}.firstitem()}
 > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
 > +           {(2)}

  So what happens if the mapping is empty?  That's needed for all
three.
  You should also update UserDict.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From mal at lemburg.com  Thu Nov 30 14:40:12 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 14:40:12 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A2658BC.7922B8B1@lemburg.com>

Michael Hudson wrote:
> 
> Guido van Rossum <guido at python.org> writes:
> 
> > > M.-A. Lemburg:
> > > > Shouldn't be hard to do... the C API for this is already in place:
> > > > PyDict_Next(). I'd prefer a method .getitem(), though, which then
> > > > returns a tuple (key, value).
> > >
> > > IMO '.pickitem()' would be a better name, since many people would
> > > think, that 'getitem()' would take some kind of index as parameter.
> > > Nevertheless I think this is a nice idea, though.
> >
> > Pronouncement:
> >
> > It is only efficient to get the *first* item, so let's make that
> > explicit.  The method names will be:
> >
> >  .firstkey()
> >  .firstvalue()
> >  .firstitem()
> >
> > Moshe will check in a patch.
> >
> > Thinking aloud:
> >
> > Would it be useful to also implement popkey(), popvalue(), popitem(),
> > which would remove the first item and then return the relevant part of
> > it?
> 
> I must admit I can't think of a situation where .firstkey would be
> more useful than .popkey; I thought that this discussion was
> discussion was inspired by the wish to write code like:
> 
> while not set.empty():
>     x = set.popfirstkey()
>     ...
> 
> Maybe I'm jsut being dense, though.

Just curious: how would you get at the deleted value when
using .popfirstkey() ?

But you have a point: dict.pop() returning the first filled slot
in the dictionary as tuple (key,value) would probably be most
effective. It's also nice and short.

BTW, I don't get the inspiration for the "first" part in those
names... it makes you think that there's an order to dictionary
items when in reality there isn't.

> FWIW, I think
> 
>     dict.firstkey()
> 
> reads better than
> 
>     dict.firstitem()[0]
> 
> if the former's what you're actually trying to say.  Why not write
> 
>     for k,_ in dict.items():
>         ...
> 
> ? (Overstating the case slightly for effect, but I hope the point is
> clear).

No, the point is that you can modify the dictionary within
the loop. You wouldn't see any changes using the second technique.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From moshez at math.huji.ac.il  Thu Nov 30 14:43:10 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 15:43:10 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>

On Thu, 30 Nov 2000, Fred L. Drake, Jr. wrote:

> 
> Moshe Zadka writes:
>  > +   \lineiii{\var{a}.firstitem()}
>  > +           {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()}
>  > +           {(2)}
> 
>   So what happens if the mapping is empty?  That's needed for all
> three.
>   You should also update UserDict.
>

Right. Will be done soon.
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From fdrake at acm.org  Thu Nov 30 14:43:48 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 08:43:48 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib
 libstdtypes.tex,1.42,1.43
In-Reply-To: <Pine.GSO.4.10.10011301542250.4645-100000@sundial>
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com>
	<Pine.GSO.4.10.10011301542250.4645-100000@sundial>
Message-ID: <14886.22932.50399.879012@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > Right. Will be done soon.

  Given Guido's surprise at the checkin, you might want to wait until
the discussions are over and just make one set of checkins, with all
the results.  ;)
  Hmm.  I need to think about adding version annotations to those
tables of operations on built-in types.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From guido at python.org  Thu Nov 30 14:52:07 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 08:52:07 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 14:40:12 +0100."
             <3A2658BC.7922B8B1@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>  
            <3A2658BC.7922B8B1@lemburg.com> 
Message-ID: <200011301352.IAA26550@cj20424-a.reston1.va.home.com>

> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?
> 
> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Nice!

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

Actually there is -- the (arbitrary) order revealed by .keys().

--Guido van Rossum (home page: http://www.python.org/~guido/)



From moshez at zadka.site.co.il  Thu Nov 30 23:01:14 2000
From: moshez at zadka.site.co.il (Moshe Zadka)
Date: Fri, 01 Dec 2000 00:01:14 +0200
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43 
In-Reply-To: Your message of "Thu, 30 Nov 2000 08:43:48 EST."
             <14886.22932.50399.879012@cj42289-a.reston1.va.home.com> 
References: <14886.22351.263040.575270@cj42289-a.reston1.va.home.com> <Pine.GSO.4.10.10011301542250.4645-100000@sundial>  <14886.22932.50399.879012@cj42289-a.reston1.va.home.com> 
Message-ID: <20001130220114.56E85A9BB@darjeeling.zadka.site.co.il>

>   Given Guido's surprise at the checkin, you might want to wait until
> the discussions are over and just make one set of checkins, with all
> the results.  ;)

Hence my not doing it right now.

-- 
Moshe Zadka <sig at zadka.site.co.il>
This is a signature anti-virus. 
Please stop the spread of signature viruses!



From mal at lemburg.com  Thu Nov 30 15:00:36 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 15:00:36 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk>  
	            <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>
Message-ID: <3A265D84.9A80267A@lemburg.com>

Guido van Rossum wrote:
> 
> > Just curious: how would you get at the deleted value when
> > using .popfirstkey() ?
> >
> > But you have a point: dict.pop() returning the first filled slot
> > in the dictionary as tuple (key,value) would probably be most
> > effective. It's also nice and short.
> 
> Nice!
>
> > BTW, I don't get the inspiration for the "first" part in those
> > names... it makes you think that there's an order to dictionary
> > items when in reality there isn't.
> 
> Actually there is -- the (arbitrary) order revealed by .keys().

Hmm, but this can change if you modify the dictionary inside the
loop or if you modify the dictionary from some other thread.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From fredrik at pythonware.com  Thu Nov 30 15:02:17 2000
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 30 Nov 2000 15:02:17 +0100
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <02d301c05ad6$26851150$0900a8c0@SPIFF>

thanks, mal !

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

+1 on "pop" (short for dist.items().pop())

> BTW, I don't get the inspiration for the "first" part in those
> names... it makes you think that there's an order to dictionary
> items when in reality there isn't.

-0 on "first" (on the other hand, are we really
allowed to vote on pronouncements? ;-)

</F>




From mwh21 at cam.ac.uk  Thu Nov 30 15:06:58 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 14:06:58 +0000
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 14:40:12 +0100"
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com>
Message-ID: <m3pujd5zb1.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:
> Michael Hudson wrote:
> > I must admit I can't think of a situation where .firstkey would be
> > more useful than .popkey; I thought that this discussion was
> > discussion was inspired by the wish to write code like:
> > 
> > while not set.empty():
> >     x = set.popfirstkey()
> >     ...
> > 
> > Maybe I'm jsut being dense, though.
> 
> Just curious: how would you get at the deleted value when
> using .popfirstkey() ?

You wouldn't.  I was thinking of the dict-as-sets code that was being
flung around earlier.

> But you have a point: dict.pop() returning the first filled slot
> in the dictionary as tuple (key,value) would probably be most
> effective. It's also nice and short.

Yes.

> > FWIW, I think
> > 
> >     dict.firstkey()
> > 
> > reads better than
> > 
> >     dict.firstitem()[0]
> > 
> > if the former's what you're actually trying to say.  Why not write
> > 
> >     for k,_ in dict.items():
> >         ...
> > 
> > ? (Overstating the case slightly for effect, but I hope the point is
> > clear).
> 
> No, the point is that you can modify the dictionary within
> the loop. You wouldn't see any changes using the second technique.

I was trying to make an analogy between .firstkey/.firstitem and
.keys/.items - the 

    for k,_ in dict.items():
        ...

was meant to be compared to 

    for k in dict.keys():
        ...

Sorry for being obscure.

Cheers,
M.

-- 
  "declare"?  my bogometer indicates that you're really programming
  in some other language and trying  to force Common Lisp into your
  mindset.  this won't work.            -- Erik Naggum, comp.lang.lisp




From guido at python.org  Thu Nov 30 15:10:34 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 09:10:34 -0500
Subject: [Python-Dev] {}.getitem() (was Re: PEP-0218)
In-Reply-To: Your message of "Thu, 30 Nov 2000 15:00:36 +0100."
             <3A265D84.9A80267A@lemburg.com> 
References: <m141PyD-000Dm8C@artcom0.artcom-gmbh.de> <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m3sno961mr.fsf@atrus.jesus.cam.ac.uk> <3A2658BC.7922B8B1@lemburg.com> <200011301352.IAA26550@cj20424-a.reston1.va.home.com>  
            <3A265D84.9A80267A@lemburg.com> 
Message-ID: <200011301410.JAA26719@cj20424-a.reston1.va.home.com>

> > > BTW, I don't get the inspiration for the "first" part in those
> > > names... it makes you think that there's an order to dictionary
> > > items when in reality there isn't.
> > 
> > Actually there is -- the (arbitrary) order revealed by .keys().
> 
> Hmm, but this can change if you modify the dictionary inside the
> loop or if you modify the dictionary from some other thread.

Let me explain.  When Moshe first proposed a .key() method, I was a
bit confused.  I thought he wanted a way to get to the i-th key (which
can't be done in O(1) time).  Then I thought he wanted a way to get a
random key.  Finally I realized that all he wanted was *a* key, and
the first key in the hash table would work fine.  Since that's also
the first key of .keys(), I figured "first" was an appropriate
qualification.  Sure, it's not guaranteed to remain the first key, but
neither is the first item in a list.

That said, getting rid of the first*() suite and adding popitem()
instead sounds attractive...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Thu Nov 30 16:14:53 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:14:53 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com>
	<m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>
Message-ID: <14886.28397.991041.190430@anthem.concentric.net>

>>>>> "PF" == Peter Funk <pf at artcom-gmbh.de> writes:

    PF> Since [].pop() takes in optional index as parameter which
    PF> defaults to the *LAST* element, it would be better to make the
    PF> name explicit: .popfirstitem()

{}.pop() sounds like the right thing given its symmetry with [].pop().

It would have the semantics that it would remove and return a random
/item/ from the dictionary, which the implementation can make the
"first" item for convenience purposes.

It shouldn't dictate though, that {}.pop() == dict.items().pop() or
make any other guarantees about which item gets popped, because other
implementations (or indeed, other mapping-conformant objects) may use
different underlying representations that could make this inefficient.

For added symmetry, {}.pop() should take an optional argument which is
the key of the item to remove and return.

-Barry



From barry at digicool.com  Thu Nov 30 16:23:21 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 10:23:21 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
	<200011301240.HAA26261@cj20424-a.reston1.va.home.com>
Message-ID: <14886.28905.691007.74511@anthem.concentric.net>

>>>>> "GvR" == Guido van Rossum <guido at python.org> writes:

    GvR> Aargh!  I thought I told you to submit a patch (to SF), not
    GvR> to check it in.

    GvR> The code looks okay but we'll have to discuss whether we
    GvR> really want all three (as opposed to only firstitem()).  I'm
    GvR> also not sure whether KeyError might be a better exception.

    GvR> Don't touch it for now, but let's think this over a bit.

And don't forget that when we /do/ decide on the right thing, that
UserDict needs to map <ha ha> the change too.

-Barry



From guido at python.org  Thu Nov 30 16:22:06 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 10:22:06 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: Your message of "Thu, 30 Nov 2000 10:14:53 EST."
             <14886.28397.991041.190430@anthem.concentric.net> 
References: <200011301110.GAA25768@cj20424-a.reston1.va.home.com> <m141Rxy-000Dm8C@artcom0.artcom-gmbh.de>  
            <14886.28397.991041.190430@anthem.concentric.net> 
Message-ID: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>

> {}.pop() sounds like the right thing given its symmetry with [].pop().
> 
> It would have the semantics that it would remove and return a random
> /item/ from the dictionary, which the implementation can make the
> "first" item for convenience purposes.
> 
> It shouldn't dictate though, that {}.pop() == dict.items().pop() or
> make any other guarantees about which item gets popped, because other
> implementations (or indeed, other mapping-conformant objects) may use
> different underlying representations that could make this inefficient.
> 
> For added symmetry, {}.pop() should take an optional argument which is
> the key of the item to remove and return.

That seems a nice touch, until you realize that [].pop() returns the
value, while {}.pop() returns a (key, value) pair.  Also: if you
already know the key, the argument that you can't do it in O(1) time
is no longer valid.  So, a -0.5.

One more concern: if you repeatedly remove the *first* item, the hash
table will start looking lobsided.  Since we don't resize the hash
table on deletes, maybe picking an item at random (but not using an
expensive random generator!) would be better.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From gvwilson at nevex.com  Thu Nov 30 16:21:24 2000
From: gvwilson at nevex.com (Greg Wilson)
Date: Thu, 30 Nov 2000 10:21:24 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <14886.28397.991041.190430@anthem.concentric.net>
Message-ID: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>

> Barry Warsaw wrote:
> {}.pop() sounds like the right thing given its symmetry with [].pop().
> [details]

Should we then add dict.push(), which would add a key/value pair to the
dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
but I think anyone who's gone through a CS course in data structures
and algorithms will expect push() where there's pop(), and it will allow
functions to be polymorphic over lists and dicts.

Greg (who's not lobbying for it, just wondering if it's a loose end)



From mwh21 at cam.ac.uk  Thu Nov 30 16:35:31 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: Thu, 30 Nov 2000 15:35:31 +0000 (GMT)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re:
 {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <Pine.LNX.4.10.10011301534070.27976-100000@localhost.localdomain>

On Thu, 30 Nov 2000, Greg Wilson wrote:

> > Barry Warsaw wrote:
> > {}.pop() sounds like the right thing given its symmetry with [].pop().
> > [details]
> 
> Should we then add dict.push(), which would add a key/value pair to the
> dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),
> but I think anyone who's gone through a CS course in data structures
> and algorithms will expect push() where there's pop(), and it will allow
> functions to be polymorphic over lists and dicts.

'cept there's no list.push - and I don't think you're proposing spelling
"dict.push" "dict.append", are you?

Cheers,
M.




From fdrake at acm.org  Thu Nov 30 16:34:50 2000
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 30 Nov 2000 10:34:50 -0500 (EST)
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
References: <14886.28397.991041.190430@anthem.concentric.net>
	<NEBBIACCCGNFLECLCLMHGEOICAAA.gvwilson@nevex.com>
Message-ID: <14886.29594.947340.480556@cj42289-a.reston1.va.home.com>

Greg Wilson writes:
 > Should we then add dict.push(), which would add a key/value pair to the
 > dictionary?  I realize it's redundant (same effect as 'dict[x] = y'),

  We don't have [].push(), and I don't think the metaphor really works
well with dictionaries.  There's also an implied ordering relationship
between push() and pop(), and this doesn't hold for dictionaries
either.

 > but I think anyone who's gone through a CS course in data structures
 > and algorithms will expect push() where there's pop(), and it will allow
 > functions to be polymorphic over lists and dicts.

  There's only a limited amount on conceptual polymorphism between the
two; artificial extending that may lead to a lot of programmers using
the wrong structure for their application.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Digital Creations




From gstein at lyra.org  Thu Nov 30 18:08:10 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 09:08:10 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <200011301231.EAA22228@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Thu, Nov 30, 2000 at 04:31:07AM -0800
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
Message-ID: <20001130090810.A25840@lyra.org>

On Thu, Nov 30, 2000 at 04:31:07AM -0800, Moshe Zadka wrote:
>...
> *** dictobject.c	2000/09/01 23:29:27	2.65
> --- dictobject.c	2000/11/30 12:31:00	2.66
> ***************
> *** 710,713 ****
> --- 710,782 ----
>   
>   static PyObject *
> + dict_firstkey(register dictobject *mp, PyObject *args)
> + {
> + 	register int i;
> + 
> + 	if (!PyArg_NoArgs(args))
> + 		return NULL;

PyArg_NoArgs() is deprecated because you cannot specify the method name. The
code should be:

    if (!PyArg_ParseTuple(args, ":firstkey"))
        return NULL;

(same in the next two funcs)

>...
> + 		if (mp->ma_table[i].me_value != NULL) {
> + 			PyObject *key = mp->ma_table[i].me_key;
> + 			PyObject *value = mp->ma_table[i].me_value;
> + 			PyObject *item = PyTuple_New(2);
> + 			if (item == NULL) {
> + 				return NULL;
> + 			}
> + 			Py_INCREF(key);
> + 			PyTuple_SetItem(item, 0, key);
> + 			Py_INCREF(value);
> + 			PyTuple_SetItem(item, 1, value);

These can be PyTuple_SET_ITEM() since you know you have a tuple.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From jeremy at alum.mit.edu  Thu Nov 30 18:23:04 2000
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 30 Nov 2000 12:23:04 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130090810.A25840@lyra.org>
References: <200011301231.EAA22228@slayer.i.sourceforge.net>
	<20001130090810.A25840@lyra.org>
Message-ID: <14886.36088.34790.149929@bitdiddle.concentric.net>

Let's back out these changes and put them in a patch.  There is still
a lot of discussion going on.  The patch itself has some small
problems.  And it is incomplete -- no test cases, no documentation,
etc.

Jeremy



From guido at python.org  Thu Nov 30 18:39:16 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 12:39:16 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: Your message of "Thu, 30 Nov 2000 12:23:04 EST."
             <14886.36088.34790.149929@bitdiddle.concentric.net> 
References: <200011301231.EAA22228@slayer.i.sourceforge.net> <20001130090810.A25840@lyra.org>  
            <14886.36088.34790.149929@bitdiddle.concentric.net> 
Message-ID: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>

> Let's back out these changes and put them in a patch.  There is still
> a lot of discussion going on.  The patch itself has some small
> problems.  And it is incomplete -- no test cases, no documentation,
> etc.

Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
clever with cvs admin, just use -j.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From barry at digicool.com  Thu Nov 30 19:13:24 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 13:13:24 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
Message-ID: <14886.39108.175356.483917@anthem.concentric.net>

I was just surprised by something I've never observed before from
string.capitalize().  There is clearly either a bug (really two) in
the code or in the documentation.

>>> '__Myfoo'.capitalize()
'__myfoo'

Huh?  The documentation says:

    capitalize () 
       Return a copy of the string with only its first character capitalized.

So it's only doing one of the three things it's promising!  For one
thing, it's modifying something other than the first character, and
for another, it's actually swapping the case of the character it /is/
changing.  At least it returns a copy. :)

Given the documentation, I would have expected the return value to be
the same as the original string, i.e. unchanged.

So which is it?  Does the description of the method need to be made
more complicated, or does the code need to be simplified <wink>?

-Barry



From mal at lemburg.com  Thu Nov 30 19:19:29 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:19:29 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A269A31.35FE0AEF@lemburg.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize ()
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)
> 
> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

Huh ? The "documentation" says that you will get a copy in which
only the first character is capitalized (and all others are
converted to lower-case).

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

I'd say this is a documentation bug (and I'm authorized since
I wrote that snippet called "documentation" ;-).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mwh21 at cam.ac.uk  Thu Nov 30 19:21:42 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:21:42 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: barry@digicool.com's message of "Thu, 30 Nov 2000 13:13:24 -0500"
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>

barry at digicool.com (Barry A. Warsaw) writes:

> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:
> 
>     capitalize () 
>        Return a copy of the string with only its first character capitalized.
> 
> So it's only doing one of the three things it's promising!  For one
> thing, it's modifying something other than the first character, and
> for another, it's actually swapping the case of the character it /is/
> changing.  At least it returns a copy. :)

>>> 'AaAaAa'.capitalize()
'Aaaaaa'

That's not a ridiculous reading of the above docs.  It all depends
whether you think "being capitalized" is a property or an action, I
guess.

> Given the documentation, I would have expected the return value to be
> the same as the original string, i.e. unchanged.

I would probably have expected that, too.  But I'm not really sure
why.

> So which is it?  Does the description of the method need to be made
> more complicated, or does the code need to be simplified <wink>?

Clarifying the docs won't break any code.  Not sure that changing the
code will much, either.

Oooh, here's something a bit more serious though:

>>> u'aAaAaA'.capitalize()
u'AAaAaA'

Something obviously Needs To Be Done.  My hunch is to change
string_capitalize, but that may be just me (and probably Barry).

Cheers,
M.

-- 
  Famous remarks are very seldom quoted correctly.
                                                    -- Simeon Strunsky




From mal at lemburg.com  Thu Nov 30 19:38:04 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:38:04 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A269E8C.6E1E0B05@lemburg.com>

> Oooh, here's something a bit more serious though:
> 
> >>> u'aAaAaA'.capitalize()
> u'AAaAaA'
> 
> Something obviously Needs To Be Done.  My hunch is to change
> string_capitalize, but that may be just me (and probably Barry).

Ouch. That's a bug.

Here's what the string.py module has to say about this BTW:

# Capitalize a string, e.g. "aBc  dEf" -> "Abc  def".
def capitalize(s):
    """capitalize(s) -> string

    Return a copy of the string s with only its first character
    capitalized.

    """
    return s.capitalize()


Note that .title() is very similar to the way string.capitalize()
works. unicode.title() also uses the title case information available
for Unicode characters.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From mwh21 at cam.ac.uk  Thu Nov 30 19:47:37 2000
From: mwh21 at cam.ac.uk (Michael Hudson)
Date: 30 Nov 2000 18:47:37 +0000
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: "M.-A. Lemburg"'s message of "Thu, 30 Nov 2000 19:38:04 +0100"
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>
Message-ID: <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>

"M.-A. Lemburg" <mal at lemburg.com> writes:

> > Oooh, here's something a bit more serious though:
> > 
> > >>> u'aAaAaA'.capitalize()
> > u'AAaAaA'
> > 
> > Something obviously Needs To Be Done.  My hunch is to change
> > string_capitalize, but that may be just me (and probably Barry).
> 
> Ouch. That's a bug.

Yes.
 
> Here's what the string.py module has to say about this BTW:

I said "string_capitalize", i.e. "stringobject.c:string_capitalize",
i.e I think 

>>> 'aAaA'.capitalize()

should result in

'AAaA'

We're not doing too well at understanding each other today, are we?

> Note that .title() is very similar to the way string.capitalize()
> works. unicode.title() also uses the title case information available
> for Unicode characters.

Ah yes.  So in the interests of reducing redunancy, .capitalize should
probably do something usefully different...

Cheers,
M.

-- 
59. In English every word can be verbed. Would that it were so in
    our programming languages.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html




From jim at interet.com  Thu Nov 30 19:49:28 2000
From: jim at interet.com (James C. Ahlstrom)
Date: Thu, 30 Nov 2000 13:49:28 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
Message-ID: <3A26A138.350235C@interet.com>

"Barry A. Warsaw" wrote:
> 
> I was just surprised by something I've never observed before from
> string.capitalize().  There is clearly either a bug (really two) in
> the code or in the documentation.
> 
> >>> '__Myfoo'.capitalize()
> '__myfoo'
> 
> Huh?  The documentation says:

I guess I am a total geek, but it seems right to me.  The
capital of "_" is "_", and the rest was converted to lower
case.

JimA



From moshez at math.huji.ac.il  Thu Nov 30 19:55:56 2000
From: moshez at math.huji.ac.il (Moshe Zadka)
Date: Thu, 30 Nov 2000 20:55:56 +0200 (IST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects
 dictobject.c,2.65,2.66
In-Reply-To: <200011301739.MAA27646@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>

On Thu, 30 Nov 2000, Guido van Rossum wrote:

> > Let's back out these changes and put them in a patch.  There is still
> > a lot of discussion going on.  The patch itself has some small
> > problems.  And it is incomplete -- no test cases, no documentation,
> > etc.
> 
> Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> clever with cvs admin, just use -j.

OK, I'll try. If CVS gets the best of me, I'll ask for your help...
--
Moshe Zadka <moshez at math.huji.ac.il> -- 95855124
http://advogato.org/person/moshez




From mal at lemburg.com  Thu Nov 30 19:56:45 2000
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu, 30 Nov 2000 19:56:45 +0100
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com> <m3itp55mba.fsf@atrus.jesus.cam.ac.uk>
Message-ID: <3A26A2ED.EFF487C3@lemburg.com>

Michael Hudson wrote:
> 
> "M.-A. Lemburg" <mal at lemburg.com> writes:
> 
> > > Oooh, here's something a bit more serious though:
> > >
> > > >>> u'aAaAaA'.capitalize()
> > > u'AAaAaA'
> > >
> > > Something obviously Needs To Be Done.  My hunch is to change
> > > string_capitalize, but that may be just me (and probably Barry).
> >
> > Ouch. That's a bug.
> 
> Yes.
> 
> > Here's what the string.py module has to say about this BTW:
> 
> I said "string_capitalize", i.e. "stringobject.c:string_capitalize",

The string methods were built closely after the string module
functions, so that's what the reference is...

> i.e I think
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'
> 
> We're not doing too well at understanding each other today, are we?

:)
 
> > Note that .title() is very similar to the way string.capitalize()
> > works. unicode.title() also uses the title case information available
> > for Unicode characters.
> 
> Ah yes.  So in the interests of reducing redunancy, .capitalize should
> probably do something usefully different...

Depends... we'll still need some sort of compatibility with
string.py.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/



From gstein at lyra.org  Thu Nov 30 20:12:54 2000
From: gstein at lyra.org (Greg Stein)
Date: Thu, 30 Nov 2000 11:12:54 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <Pine.GSO.4.10.10011302055290.20698-100000@sundial>; from moshez@math.huji.ac.il on Thu, Nov 30, 2000 at 08:55:56PM +0200
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial>
Message-ID: <20001130111254.N25840@lyra.org>

On Thu, Nov 30, 2000 at 08:55:56PM +0200, Moshe Zadka wrote:
> On Thu, 30 Nov 2000, Guido van Rossum wrote:
> 
> > > Let's back out these changes and put them in a patch.  There is still
> > > a lot of discussion going on.  The patch itself has some small
> > > problems.  And it is incomplete -- no test cases, no documentation,
> > > etc.
> > 
> > Agreed.  Sorry, Moshe.  Can you do this yourself?  Don't try to be
> > clever with cvs admin, just use -j.
> 
> OK, I'll try. If CVS gets the best of me, I'll ask for your help...

Not sure what -j does, but you can just do a reverse diff. For example:

$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

Apply the patch to your working copy, then check it in.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/



From akuchlin at mems-exchange.org  Thu Nov 30 20:18:29 2000
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 30 Nov 2000 14:18:29 -0500
Subject: [Python-Dev] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
In-Reply-To: <20001130111254.N25840@lyra.org>; from gstein@lyra.org on Thu, Nov 30, 2000 at 11:12:54AM -0800
References: <200011301739.MAA27646@cj20424-a.reston1.va.home.com> <Pine.GSO.4.10.10011302055290.20698-100000@sundial> <20001130111254.N25840@lyra.org>
Message-ID: <20001130141829.B20259@kronos.cnri.reston.va.us>

On Thu, Nov 30, 2000 at 11:12:54AM -0800, Greg Stein wrote:
>Not sure what -j does, but you can just do a reverse diff. For example:
>
>$ cvs diff -r2.66 -r2.65 dictobject.c > reverse.patch

-j basically does the work for you; you could do:
$ cvs update -j2.66 -j2.65 dictobject.c 

And then do a commit.  See the CVS book: 
http://cvsbook.red-bean.com/cvsbook.html#Examining%20And%20Reverting%20Changes

--amk



From barry at digicool.com  Thu Nov 30 20:31:00 2000
From: barry at digicool.com (Barry A. Warsaw)
Date: Thu, 30 Nov 2000 14:31:00 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
References: <14886.39108.175356.483917@anthem.concentric.net>
	<3A269A31.35FE0AEF@lemburg.com>
Message-ID: <14886.43764.472482.888678@anthem.concentric.net>

    MAL> Huh ? The "documentation" says that you will get a copy in
    MAL> which only the first character is capitalized (and all others
    MAL> are converted to lower-case).

I think if the documentation actually said exactly that, I wouldn't
have been confused. :)

>>>>> "MH" == Michael Hudson <mwh21 at cam.ac.uk> writes:

    MH> That's not a ridiculous reading of the above docs.  It all
    MH> depends whether you think "being capitalized" is a property or
    MH> an action, I guess.

Good way to look at it.

I vote to clarify the documentation.

-Barry



From guido at python.org  Thu Nov 30 20:53:23 2000
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2000 14:53:23 -0500
Subject: [Python-Dev] surprising bug in s.capitalize()?
In-Reply-To: Your message of "30 Nov 2000 18:47:37 GMT."
             <m3itp55mba.fsf@atrus.jesus.cam.ac.uk> 
References: <14886.39108.175356.483917@anthem.concentric.net> <m3lmu15nih.fsf@atrus.jesus.cam.ac.uk> <3A269E8C.6E1E0B05@lemburg.com>  
            <m3itp55mba.fsf@atrus.jesus.cam.ac.uk> 
Message-ID: <200011301953.OAA27996@cj20424-a.reston1.va.home.com>

> i.e I think 
> 
> >>> 'aAaA'.capitalize()
> 
> should result in
> 
> 'AAaA'

No.  The capitalize() method, which descends from the
string.capitalize(), and which definitely should return 'Aaaa' here.

This means that (a) the documentation needs some clarification, and
(b) the Unicode capitalize() method is broken and needs to be fixed.

--Guido van Rossum (home page: http://www.python.org/~guido/)




From tim.one at home.com  Thu Nov 30 23:46:52 2000
From: tim.one at home.com (Tim Peters)
Date: Thu, 30 Nov 2000 17:46:52 -0500
Subject: [Python-Dev] {}.first[key,value,item] and {}.pop (was Re: {}.getitem())
In-Reply-To: <200011301522.KAA27116@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEADICAA.tim.one@home.com>

[Guido]
> ...
> One more concern: if you repeatedly remove the *first* item, the hash
> table will start looking lobsided.  Since we don't resize the hash
> table on deletes, maybe picking an item at random (but not using an
> expensive random generator!) would be better.

Which is the reason SETL doesn't specify *which* set item is removed:  if
you always start looking at "the front" of a dict that's being consumed, the
dict fills with turds without shrinking, you skip over them again and again,
and consuming the entire dict is still quadratic time.

Unfortunately, while using a random start point is almost always quicker
than that, the expected time for consuming the whole dict remains quadratic.

The clearest way around that is to save a per-dict search finger, recording
where the last search left off.  Start from its current value.  Failure if
it wraps around.  This is linear time in non-pathological cases (a
pathological case is one in which it isn't linear time <wink>).




From gregor at hoffleit.de  Sat Nov  4 00:14:15 2000
From: gregor at hoffleit.de (Gregor Hoffleit)
Date: Sat, 4 Nov 2000 00:14:15 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Include pyport.h,2.20,2.21
In-Reply-To: <200010050142.SAA08326@slayer.i.sourceforge.net>; from tim_one@users.sourceforge.net on Wed, Oct 04, 2000 at 06:42:32PM -0700
References: <200010050142.SAA08326@slayer.i.sourceforge.net>
Message-ID: <20001104001415.A2093@53b.hoffleit.de>

FYI: This misdefinition with LONG_BIT was due to a bug in glibc's limits.h. It
has been fixed in glibc 2.96.

    Gregor
    


On Wed, Oct 04, 2000 at 06:42:32PM -0700, Tim Peters wrote:
> Update of /cvsroot/python/python/dist/src/Include
> In directory slayer.i.sourceforge.net:/tmp/cvs-serv5758/python/dist/src/Include
> 
> Modified Files:
> 	pyport.h 
> Log Message:
> Move LONG_BIT from intobject.c to pyport.h.  #error if it's already been
> #define'd to an unreasonable value (several recent gcc systems have
> misdefined it, causing bogus overflows in integer multiplication).  Nuke
> CHAR_BIT entirely.