From esteban@ccpgames.com  Tue Oct  1 05:54:46 2002
From: esteban@ccpgames.com (Esteban U.C.. Castro)
Date: Tue, 1 Oct 2002 04:54:46 -0000
Subject: [Python-Dev] proposal for interfaces
Message-ID: <25A39AFEB31B06408C675CEF28199E5B073CAA@postur.ccp.cc>

In case you have missed the link (I did, at first); this is Guido's
opinion on the interfaces PEP as of March 7, 2002.

http://www.python.org/doc/essays/pepparade.html

Jim Fulton has said that this PEP was premature. I agree. It introduces
a new keyword, 'interface', and I'm not yet convinced that that is
needed. On the other hand, the way this is currently done in Zope also
looks butt-ugly, so something may indeed be needed. I think that at some
point in the future when we have more experience with using interfaces
(especially in Zope 3) we'll go back to this PEP and see how much of it
we can use. Maybe there should be a special status "frozen" meaning not
rejected but also not under consideration for the near future? But with
a different meaning than Py-in-the-sky -- this PEP at least has lots of
concrete proposals and studies the consequences.=20



From martin@v.loewis.de  Tue Oct  1 07:07:22 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Tue, 1 Oct 2002 08:07:22 +0200
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
Message-ID: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>

In _tkinter.c, there is currently special code for Tcl 8.0 (no Tcl
objects) and Tcl 8.1 (no Unicode type). Tcl 8.2, 8.3, and 8.4 are
supported uniformly.

Can we drop support for those two releases, in Python 2.3?

Regards,
Martin


From drifty@bigfoot.com  Tue Oct  1 08:44:09 2002
From: drifty@bigfoot.com (Brett Cannon)
Date: Tue, 1 Oct 2002 00:44:09 -0700 (PDT)
Subject: [Python-Dev] Python-dev Summary for 2002-09-16 to 2002-09-30
Message-ID: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>

Once again, here is the rough draft for the next Summary.  Usual review
deal (24 hours, blah blah).

Have fun.


========================================
`flextype.c -- extended type system`__
========================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028736.html


(continuation from `last summary`_)

To clarify what Christian Tismer's metatype patch basically implements, he
said that his "additions support a subset of C++ virtual methods".  And in
response to a question from me about how this might affect magic methods,
he said he "*could* support any magic slot and put it into the extended
type object with a Python name. And even better, this version could have
full type checking, as my other methods have as well!"

Martin v. Loewis doesn't see the use of this because extensions in C are
usually just wrappers and C++ extensions already have the functionality.
Thomas Heller, though, likes it because he wants a way "to add additional
C accessible structure fields to types. Currently [he is] attaching them
to the the type's dict, as I reported before" by David Abrahams.

As an aside, a suggestion to David about having to put instance-specific
info in __dict__, Thomas said "You can (but you probably know this
already) replace the type's tp_dict by a custom subclass of PyDict_Object,
which adds additional fields."

.. _last summary:
http://www.ocf.berkeley.edu/python-dev/summaries/2002-09-01--2002-09-15.html

==================================
`Moratorium on changes to IDLE`__
==================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028776.html

Guido said that he would "like to put a stop to all changes to the version
of IDLE in the Python source tree (`Tools/idle/*`_ -- let's call it
Python-idle)".  All changes should go through IDLEfork_ .  It will then
get merged into CVS before 2.3 is released.  Currently Idlefork is working
with 2.2 and Guido would like to keep it that way so that more people use
IDLE.

.. _Tools/idle/*:
.. _IDLEfork:

===========================================
`Weeding out obsolete modules and Demos`__
===========================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028718.html

(continuation from `last summary`_)

Guido removed the obsolete SGI demos from Demo_ as mentioned from the last
summary.

If anyone else out there has any demos that they think are no longer
needed, submit a `SF patch`_ mentioning what you think should be removed
and why.

.. _Demo:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Demo/
.. _SF patch: http://sourceforge.net/tracker/group_id=5470&atid=305470

=======================
`ReST-ing Misc/NEWS`__
=======================
__ http://mail.python.org/pipermail/python-dev/2002-September/028799.html

Skip changed `Misc/NEWS`_ into a reST document.  Hopefully this will end
up somewhere on http://www.python.org so people can be kept very
up-to-date on what changes have been made in CVS_ .

.. _Misc/NEWS:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Misc/NEWS
.. _CVS: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/

====================================
`mysterious hangs in socket code`__
====================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028555.html

(continued from `last summary`_)

Still trying to weed out issues with threaded code that seems to act
single-threaded.  The suspect is still glibc's getaddrinfo()
implementation; POSIX says it is supposed to be thread-safe but not
everyone is believing that.

=====================================================
`ATTENTION!  Releasing Python 2.2.2 in a few weeks`__
=====================================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028806.html

Guido announced that he would like to get Python 2.2.2 out the door by
October 8.  PythonLabs is busy as it is and thus do not have time to
thoroughly make sure everything gets done.  What can you do to help?  Glad
you asked: "the best thing you can do is take your favorite set of modules
or core files and systematically backport anything that's clearly a bugfix
and backports easily.  Or you could simply make sure that your favorite
bugfix is backported".

If you don't have checkin, either team up with someone who does or request
checkin rights (decision will be based on reputation).

SF should not be used for saying what should and shouldn't be backported.
This should all be done through Python-dev.  Michael McLay suggest a wiki
to handle all of this (thank goodness; the amount of traffic this would
have generated would have killed me).  Skip said he could set one up
rather quickly and he did at http://manatee.mojam.com/py222wiki/ .

Laura Creighton mentioned what she planned to do for Python-in-a-Tie
(Py-tie).  Looks like the current plan is to make Python 2.2.2 the initial
release and add in things from there based on developer feedback instead
of taking future releases and removing what they don't need.  Perhaps
Py-tie will end up taking the place of the elusive `Sumo distro`_?

Ralf W. Grosse-Kunstleve brought up how a bug report about C++
compatibility had not been handled yet.  Both Tim and Guido said that if
he wanted a solution put in they would need a patch; neither of them are
C++ users.  C++ uses should keep that in mind when trying to get C++ stuff
into the core.

Gustavo Niemeyer said that he has a script for adding moin pages using
your favorite editor.  He also has a syntax highlight file for Vim.  All
of this can be found at http://moin.conectiva.com.br/EditMoin .

.. _Sumo distro: http://www.python.org/peps/pep-0206.html

================================
`Built-in functions, kw args`__
================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028811.html

Jp Calderone brought up the question of whether ParseTupleAndKeywords
should be used more often then ParseTuple in the core.

Guido pointed out that ParseTupleAndKeywords did not exist since the
beginning of time, it's slower, and much harder to use.  Guido basically
ended up saying use it if you want, it isn't required.

=======================
`Python for OpenVMS`__
=======================
__ http://mail.python.org/pipermail/python-dev/2002-September/028814.html

Skip Montanaro announced that Jean-Fran?ois Pi?ronne had ported Python
2.1.3 over to OpenVMS_ and had begun work on porting the 2.3 tree.

.. _OpenVMS: http://www.openvms.org/

================
`-zcombreloc`__
================
__ http://mail.python.org/pipermail/python-dev/2002-September/028822.html

In Andrew Koenig's continual quest to get Python to build with Binutils
2.13 under Solaris, Andrew discovered through discussions with the
Binutils developers "that -zcombreloc just plain doesn't work in any
release of binutils".  Andrew thinks this will get fixed in Binutils
2.13.1 .

But what about until then?  Well, you are out of luck on Solaris with
Binutils 2.13 .  There is a test you can run to see if you will be
affected by this bug in this email (link to email).

=======================================
`binutils/solaris -- one more thing`__
=======================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028826.html

Andrew Koenig discovered another issue with Solaris/Binutils.  In Binutils
2.12, the output from ``ld -V`` which is used to find out if dynamic
linking is available changed and broke the test.  It turns out Martin v.
Loewis already committed a patch to Python 2.2.2 and 2.3 that deals with
this issue.

===================================================
`Need advice: cloning python cvs for CE project`__
===================================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028842.html

Brad Clements of the "python ce" project asked for advice on how to
"maintain a single working directory that can be checked into two
different CVS systems".  Guido said they could have a branch in the CVS
tree if they were all Python developers "or can be sworn in easily".  The
other option Guido presented was for them to start their own SF project
and do a vendor branch checking; this is how IDLE-fork does it.

Martin gave some instructions on how to do this in an email which can be
reached at XXX .

=============================
`Assign to errno allowed?`__
=============================
__ http://mail.python.org/pipermail/python-dev/2002-September/028868.html

Thomas Heller was patching selectmodule.c_ for Windows to deal with the
fact that it does not set errno.  Thomas wondered if it would be okay to
set errno himself.  Everyone in the end said it was fine (even the man
page for errno).

Through the discussion over whether not setting errno was proper, it was
pointed out by Thomas that the entire reason he was fixing this was for
asyncore_ to work; apparently it has always been broken on Windows.  Guido
suggested a simple fix to the module to make it work.

The other point that came up that Windows CE and NetWare don't allow for
direct signing to errno; ``errno = 0`` is an error on those platforms.
Brad Clements submitted a patch to add a macro to Python for setting
errno.  Well, Tim pointed out that it is an ISO C requirement that errno
be assignable.  This led Guido to "have a very strong urge to tell [Brad's
patch] to go away".  Let me say that from my personal experience on
Python-dev I have noticed that Python-dev does not like work-arounds for
things that don't work because something does not follow a standard; this
is especially true for anything dealing with ISO C.  So if you have a
problem and it is because your platform is not standards-compliant, I
would not expect to get much sympathy from Python-dev (unless it is a
Windows desktop OS =).

In the end it was suggested to the CE guys to just keep a branch of the
CVS where they fixed the whole errno issue.  That way they can just sync
with the core and not have to go through and change all of the errno lines
every time they sync.

This thread even split into another one entitled `Reserved keywords in
source`_ in which Brad Clements pointed out that ``finally`` is a reserved
word on Windows CE.  This, once again, flies in the face of ISO C.

.. _selectmodule.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/selectmodule.c
.. _asyncore:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/asyncore.py
.. _Reserved keywords in source:
http://mail.python.org/pipermail/python-dev/2002-September/028912.html

=============
`sorted()`__
=============
__ http://mail.python.org/pipermail/python-dev/2002-September/028925.html

Francois Pinard brought up the point that most beginners don't realize
that ``.sort()`` does not return anything; the sorting is done in-place.
He thought that adding a ``.sorted()`` method to lists that would work
just like ``.sort()`` but also return the list would be nice.  Would not
only possibly simplify some code by allowing a sort method to be called in
the header of a ``for`` loop, but possibly prevent newbie errors as
previously mentioned.

Greg Ewing suggested a ``sorted()`` function instead that would basically
do what Francois wanted.  This causeed Timothy Delaney to suggest possible
magic methods ``__sort__`` and ``__isort__``.

Guido responded saying that "this is a problem that should not be
addressed by changes to the language, builtins or library".  So there you
go, the BDFL has spoken.

==============================
`Lib/plat-xxxx directories`__
==============================
__ http://mail.python.org/pipermail/python-dev/2002-September/028933.html

In case people are interested, the Lib/plat-xxx directories are "for
platform-specific modules".  Most of them, apparently, "are collections of
system constants generated by Tools/scripts/h2py.py" .

.. _Tools/scripts/h2py.py:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Tools/scripts/h2py.py

==================================
`How to add an encoding alias?`__
==================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028936.html

For the spambayes_ project, Guido asked how to add an encoding to
``encodings.aliases``.  He wondered if there was an official API or if::

	import encodings.aliases
	encodings.aliases.aliases['ansi-x3-4-1968'] = 'ascii'  # This is
wrong!  Read on for why...

was the way to do it.

M.A. Lemburg pointed responded that the above-mentioned way is the only
way to do it in 2.2 .  He also mentioned that hyphens must be changed to
underscores for names.  MAL suggested that all non-alphanumeric characters
be mapped to underscores before referring to the aliases dictionary.

Guido voted +1 for the change and +1 to backporting to 2.2.2 .

.. _spambayes: http://spambayes.sf.net

========================================
`User extendable literal modifiers?!`__
========================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028945.html

M.A. Lemburg asked what everyone thought of having the ability to add new
literal modifiers (e.g. ``L`` in ``123L``) to the interpreter.  Gareth
McCaughan suggest "uder-definable *prefixes*".  His example is of Common
Lisp's ability to have you define something as ``#<character><anything>``
and then parse it based on <character>.  Alex Martelli liked the idea and
suggested that ``@`` be the proposed symbol instead of ``$`` as originally
suggested.  Alex also said that being able to parse multiple tokens would
be important.

Jeff Epler pointed out that this could all be done right now with some
ugly syntax::

	class Dimension: ...

	class DimensionMaker:
		def __call__(self, v):
			return Dimension(v)

		def __add__(self, v):
			return Dimension(v)

	D = DimensionMaker()

	>>> D+"123"  # Can be considered same as defining D"123"

Guido said that "Given all the discussion, this will need a PEP".  He
later pointed out that a possible problem is that if you are not very
familiar with a program you might probably won't know what a specific
prefix is.  With a function the worst would be having to look through a
module that was imported with ``from <module> import *`` ("that's one
reason to avoid these" types of imports).  MAL said that Guido was
"probably right" about this possible confusion.

===========================================
`Keyword for first argument of methods?`__
===========================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028960.html

David Abrahams noticed that if he defined the code::

	class X:
		def foo(self, y): print y

that ``X.foo(y=1, self=X())`` fails with a TypeError.

Guido said that "You can't pass in self to an unbound method as a keyword
argument".  Best way to have an unbound method is with __call__ .  I think
using something like the new staticmethod() or classmethod() functions in
Python 2.2 would get  the job done.

============================================
`Strange bug only happens with Python 2.2`__
============================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028966.html

Gerhard Haring discovered a funky bug dealing with an error saying that a
custom object was a string and lacking a __bases__ attribute.  What is
pertinent to this thread is not the bug (it got fixed in Python 2.3 and
has been backported to 2.2.2), but what caused the bug.

In abstract.c there was some code that forgot to call PyErr_Clear() .
When your extension code catches an error and you decide to either ignore
it or go another direction, you must call this function in order to
prevent the exception from propogating to the top when the call is
finished and raising the exception in the end.

=================================================
`builtins instance have modifiable __class__?`__
=================================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028978.html

Samuele Pedroni asked if being able to directly assigning to __class__ was
allowed.  Guido said "In 2.3, it's disallowed".  It is highly discouraged
in 2.2.2 but  still doable.

==========================================================
`Why are useful tools omitted from the Win bin distro?`__
==========================================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028988.html

Alex Martelli asked why the Windows distribution lacked some of the Tools
subdirectory.  Most of the scripts are left out since they are meant for
developers. Since if you are using the pre-packaged Windows installer
chances are you are not a developer since you would have compiled it. Thus
these scripts are left out.

Tim Peters (who maintains the WISE installer) said that in order to add a
script he would have to add an individual line to the installer for each
file; a definite PITA (Pain In The Ass) for Tim.

There have been calls (which still stand) to document, fix, and clean out
the Tools_ and Demo_ directories.  No one has ever really stepped up to
take this on.  Any help, though, no matter how small, would be appreciated
in cleaning these directories up.

.. _Tools:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Tools/

=======================================
`Extension module difficulty w/pgen`__
=======================================
__ http://mail.python.org/pipermail/python-dev/2002-September/028976.html

Jonathan Riehl was having difficulty getting his extension module that
exposes pgen to link properly.  He got it to work with a little
command-line hack; that does not work for the official distro and thus he
was searching for a more proper solution.  He discovered that if he put
dummy calls into a global function that seemed to allow it to be linked.
Jonathan hoped there was a better solution.  Guido, unfortunately,
destroyed that hope by stating "AFAIK you have to create a dummy reference
somewhere".  He suggested at the end of `Python/pythonrun.c`_ to Jonathan.

.. _Python/pythonrun.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/pythonrun.c

=========================
`New logmerge feature`__
=========================
__ http://mail.python.org/pipermail/python-dev/2002-September/028998.html

Guido added a new feature to `Tools/logmerge.py`_ that "restricts the
output to a specific branch tag".  This is  useful if you are helping to
sort through all of the commits to Python 2.3 to see what should go into
2.2.2 .

.. _Tools/logmerge.py:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Tools/logmerge.py

==========================================
`Snapshot win32all builds of interest?`__
==========================================
__ http://mail.python.org/pipermail/python-dev/2002-September/029005.html

Mark Hammond was wondering if people would be interested in him "making
regular, basically untested win32all builds against the current Python CVS
tree".  If you would like Mark to do this, let him know.





From pedronis@bluewin.ch  Tue Oct  1 12:48:44 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 1 Oct 2002 13:48:44 +0200
Subject: [Python-Dev] Python-dev Summary for 2002-09-16 to 2002-09-30
References: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>
Message-ID: <006d01c26940$7f3999e0$6d94fea9@newmexico>

> =================================================
> `builtins instance have modifiable __class__?`__
> =================================================
> __ http://mail.python.org/pipermail/python-dev/2002-September/028978.html
>
> Samuele Pedroni asked if being able to directly assigning to __class__ was
> allowed.  Guido said "In 2.3, it's disallowed".  It is highly discouraged
> in 2.2.2 but  still doable.
>

my question and the limitation refer only to builtin types (int,dict ...), for
instances of (user-defined) subtypes __class__ will be still modifiable.

regards.



From skip@pobox.com  Tue Oct  1 13:41:29 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 1 Oct 2002 07:41:29 -0500
Subject: [Python-Dev] Python-dev Summary for 2002-09-16 to 2002-09-30
In-Reply-To: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>
Message-ID: <15769.38905.853668.772022@12-248-11-90.client.attbi.com>

    Brett> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
    Brett> `Python for OpenVMS`__
    Brett> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
    Brett> __ http://mail.python.org/pipermail/python-dev/2002-Septembe=
r/028814.html

    Brett> Skip Montanaro announced that Jean-Fran?ois Pi?ronne had por=
ted
    Brett> Python 2.1.3 over to OpenVMS_ and had begun work on porting =
the
    Brett> 2.3 tree.

    Brett> .. _OpenVMS: http://www.openvms.org/

Looks like your text editor or mailer has trouble with accented charact=
ers.
It's "Jean-Fran=E7ois Pi=E9ronne". ;-) Also, Jean-Fran=E7ois recently s=
ubmitted
some patches for 2.3 against the code in CVS.

Skip


From neal@metaslash.com  Tue Oct  1 14:17:37 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 01 Oct 2002 09:17:37 -0400
Subject: [Python-Dev] Python-dev Summary for 2002-09-16 to 2002-09-30
References: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>
Message-ID: <3D99A071.B7436B18@metaslash.com>

Brett Cannon wrote:
>
> Ralf W. Grosse-Kunstleve brought up how a bug report about C++
> compatibility had not been handled yet.  Both Tim and Guido said that if
> he wanted a solution put in they would need a patch; neither of them are
> C++ users.  C++ uses should keep that in mind when trying to get C++ stuff
> into the core.

I've assigned the bug to me.  Ralf provided a patch and 
I will checkin/backport soon.  http://pyton.org/sf/607253

Neal


From guido@python.org  Tue Oct  1 14:38:47 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 01 Oct 2002 09:38:47 -0400
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: Your message of "Tue, 01 Oct 2002 08:07:22 +0200."
 <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
Message-ID: <200210011338.g91Dclo08817@odiug.zope.com>

> In _tkinter.c, there is currently special code for Tcl 8.0 (no Tcl
> objects) and Tcl 8.1 (no Unicode type). Tcl 8.2, 8.3, and 8.4 are
> supported uniformly.
> 
> Can we drop support for those two releases, in Python 2.3?

If it's more work to keep support around than to drop it, sure.  I
agree there's probably nobody who's still using any Tcl version before
8.3 anyway; that's been out several years now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct  1 14:50:18 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 01 Oct 2002 09:50:18 -0400
Subject: [Python-Dev] Python-dev Summary for 2002-09-16 to 2002-09-30
In-Reply-To: Your message of "Tue, 01 Oct 2002 13:48:44 +0200."
 <006d01c26940$7f3999e0$6d94fea9@newmexico>
References: <Pine.SOL.4.44.0210010040390.16395-100000@death.OCF.Berkeley.EDU>
 <006d01c26940$7f3999e0$6d94fea9@newmexico>
Message-ID: <200210011350.g91DoIO08897@odiug.zope.com>

> > =================================================
> > `builtins instance have modifiable __class__?`__
> > =================================================
> > __ http://mail.python.org/pipermail/python-dev/2002-September/028978.html
> >
> > Samuele Pedroni asked if being able to directly assigning to __class__ was
> > allowed.  Guido said "In 2.3, it's disallowed".  It is highly discouraged
> > in 2.2.2 but  still doable.
> 
> my question and the limitation refer only to builtin types (int,dict
> ...), for instances of (user-defined) subtypes __class__ will be
> still modifiable.

Yes, setting __class__ for instances of user-defined new-style classes
will be allowed (if the new __class__ value is compatible with the old).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Tue Oct  1 16:17:05 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 01 Oct 2002 17:17:05 +0200
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: <200210011338.g91Dclo08817@odiug.zope.com>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
 <200210011338.g91Dclo08817@odiug.zope.com>
Message-ID: <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> If it's more work to keep support around than to drop it, sure.

I'm planning a number of changes which can be simplified if I don't
have that many versions to support.

Regards,
Martin



From guido@python.org  Tue Oct  1 16:19:58 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 01 Oct 2002 11:19:58 -0400
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: Your message of "Tue, 01 Oct 2002 17:17:05 +0200."
 <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de> <200210011338.g91Dclo08817@odiug.zope.com>
 <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210011519.g91FJwI21585@odiug.zope.com>

> > If it's more work to keep support around than to drop it, sure.
> 
> I'm planning a number of changes which can be simplified if I don't
> have that many versions to support.

OK, +1.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Tue Oct  1 18:48:11 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 1 Oct 2002 13:48:11 -0400
Subject: [Python-Dev] Re: proposal for interfaces
In-Reply-To: <20021001134101.18003.62032.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEOECLAA.gsw@agere.com>

I'm not sure why you'd need a new keyword for this feature when you can
simply create a class that can check if methods in a derived class have
been implemented. You can also check for other attributes if you use a
live object instead of a class. All it needs to do is raise an exception
if the interface isn't fully implemented.

Here is an (admittedly simplified) example:

    import types
    from exceptions import TypeError

    class Interface(object):
        def implemented_by(self,implementer):
            # Verify against class methods:
            methods = []
            for i in dir(self.__class__):
                if i not in dir(Interface):
                    methods.append(i)
            for i in methods:
                if i not in dir(implementer):
                    raise TypeError, "%s not implemented" % str(i)
                # and verify its type, parameters, ...

            # For objects, can also verify that attributes created in the
            # interface's __init__ exist:
            if type(implementer) not in (type(Interface), types.ClassType):
                # Assume it's an object. There's probably a better way.
                # (type(implementer) is types.ObjectType) works, but only
                # for new-style classes.
                methods += dir(Interface)
                attributes = []
                for i in dir(self):
                    if i not in methods:
                        attributes.append(i)
                for i in attributes:
                    if i not in dir(implementer):
                        raise TypeError, "%s not implemented" % str(i)

Something as simple as this would let you verify that a class or object
has implemented an interface. For example:

    class TestInterface(Interface):
        def __init__(self):
            # object attributes to look for:
            self.attrib = "Some attribute"
        # class methods to look for:
        def method(self):
            "Some method"

    # To verify only methods, check the class
    TestInterface().implemented_by(Foo)

    # To verify attributes, check an object
    TestInterface().implemented_by(Foo())

I don't know what Zope does, but this doesn't seem that ugly to me.

-Jerry


From barry@python.org  Tue Oct  1 19:18:27 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 1 Oct 2002 14:18:27 -0400
Subject: [Python-Dev] RELEASED email package version 2.4
Message-ID: <15769.59123.944370.58736@gargle.gargle.HOWL>

I've just released the (standalone) email package version 2.4.  This
is identical to the version that is in the current cvs development
tree for Python 2.3.  It is also the version that I plan to back port
to Python 2.2.2.  I've also done a massive update to the web site
documentation.  Thanks to Fred Drake, the latter is actually in a
usable format. :)

email 2.4 is made available as a distutils package for installing in
earlier versions of Python.  It is compatible with Python 2.1.3,
Python 2.2.1, and Python 2.3 cvs.  email 2.4 is backward compatible
with email version 1, which was released as part of Python 2.2.1.

The email package is a library for parsing, handling, and generating
email messages and other RFC 2822 style documents.  It conforms to
most of the email related RFCs including 2045-2047 (the MIME RFCs) and
2231.  It is intended to replace several older modules in the standard
distribution, such as rfc822, mimetools, multifile, mimify, and
MIMEWriter, and such non-standard modules as mimecntl.

email-2.4.tar.gz is available from the mimelib project on SourceForge:

    http://sf.net/projects/mimelib

More information, as well as on-line documentation is available on the
home page:

    http://mimelib.sf.net

As this is the version I plan to back port to Python 2.2.2 in a few
days, feedback, either on the code or the documentation, will be
welcome.

Enjoy,
-Barry


From pedronis@bluewin.ch  Tue Oct  1 20:59:00 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Tue, 1 Oct 2002 21:59:00 +0200
Subject: [Python-Dev] perplexed by mro
Message-ID: <01e101c26984$fc43da60$6d94fea9@newmexico>

I'm trying to wrap my head around the mro computation in 2.2.

First issue: PEP 253 and the descrintro document describe an algorithm,
typeobject.c implements a rather different one. Descrintro mentions that.

The algorithm described by descrintro, PEP253:

"Using the classic lookup rule, construct the list of classes that would be
searched, including duplicates. Now for each class that occurs in the list
multiple times, remove all occurrences except for the last."

which, if I understand correctly, means : do a left-right dfs of the class
hierarchy and then discard all but the last occurence of each class.

Now descrintro says that the result of this and the real algorithm in
typeobject are the same, unless

"two given base classes occur in a different order in the inheritance list of
two different derived classes, and those derived classes are both inherited by
yet another class"

But here is an example where the criterium does not apply but still the
algorithms' results diverge:

>>> cpl.ex5(globals())
class a(object): pass
class b(object): pass
class c(object): pass
class x(a): pass
class y(a): pass
class z(x,b,y,c): pass
>>> cpl.names(z.__mro__) # just the class names, real 2.2 mro algo
['z', 'x', 'y', 'a', 'b', 'c', 'object']
>>> cpl.names(cpl.keeplast(cpl.lrdfs(z))) # descrintro simple algo
['z', 'x', 'b', 'y', 'a', 'c', 'object']

I was not able to find a new rule. My intuition is that when the result of the
real mro has something to do with the dfs, it is sure from the beginning of it
not the end, so the relation of the two algos is unclear.

In general I have an hard time trying to predict the result of the real mro
algo.

Does it a have a list of constraints it enforces/preserves, of the qualities it
has?

I have tried to compare it to the algorithms used by Dylan and CLOS and others,
described in this paper

http://www.webcom.com/haahr/dylan/linearization-oopsla96.html

they are described in terms such as quality criteria and constraints they
preserve.

Let's consider this example:

>>> cpl.ex9(globals())
class a(object): pass
class b(object): pass
class c(object): pass
class d(object): pass
class e(object): pass
class k1(a,b,c): pass
class k2(d,b,e): pass
class k3(d,a): pass
class z(k1,k2,k3): pass
>>> cpl.names(z.__mro__)
['z', 'k1', 'k3', 'a', 'k2', 'd', 'b', 'c', 'e', 'object']

the thing that leaves me most perplexed is that k3 appears before k2. All other
algortihms I know would put k2 before k3.
E.g. C3 (see paper) ...

>>> cpl.names(cpl.c3_cpl(z))
['z', 'k1', 'k2', 'k3', 'd', 'a', 'b', 'c', 'e', 'object']

or the descrintro simple algo:

>>> cpl.names(cpl.keeplast(cpl.lrdfs(z)))
['z', 'k1', 'c', 'k2', 'b', 'e', 'k3', 'd', 'a', 'object']

This means that it does not preserve the local precedence lists (i.e. the
inheritance list), it is not monotonic* because otherwise it would put d before
a, it seems to preserve "some" aspects of the dfs traversal a,d vs d,a but
still it puts k3 before k2.

The other hypothesis, is that in this case, the algorithm should fail, because
it cannot merge the result of mro(k1) merged with mro(k2) with mro(k3), because
the latter is inconsistent with what obtained so far?

>>> l=cpl.names(k1.__mro__)
>>> l
['k1', 'a', 'b', 'c', 'object']
>>> r=cpl.names(k2.__mro__)
>>> cpl.conservative_merge(l,r)
 ...
>>> l
['k1', 'a', 'k2', 'd', 'b', 'c', 'e', 'object']

vs.

>>> cpl.names(k3.__mro__)
['k3', 'd', 'a', 'object']

a<d is incosistent with d<a

But then this inconsistency rule is different/subtler than the one sketched in
descrintro.

So can someone (Guido?) summarize the qualities of this rule? is there a paper
that describes it? or should I buy the book?

regards.

* the mro of a subclass is an extension without re-ordering of the mros of the
superclasses.




From pedronis@bluewin.ch  Tue Oct  1 23:45:39 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 2 Oct 2002 00:45:39 +0200
Subject: [Python-Dev] perplexed by mro
References: <01e101c26984$fc43da60$6d94fea9@newmexico>
Message-ID: <02a501c2699c$44718460$6d94fea9@newmexico>

[me]
> >>> cpl.ex5(globals())
> class a(object): pass
> class b(object): pass
> class c(object): pass
> class x(a): pass
> class y(a): pass
> class z(x,b,y,c): pass
> >>> cpl.names(z.__mro__) # just the class names, real 2.2 mro algo
> ['z', 'x', 'y', 'a', 'b', 'c', 'object']
> >>> cpl.names(cpl.keeplast(cpl.lrdfs(z))) # descrintro simple algo
> ['z', 'x', 'b', 'y', 'a', 'c', 'object']
>
> I was not able to find a new rule. My intuition is that when the result of
the
> real mro has something to do with the dfs, it is sure from the beginning of
it
> not the end, so the relation of the two algos is unclear.

to clarify this point: even in the absence of any kind of inconsistency, if a
class INB (e.g. b above) get ordered as INB > R wrt to a class R recurring
multiple times in the different superclasses' mros (e.g. a above) in between
the first and last recurrence of that class R, the class INB will appear after
R in the final order, while the simple algo (see also example) would put INB
before R.
In fact in the example:  b follows a using the 2.2 mro; and b precedes a wrt
the simple algo. Although in the example there's no kind of inconsistency. The
mros of the superclasses of z being:

x a object [mro x]
b object [mro b]
y a object [mro y]
c object [mro c]

The mro 2.2 algo merges these mros successively, also mro x is merged with mro
b and the result is then merged with mro y and finally what has been obtained
so far is merged with mro c.

The merge step is as follows [python version of the c code in typeobject.c]

def conservative_merge(left,right):
    while 1:
        left_size = len(left)
        right_size = len(right)
        eq = 0
        for i in xrange(left_size):
            for j in xrange(right_size):
                if left[i] == right[j]:
                    print i,j
                    eq = 1
                    break
            if eq: break
        if eq:
            temp = []
            for r in range(j):
                rr = right[r]
                if rr not in left:
                    temp.append(rr)
            left[i:i] = temp
            right[0:j+1] = []
            continue
        break
    left.extend(right)

The algorithm move portions of right to left before merge points [shared
elements], the rest of right is attached at the end of left. That's way the
behavior explained above.

The algorithm merges left and right as sorted sequences, favoring left in the
case of a tie (assuming no incosistencies).


Btw: the algorithms in the paper I have cited all use some kind of parallel
merge of the superclasses' mros or some other constraints, not a sequential one
as the 2.2 mro algo.

regards.



From pedronis@bluewin.ch  Wed Oct  2 00:01:12 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Wed, 2 Oct 2002 01:01:12 +0200
Subject: [Python-Dev] perplexed by mro
References: <01e101c26984$fc43da60$6d94fea9@newmexico> <02a501c2699c$44718460$6d94fea9@newmexico>
Message-ID: <02d101c2699e$7067df40$6d94fea9@newmexico>

> The algorithm move portions of right to left before merge points [shared
> elements], the rest of right is attached at the end of left. That's way the
> behavior explained above.
s/way/why/

sorry.


From arigo@ulb.ac.be  Wed Oct  2 00:33:47 2002
From: arigo@ulb.ac.be (Armin Rigo)
Date: Wed, 02 Oct 2002 01:33:47 +0200
Subject: [Python-Dev] Psyco requests and patches
Message-ID: <3D9A30DB.FE2E042E@ulb.ac.be>

Hello everybody,

As I have been putting more work into Psyco to make sure it will work
well with Python 2.3, I came up with a few support requests.

Non-technical stuff first.  People tend to think about Psyco as an
obscure and complex piece of software for gurus -- should they know
about it at all.  Of course I would argue instead that it is exactly the
piece of software that Python users should not have to worry about,
because it should be perfectly transparent.  As you can guess I would
even say that the first widely-used dynamic programming language to
natively include this kind of techniques would see a huge widening of
its application domain.  For Psyco to be a first step in that direction,
here are a few goals that I would like you to comment.

Publicity: well, I could do with a little more advertising, e.g. on
www.python.org or better yet in the docs and tutorials, at places where
performance is discussed.  Right now I am not sure that you can learn
about Psyco's existence unless you read python-dev or comp.lang.python.

Ease of use: currently the psyco.bind() interface is easy to use, but it
could be trivial instead of just easy.  It should actually be fully
transparent.  (That's what psyco.jit() promised to do, but right now it
badly sucks; keeping it around may have been an error, as people is
actually trying it and getting bad results.)

Portability: the easy part! Indeed, an empty module named 'psyco'
portably provides exactly the same functionality as Psyco everywhere... 
On non-supported platforms you just get nothing better than the current
interpreter performance, but nothing worse either.  (Besides, emitting
portable pseudo-code could still be a future option.  Of course,
emitting a platform's machine code is not that easy at all, but this
fact doesn't compromise portability.)

Ease of maintenance: the hard part.  Psyco is not currently ready for
that.  The implementation is way too obscure.  You have no guarantee if
I tell -- and I do -- that I plan to upgrade Psyco to all newer Python
versions for at least as long as the 2.x series is alive.  We need more
than one person involved.  (Note that upgrades are not as time-consuming
as, say, in the case of Christian Tismer's first Stackless
implementation that required a complete rewrite of the interpreter.)
Let's defer this point to the bottom of this e-mail.

Inclusion in the Standard Library or the interpreter core itself: not
right now! This is a guideline for the goals discussed above.  I might
be inclined to think that completion of all the previous goals would be
enough for consideration...?


-=- Python patches -=-

Psyco requires no change in the interpreter core -- however, there are a
couple of things that needed obfuscated workarounds or just don't
completely work at all.  A few minor changes would make it all much
cleaner.  I submitted three patches to SF
(http://www.python.org/sf/617309, http://www.python.org/sf/617311,
http://www.python.org/sf/617312).  The patches should be clear enough,
not introducing obscure "features" or measurable performance loss.  The
diffs are currently against the 2.2 branch -- it would greatly help
Psyco if the patches made their way into Python 2.2.2, which I
understood to be the candidate Python-in-a-tie.  I will submit 2.3
versions of the patches as well, of course.


-=- Maintenance of Psyco -=-

Psyco is currently highly obscure.  I want to rewrite it in Python. 
Still, however, there will always be the problem that for any feature
supported by Psyco someone must track the changes in the corresponding
feature in the interpreter, and port them.  (This doesn't concern the
features not supported by Psyco: for these it always calls their
up-to-date C implementation.) This process might be semi-automated with
a tool that would be able to dig into C code and analyse it, but this
would be sort of a hack.

We might do some thinking about this synchronization problem when the
discussion about Python 3 is raised again: how hard would it be e.g. to
write some parts of the interpreter directly in Python -- or at least in
some specific dialect that can be turned into C (a la Pyrex) for ease of
compilation? It may actually help make the interpreter more
understandable.  This looks a bit far-fetched, but I will never give up
my claims that high-level languages are just better than low-level ones
for most purposes, and writing an interpreter is no exception!

For example, there was a December 2000 proposal to rewrite the "cmath"
module in Python (http://www.python.org/sf/210838).  It never actually
made its way into the Standard library, which is too bad for Psyco,
because Psyco-with-cmath-in-Python would be faster than
Psyco-with-cmath-in-C.

Practically, the whole point of having a single high-level source code
is that you can do more than just one thing with it.  With C code, it is
a horrible hack to do anything else than compile and execute it.  With
Python code it's a different matter.  You can use this single source to
produce both the equivalent C code and to produce Psyco itself.  In each
case there is still an important part written directly in C, of course;
I am not (yet;-) suggesting that Python should be written *only* in
Python! Very roughly, the separation would be between the built-in
objects + abstract protocols in 'Objects/' (which could be written in
Python) and the core in 'Python/' (which could not).

The framework required to do such a thing looks complicated.  I would
argue that it probably isn't (and in any case it might be worth being
carefully considered); this extra complexity might be favorably balanced
if only just because writing Python in Python is easier than in C  :-)


A bientot,

Armin

PS: I just obtained my math Ph.D. -- it's now Psyco time again!


From tismer@tismer.com  Wed Oct  2 00:42:14 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 01 Oct 2002 16:42:14 -0700
Subject: [Python-Dev] Psyco requests and patches
References: <3D9A30DB.FE2E042E@ulb.ac.be>
Message-ID: <3D9A32D6.8070405@tismer.com>

Armin Rigo wrote:
...

> PS: I just obtained my math Ph.D. -- it's now Psyco time again!

Congratulations!

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Wed Oct  2 00:59:56 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 01 Oct 2002 16:59:56 -0700
Subject: [Python-Dev] regex Perl vs Python
Message-ID: <3D9A36FC.304@tismer.com>

Hi Friends,

It is said that with the new re module, Python "has the regular
expressions like Perl".
Can somebody tell me to what extent this is true?
I need to convince s.o. not to use Perl for a larger project,
but I cannot answer the question up to which version the
Python re's are covering all of Perl's?

thanks a lot - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From martin@v.loewis.de  Wed Oct  2 07:41:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 02 Oct 2002 08:41:59 +0200
Subject: [Python-Dev] Psyco requests and patches
In-Reply-To: <3D9A30DB.FE2E042E@ulb.ac.be>
References: <3D9A30DB.FE2E042E@ulb.ac.be>
Message-ID: <m3smzppcjs.fsf@mira.informatik.hu-berlin.de>

Armin Rigo <arigo@ulb.ac.be> writes:

> The diffs are currently against the 2.2 branch -- it would greatly
> help Psyco if the patches made their way into Python 2.2.2

That can't be done: they break binary compatibility (atleast 617309
does, haven't checked the others), which is unacceptable for a micro
release.

Regards,
Martin


From jrw@pobox.com  Wed Oct  2 08:10:22 2002
From: jrw@pobox.com (John Williams)
Date: Wed, 02 Oct 2002 02:10:22 -0500
Subject: [Python-Dev] *very* revised proposal for interfaces
Message-ID: <3D9A9BDE.7040304@pobox.com>

After re-reading PEPs 245 (interface syntax) and 246 (object 
adaptation), I hit on a new idea that seems to do everything these two 
PEPs and my original proposal (plus revisions) do, all with much less 
fuss.  I also realized I was adding a lot of excess complexity in order 
to implement all the rigorous error checking that doesn't really make 
sense in the Python world anyway (thanks for nudging me in that 
direction!)  As a side effect it's also a lot easier to explain.

Here's the new version, in a no-frills stripped-down form, explained by 
example.


# Create an interface--not much new here, just simpler syntax and fewer 
frills.
class I(interface):
  prop1 = "Docstring for property prop1."
  # It would be good form to put an argument list here,
  # but this method will never be called so it's not necessary.
  def meth1(): "Docstring for method meth1."

# Create a class implementing the interfac.
# It inherits from "object" explicitly, since a class whose bases
# are all interfaces is itself an interface, not an implementation.
class C(object, I):
  def meth(self,a,b): print "This is I.meth1 implemented by C."
  # (I'm not quite sure how to handle properies yet.)

assert I not in C.__bases__ # Interfaces are not really base classes!

# Example method requiring an object supporting interface I.
def usingI(x,y):
  # Check that x supports interface I.  Return either x or a suitable
  # wrapper around x that supports I's methods.  If x doesn't support I,
  # raise an exception.
  x = I(x)
  # Check that y also supports I.  If it doesn't, return None instead of
  # raising an exception.
  y = I(y, None)
  ...

# Example of binding an existing class to an interface.  Like creating
# an ordinary implementation, the bases are an interface and a regular
# class, but instead of creating a usable new class, it just associates
# an interface with a class and defines how the interface's methods are
# implmented.  The magic class name __binding__ triggers this behavior.
# Instances of this wrapper class are created by calling interface I.
class __binding__(str, I):
  def meth1(self,a,b): # Say what meth1 does on strings.
     assert isinstance(self, str)
     print "I.meth1 called on", self

# Example of using __binding__ to resolve a conflict between two interfaces.
class J(interface):
  def meth1(): "No relation to I.meth1!"
class D(object, I, J): # Implement two interfaces.
  # Define *both* I.meth1 and J.meth1 for this class.
  def meth1(self,a,b): print "This is I.meth1 implemented by D."
class __binding__(D,J):
  def meth1(self): print "This is J.meth1 implemented by D."

# Now to call these two methods, we can do this:
d = D()
i = I(d)     # Note i could really just be d here, not that it matters.
i.meth1(1,2) # Prints "This is I.meth1 implemented by D."
j = J(d)     # Here j must be a wrapper to make things work right.
j.meth1()    # Prints "This is J.meth1 implemented by D."
# Or use different syntax to be more explicit.
I.meth1(d,1,2) # Same as I(d).meth1(1,2)
J.meth1(d)     # Same as J(d).meth1()


Please let me know if I'm just insane for sleep deprivation!

jw



From arigo@tunes.org  Wed Oct  2 11:54:50 2002
From: arigo@tunes.org (Armin Rigo)
Date: Wed, 2 Oct 2002 12:54:50 +0200 (CEST)
Subject: [Python-Dev] Psyco requests and patches
In-Reply-To: <m3smzppcjs.fsf@mira.informatik.hu-berlin.de>
Message-ID: <Pine.OSF.3.96.1021002124504.1830063A-100000@aster3.ulb.ac.be>

Hello Martin,

On 2 Oct 2002, Martin v. Loewis wrote:
> (about Psyco-friendly patches for 2.2.2)
> That can't be done: they break binary compatibility (atleast 617309
> does, haven't checked the others), which is unacceptable for a micro
> release.

No, I don't think any of the patches breaks binary compatibility.  I have
kept this in mind while making them.  Two patches add an extra field each
to the PyThreadState structure, but at the end.

617309: the new hook is of no concern to extension modules.  Reading the
'frame' field is still allowed.  The only real change is into the
implementation of PyEval_GetFrame().


A bientot,

Armin.



From gsw@agere.com  Wed Oct  2 13:59:03 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 2 Oct 2002 08:59:03 -0400
Subject: [Python-Dev] Re: *very* revised proposal for interfaces
In-Reply-To: <20021002105618.14812.46853.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEOGCLAA.gsw@agere.com>

John Williams wrote:
> Here's the new version, in a no-frills stripped-down form, explained by 
> example.
...
> class I(interface):
>   prop1 = "Docstring for property prop1."
>   # It would be good form to put an argument list here,
>   # but this method will never be called so it's not necessary.
>   def meth1(): "Docstring for method meth1."

This is along the lines of what I was suggesting. Note that
you can optionally do some argument list checking. You can
also check other properties but that requires you to check
against an instantiated object and indicate the presence of
these properties when instantiating an interface object.

> class C(object, I):
...
> assert I not in C.__bases__ # Interfaces are not really base classes!

Isn't this assertion going to fail the way you showed it?
Inheritance needn't be used at all for interfaces.

> # Example of binding an existing class to an interface.  Like creating
> # an ordinary implementation, the bases are an interface and a regular
> # class, but instead of creating a usable new class, it just associates
> # an interface with a class and defines how the interface's methods are
> # implmented.  The magic class name __binding__ triggers this behavior.
> # Instances of this wrapper class are created by calling interface I.

I don't think you need any magic class names. If a class
wants to present an interface, it can provide a derived
class and/or surrogate if needed. For example:

class MyDerivedInterface(MyClass):
    def interface_name(self,x): return MyClass.my_name(self,x,1)

class MySurrogateInterface(object):
    def __init__(self,me):      self.me = me
    def __getattr__(self,attr): return self.me.__getattribute__(attr)
    def interface_name(self,x): return self.me.my_name(x,1)

Either one implements the interface.

> d = D()
> i = I(d)     # Note i could really just be d here, not that it matters.

But do interfaces really need to have this capability
built-in? If you want to present an interface that's
constrained, you can do it using a derived class or
surrogate. I would view "d" as the correct thing to
use in this example.

-----

One thing my simplified example did not show (though
it was in my local versions) is the ability to have
implemented_by() register the interface in addition
to testing compliance. This requires something like
"implementerClass.__implements__ += interfaceClass"
(with the appropriate safety checks). This would
allow a more "contractual" approach to interfaces,
since you could check to see if a class has been
registered as supporting an interface before trying
to use that interface.

-----

I think we're both saying the same thing--there is
a core issue that can be addressed without language
extensions or similar things that complicate the
implementation. But I have a more minimalist view:
don't impact anything if your class presents the
interface already. All you need is a simple means
to declare (and verify) that a class implements an
interface.

-Jerry



From jrw@pobox.com  Wed Oct  2 20:55:58 2002
From: jrw@pobox.com (John Williams)
Date: Wed, 02 Oct 2002 14:55:58 -0500
Subject: [Python-Dev] re: *very* revised proposal for interfaces
Message-ID: <3D9B4F4E.8080208@pobox.com>

Gerald S. Williams wrote:
 > This is along the lines of what I was suggesting. Note that
 > you can optionally do some argument list checking. You can
 > also check other properties but that requires you to check
 > against an instantiated object and indicate the presence of
 > these properties when instantiating an interface object.

I think we're trying to solve different problems.  You seem to want to
very that a class supports a certain set of methods, but I just want to
verify that the class *claims* to implement the interface and provide a
framework to make that happen, like a more automated version of PEP 245.

For my purposes, I'm not sure the error checking is really a Good Thing
after some of the things Esteban Castro pointed out.  If you fail to
implement the nesessary methods, that's your own problem.  This approach
is a lot less work for me, and handles strange cases of late binding
more gracefully.  Of course a warning would be helpful in the 95% of
cases where the assumptions made for error checking are correct.

 >>class C(object, I):
 >
 > ...
 >
 >>assert I not in C.__bases__ # Interfaces are not really base classes!
 >
 > Isn't this assertion going to fail the way you showed it?
 > Inheritance needn't be used at all for interfaces.

I use inheritence syntax for interfaces, since it's there and
convenient, but I wanted to emphasize that although interfaces can be
subclasses of interfaces, regular classes *cannot* be subclasses of
interfaces, even though the syntax makes it look that way.

It might be worthwhile to add an "implements(class, interface)"
function.  If you want to get fancy, let the first argument be an
instance as well, and let the second argument be a list of interfaces,
all of which must be supported (the opposite of isinstance's disjunctive
behavior!)

 > I don't think you need any magic class names. If a class
 > wants to present an interface, it can provide a derived
 > class and/or surrogate if needed. For example:
 >
 > class MyDerivedInterface(MyClass):
 >     def interface_name(self,x): return MyClass.my_name(self,x,1)
 >
 > class MySurrogateInterface(object):
 >     def __init__(self,me):      self.me = me
 >     def __getattr__(self,attr): return self.me.__getattribute__(attr)
 >     def interface_name(self,x): return self.me.my_name(x,1)

This only works when you can add methods to classes that implement the
interface.  It's a bit awkward for retrofitting classes you didn't
write, and just not possible for classes implmented in C, like the
builtin classes.

 >>d = D()
 >>i = I(d)     # Note i could really just be d here, not that it matters.
 >
 > But do interfaces really need to have this capability
 > built-in? If you want to present an interface that's
 > constrained, you can do it using a derived class or
 > surrogate. I would view "d" as the correct thing to
 > use in this example.

The idea of constraining i to only implement methods of I is something
from my original proposal that I've since decided was a bad idea--extra
work for no benefit.  The real issue here is that I(d) implements the
methods of interface I.  If class D implements the methods of I
directly, then I(d) can be d, but in normal use you won't know the class
of d, and a wrapper may be necessary, so you can't assume I(d) is d.

 > One thing my simplified example did not show (though
 > it was in my local versions) is the ability to have
 > implemented_by() register the interface in addition
 > to testing compliance. This requires something like
 > "implementerClass.__implements__ += interfaceClass"
 > (with the appropriate safety checks). This would
 > allow a more "contractual" approach to interfaces,
 > since you could check to see if a class has been
 > registered as supporting an interface before trying
 > to use that interface.

This is more like what I had in mind to do behind the scenes.

 > I think we're both saying the same thing--there is
 > a core issue that can be addressed without language
 > extensions or similar things that complicate the
 > implementation. But I have a more minimalist view:
 > don't impact anything if your class presents the
 > interface already. All you need is a simple means
 > to declare (and verify) that a class implements an
 > interface.

I think we're more in agreement than you think.  My original proposal
doesn't do what you're saying here, but the revised version with
slightly relaxed rules does, by allowing an object to be its own
wrapper/proxy when it implements the interface in the most
straightforward way, so the "wrapping" step (calling the interface
class) just reduces to a type assertion.

jw



From mchermside@ingdirect.com  Wed Oct  2 21:27:31 2002
From: mchermside@ingdirect.com (Chermside, Michael)
Date: Wed, 2 Oct 2002 16:27:31 -0400
Subject: [Python-Dev] *very* revised proposal for interfaces
Message-ID: <902A1E710FEAB740966EC991C3A38A8903C35AE8@INGDEXCHANGEC1.ingdirect.com>

[Please excuse poor email format... I've just been moved=20
to outlook and haven't yet figured out how to make it
behave like a normal email program.]

John, I really like your proposal!!!

The main thing that I like about it is that, as you say:
    "It's also a lot easier to explain"

That's usually a sign of a good design... when it all falls
into place and becomes simple. I will admit, I've been trying
to follow the various interface suggestions and have mostly
been getting confused. This is simple and elegant, even if
a few bits would improve with tweaking.

Here is a running commentary on things I liked and disliked:

1. You create a new built-in type called "interface" with the
   special property that it does NOT appear in __bases__ even
   if declared to do so.

   I like this. Your syntax for the most common case of=20
   supporting an interface looks like this:
       class C(object, I):
          <... code here ...>
   If we DIDN'T create a "special" type (special because Python
   has to omit pure interfaces in generating __bases__), then
   things could still work by some sort of declaration after C
   is created:
       class C(object):
           <... code here ...>
       declareItImplements(C,I)

   But that's putting it in the wrong place. The declaration to
   support an interface should be easy to type and to read (in the=20
   common case), and should come at the TOP of the declaration of
   the class, not the end. So I like this.

2. You have a "magic class" __binding__.

   This sounds odd. Python hasn't ever used a "magic class" before.
   But it certainly makes the syntax simple. After all, you need
   to define a bunch of methods (the methods needed to conform to
   the interface). Declaring a class is the handy way to define a
   bunch of methods. So I think this is a little weird, but I don't
   really object. If there's another solution, though, I'd like to
   hear it.

3. Where does the "binding" live?

   Creating some sort of "global binding registry" is really not
   a good idea. Global is just bad, it becomes hard to find where
   things were set, which causes mysterious code failures, and so
   forth. At first, that's what I thought you were doing, but a
   closer look changed my mind.

   It seems to me that when you create a magic "__binding__" class
   what you are ACTUALLY doing is adding to a list of "classes
   implementing it" which is maintained in and associated with I,=20
   the interface object. That has ABSOLUTELY NO effect on anyone
   who is not using the interface, and it feels like the "right"
   place to store this information. So I if I understand this
   correctly, I like it.

4. You make use of the interface by invoking it on an object:
       i =3D I(d)

   This has been discussed before, and it's a good approach. It
   allows for interfaces to return the object (thus not creating
   extra objects unnecessarily) or a proxy (thus giving lots of
   extra flexibility). Solidly supported.

5. You do NOT provide a way to do type declarations along with the
   interface.

   Python isn't picky about types, it naturally supports a policy=20
   of allowing you to use any object that will do the job. That's
   a good thing for the language (even if my own preferences tend=20
   toward a stricter typing system) and interfaces aren't about
   undermining it... just strengthening it! So +1 here.

6. You do NOT provide a way to declare preconditions, postconditions,
   or anything like this for the methods. In fact, there's no
   specification other than a docstring.

   For this I'm not sure. I like the minimalistic, simple approach
   of just giving a docstring, but I might also like some of the
   power of extra features. If I have to give up extra power to get
   a simple, workable approach then I'll live with it.

7. There's no way to "extend" interfaces, since inheriting from an
   interface means something different.

   That's OK with me -- seems like something one would only do when
   in a severe meta-programming mood.

So all in all, I'm liking what I'm seeing here.

-- Michael Chermside

  =20


From tismer@tismer.com  Wed Oct  2 22:05:49 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 02 Oct 2002 14:05:49 -0700
Subject: [Python-Dev] Urgent: regex Perl vs Python
References: <3D9A36FC.304@tismer.com>
Message-ID: <3D9B5FAD.4020300@tismer.com>

Christian Tismer wrote:
> Hi Friends,
> 
> It is said that with the new re module, Python "has the regular
> expressions like Perl".
> Can somebody tell me to what extent this is true?
> I need to convince s.o. not to use Perl for a larger project,
> but I cannot answer the question up to which version the
> Python re's are covering all of Perl's?

Please, can somebody give me just a version number?
I really need this info, and I'm not intending
to start a thread, I sware.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From pyth@devel.trillke.net  Wed Oct  2 22:14:55 2002
From: pyth@devel.trillke.net (holger krekel)
Date: Wed, 2 Oct 2002 23:14:55 +0200
Subject: [Python-Dev] Urgent: regex Perl vs Python
In-Reply-To: <3D9B5FAD.4020300@tismer.com>; from tismer@tismer.com on Wed, Oct 02, 2002 at 02:05:49PM -0700
References: <3D9A36FC.304@tismer.com> <3D9B5FAD.4020300@tismer.com>
Message-ID: <20021002231455.C30315@prim.han.de>

Christian Tismer wrote:
> Christian Tismer wrote:
> > Hi Friends,
> > 
> > It is said that with the new re module, Python "has the regular
> > expressions like Perl".
> > Can somebody tell me to what extent this is true?
> > I need to convince s.o. not to use Perl for a larger project,
> > but I cannot answer the question up to which version the
> > Python re's are covering all of Perl's?
> 
> Please, can somebody give me just a version number?
> I really need this info, and I'm not intending
> to start a thread, I sware.

hi chris,

Maybe you could mail the current regex maintainer (IIRC Frederik Lundh).

Or work yourself and dig through google :-)

ciao holger


From gsw@agere.com  Wed Oct  2 22:31:08 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 2 Oct 2002 17:31:08 -0400
Subject: [Python-Dev] Re: *very* revised proposal for interfaces
In-Reply-To: <3D9B4DCB.6040206@pobox.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFIEOJCLAA.gsw@agere.com>

John Williams wrote:
> I think we're trying to solve different problems.  You seem to want to 
> very that a class supports a certain set of methods, but I just want to 
> verify that the class *claims* to implement the interface and provide a 
> framework to make that happen, like a more automated version of PEP 245.

They're aspects of the same problem. When the claim is made,
it should be verified to some degree in case the interface
changes out from beneath the implementer (to what degree is
certainly negotiable, though).

Checking for late binding and generally anything other than
the presence of unbound class methods requires an object,
although it can still be done. Or you could have ways to
explicitly mark these as supported (perhaps a parameter to
the implemented_by method in my example).

> This only works when you can add methods to classes that implement the 
> interface.  It's a bit awkward for retrofitting classes you didn't 
> write, and just not possible for classes implmented in C, like the 
> builtin classes.

No matter what you have under the hood, you should at least
be able to wrap it entirely. My point is that if you want to
provide a new interface, just do it with existing mechanisms.
(If there are cases where this isn't possible, adding a way
to define class-type interfaces that cannot be expressed now
seems wrong to me.)

> > I think we're both saying the same thing [ ... ]
> I think we're more in agreement than you think.

Yes, we're in violent agreement. :-) Well, at least we're
converging...

Actually, there is still one sticking point: the presence of
wrappers and proxys. IMO, mechanisms exist to implement these
if needed, so I'd rather leave them out (and consequently not
have to change the Python language at all).

-Jerry


From guido@python.org  Wed Oct  2 22:34:00 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 02 Oct 2002 17:34:00 -0400
Subject: [Python-Dev] Urgent: regex Perl vs Python
In-Reply-To: Your message of "Wed, 02 Oct 2002 14:05:49 PDT."
 <3D9B5FAD.4020300@tismer.com>
References: <3D9A36FC.304@tismer.com>
 <3D9B5FAD.4020300@tismer.com>
Message-ID: <200210022134.g92LY0Q05792@pcp02138704pcs.reston01.va.comcast.net>

> > It is said that with the new re module, Python "has the regular
> > expressions like Perl".
> > Can somebody tell me to what extent this is true?
> > I need to convince s.o. not to use Perl for a larger project,
> > but I cannot answer the question up to which version the
> > Python re's are covering all of Perl's?
> 
> Please, can somebody give me just a version number?
> I really need this info, and I'm not intending
> to start a thread, I sware.

Nobody knows, Chris.  We started with Perl regexes as they were in
Perl version 5.x (for some low value of x that nobody remembers),
threw out everything that Python couldn't implement (like references
to variables with $name), threw away a few things that we felt were
only present for Perl 4 backwards compatibility (like some of the
ambiguity between octal character references and backreferences), and
added some Python specific features like named groups.  Hardly any new
Perl regex features were added, but I can't guarantee that we didn't
add anything.

Your best bet is to compare the reference docs for Python and Perl
regexes.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python-pep@ccraig.org  Wed Oct  2 23:11:41 2002
From: python-pep@ccraig.org (python-pep@ccraig.org)
Date: 02 Oct 2002 18:11:41 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
Message-ID: <t1wn0pwwkwy.fsf@kermit.wreck.org>

I just uploaded a reference implementation of how rationals might look
in Python as patch 617779 [1].  I do have some new issues for
discussion that I'd like to get some comments on before I change the
PEP.

1) Should future division return rationals rather than floats.  I had
sort of assumed this would happen, but I just had a discussion with
Kirby Urner and couldn't convince him it was a good idea, so I guess
it isn't so clear.

   Arguments for:
     - you don't lose precision on divides
     - It provides a really nice way to specify rationals (i.e. 1/3)
     - It allows you to eventually unify int/long/rationals so that
       rationals with a denominator of 1 are automagically upcast.

   Arguments against:
     - people who have already changed their code to expect floats will
       have to change it again 
     - rationals are slow

2) Should floats compare equal with rationals only when they are
equal, or whenever the are the closest float?  (i.e. will .2 
compare equal to rational(1, 5))

3) Should rationals try to hash the same as floats?  My leaning on
this is that it will be decided by (2).  If they compare equal when
'close enough' then they should hash the same, if not then they should
only hash the same when both are integral.  I would rather not see .5
hash with rational(1, 2) but not .2 with rational(1, 5).

[1] http://sourceforge.net/tracker/?func=detail&aid=617779&group_id=5470&atid=305470

-- 
Christopher A. Craig <python-pep@ccraig.org>
"[Windows NT is] not about 'Where do you want to go today?'; it's more like
'Where am I allowed to go today?'" -- Mike Mitchell, NT Systems Administrator


From esr@thyrsus.com  Wed Oct  2 23:14:21 2002
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 2 Oct 2002 18:14:21 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <t1wn0pwwkwy.fsf@kermit.wreck.org>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
Message-ID: <20021002221421.GA29958@thyrsus.com>

python-pep@ccraig.org <python-pep@ccraig.org>:
> I just uploaded a reference implementation of how rationals might look
> in Python as patch 617779 [1].  I do have some new issues for
> discussion that I'd like to get some comments on before I change the
> PEP.
> 
> 1) Should future division return rationals rather than floats.  I had
> sort of assumed this would happen, but I just had a discussion with
> Kirby Urner and couldn't convince him it was a good idea, so I guess
> it isn't so clear.
> 
>    Arguments for:
>      - you don't lose precision on divides
>      - It provides a really nice way to specify rationals (i.e. 1/3)
>      - It allows you to eventually unify int/long/rationals so that
>        rationals with a denominator of 1 are automagically upcast.
> 
>    Arguments against:
>      - people who have already changed their code to expect floats will
>        have to change it again 
>      - rationals are slow

+1 for returning rationals.  It's the right thing -- and if it fails, 
it will fail noisily, right?
 
> 2) Should floats compare equal with rationals only when they are
> equal, or whenever the are the closest float?  (i.e. will .2 
> compare equal to rational(1, 5))
>
> 3) Should rationals try to hash the same as floats?  My leaning on
> this is that it will be decided by (2).  If they compare equal when
> 'close enough' then they should hash the same, if not then they should
> only hash the same when both are integral.  I would rather not see .5
> hash with rational(1, 2) but not .2 with rational(1, 5).

APL faced this problem twenty-five years ago.  I like its solution; 
a `fuzz' variable defining the close-enough-for-equality range.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>


From ark@research.att.com  Wed Oct  2 23:27:22 2002
From: ark@research.att.com (Andrew Koenig)
Date: 02 Oct 2002 18:27:22 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021002221421.GA29958@thyrsus.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
Message-ID: <yu99r8f8lbn9.fsf@europa.research.att.com>

Eric> APL faced this problem twenty-five years ago.  I like its
Eric> solution; a `fuzz' variable defining the
Eric> close-enough-for-equality range.

I used to like APL's approach, but I've changed my mind.

Part of the reason is that there are some places where unfuzzed
comparison is essential, such as sorting.  Another part is that
fuzzy comparison destroys substitutability:  If a==b, it is
not always true that f(a)==f(b).

Much as I like APL, I'd rather use Scheme's numeric model.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From esr@thyrsus.com  Wed Oct  2 23:30:07 2002
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 2 Oct 2002 18:30:07 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <yu99r8f8lbn9.fsf@europa.research.att.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com> <yu99r8f8lbn9.fsf@europa.research.att.com>
Message-ID: <20021002223007.GA30048@thyrsus.com>

Andrew Koenig <ark@research.att.com>:
> Eric> APL faced this problem twenty-five years ago.  I like its
> Eric> solution; a `fuzz' variable defining the
> Eric> close-enough-for-equality range.
> 
> I used to like APL's approach, but I've changed my mind.
> 
> Part of the reason is that there are some places where unfuzzed
> comparison is essential, such as sorting.  Another part is that
> fuzzy comparison destroys substitutability:  If a==b, it is
> not always true that f(a)==f(b).
> 
> Much as I like APL, I'd rather use Scheme's numeric model.

Good points...but the fuzz variable could default to zero.  It didn't in APL,
which I always thought a mistake.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>


From jrw@pobox.com  Wed Oct  2 23:36:25 2002
From: jrw@pobox.com (John Williams)
Date: Wed, 02 Oct 2002 17:36:25 -0500
Subject: [Python-Dev] *very* revised proposal for interfaces
References: <902A1E710FEAB740966EC991C3A38A8903C35AE8@INGDEXCHANGEC1.ingdirect.com>
Message-ID: <3D9B74E9.5010704@pobox.com>

Chermside, Michael wrote:
> John, I really like your proposal!!!

Thanks!

> 2. You have a "magic class" __binding__....So I think this is a
 >    little weird, but I don't really object...If there's another
 >    solution, though, I'd like to hear it.

That's pretty much my feeling, too.  This is an area that could probably 
benefit from adding some language support, since I'm already pushing the 
class syntax pretty far.  The idea for the magic class names came from 
the observation that class/interface bindings really are classes in 
themselves, but there's no need to ever refer to them directly, so they 
don't need real names.

> 3. Where does the "binding" live?
> 
>    Creating some sort of "global binding registry" is really not
>    a good idea. Global is just bad, it becomes hard to find where
>    things were set, which causes mysterious code failures, and so
>    forth. At first, that's what I thought you were doing, but a
>    closer look changed my mind.
> 
>    It seems to me that when you create a magic "__binding__" class
>    what you are ACTUALLY doing is adding to a list of "classes
>    implementing it" which is maintained in and associated with I, 
>    the interface object. That has ABSOLUTELY NO effect on anyone
>    who is not using the interface, and it feels like the "right"
>    place to store this information. So I if I understand this
>    correctly, I like it.

Yes.  It was pointed out to me privately that you can still make a mess 
if you have multiple bindings for the same class/interface pair (perhaps 
in a large project).  My best answer to that concern so far is to say 
that (as a matter of style) bindings should always live with either the 
interface of the class they bind, and to enforce a rule that there may 
be only one explictit __binding__ per class/interface pair.

> 6. You do NOT provide a way to declare preconditions, postconditions,
>    or anything like this for the methods. In fact, there's no
>    specification other than a docstring.
> 
>    For this I'm not sure. I like the minimalistic, simple approach
>    of just giving a docstring, but I might also like some of the
>    power of extra features. If I have to give up extra power to get
>    a simple, workable approach then I'll live with it.

I'd really like to put more extensible error checking in, since it was 
in my original wish list.  I left it out here because the proposal looks 
complete without it, and no solution I've come up with yet seems really 
satisfactory.  I would welcome suggestions on how to do this well.

As for syntax, I'm currently learning towards using magic methods named 
__pre__foo and __post__foo to check a method named foo.  I'm kind of 
uneasy about extending special treatment from specific identifiers to 
*all* identifiers with a given prefix, but I suppose there is a 
precendent in the special treatment of module-level identifiers ("_" == 
no export) and class-level identifiers ("__" == pseudo-private).  It's 
also fairly concise and expressive.

> 7. There's no way to "extend" interfaces, since inheriting from an
>    interface means something different.
 >
 >    That's OK with me -- seems like something one would only do when
 >    in a severe meta-programming mood.

Actually you can create derived interfaces, if that's what you meant. 
If all of a class's bases are interfaces, the new class is an interface 
as well.  I've seen subclassed interfaces put to very good use in the 
C++ STL, although the interfaces are just a documentation convention 
there.  I'd be surprised if the Java standard libraries didn't contain a 
lot of dervied interfaces--I know the language allows it, even if it's 
not often used.

Thanks (to everyone) for all the commentary.  I feel like I'm making a 
lot of progress towards something that's functional and pleasant to use, 
but there are still a lot of open issues.  I'll post a minimal reference 
implementation when I get done with it and hopefully generate some more 
interest in the proposal.  I expect that when I start putting in more 
features there will be more cases where no solution is obviously the 
right one, so I'll need all the input I can get to avoid doing things in 
ways that seem backwards to 90% of my target audience.

jw



From pinard@iro.umontreal.ca  Wed Oct  2 23:56:49 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Wed, 02 Oct 2002 18:56:49 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new
 issues
In-Reply-To: <20021002221421.GA29958@thyrsus.com> ("Eric S. Raymond"'s
 message of "Wed, 2 Oct 2002 18:14:21 -0400")
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
Message-ID: <oqsmzo4fgu.fsf@carouge.sram.qc.ca>

[Eric S. Raymond]

>> 1) Should future division return rationals rather than floats.

> +1 for returning rationals.  It's the right thing -- and if it fails, 
> it will fail noisily, right?

While I agree with the theoretical arguments, I have the practical fear that
rationals could grow very big, rather quickly, in the course of a long
computation involving them in various ways.  By big, I mean the numerator
and denominator of the fraction taken in isolation, not the number itself.
Consider inversions of an integer matrices, approximations with truncated
series, or worse things like, maybe, discrete Fourier transforms.

Bigger rationals are, slower they become, and more memory they take.  The
danger is that programmers may get surprised or hurt by Python performance
degradation, raising frequent and recurrent questions here and elsewhere.

On the other hand, I would love if Python was not loosing precision on
non-truncating integer division, so let me try a bit to destroy my own
fears.  On average, most programs will not use matrices of rational numbers,
nor play with series.  Moreover, most programs do not use so many different
numeric variables anyway, nor perform long computations involving them.
Many programs do not go beyond adding or subtracting one, once in a while!

So, I would guess that _on the average_, using rational numbers might be
acceptable and go almost unnoticed by most people.  So it might be more
worth accepting as a community to warn programmers who are more prone to
numerical algorithms of the intrinsic dangers of integer division in Python.

But those feelings are no proof of anything.  How do we get the confirmation
that using rationals in Python would be easy going and innocuous in
practice, beforehand?  It would surely be nice relying in such a feature!

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard



From tismer@tismer.com  Thu Oct  3 00:02:05 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 02 Oct 2002 16:02:05 -0700
Subject: [Python-Dev] Urgent: regex Perl vs Python
References: <3D9A36FC.304@tismer.com>              <3D9B5FAD.4020300@tismer.com> <200210022134.g92LY0Q05792@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3D9B7AED.1040004@tismer.com>

[me about compatibility to which Perl version]

Holger:
 > Maybe you could mail the current regex maintainer (IIRC Frederik Lundh).
 >
 > Or work yourself and dig through google :-)

I'm at a customer in the US and cannot spend hours on
this (and re stuff is not my virtue), so I tried to
ask for help.

Guido:
> Nobody knows, Chris.  We started with Perl regexes as they were in
> Perl version 5.x (for some low value of x that nobody remembers),
> threw out everything that Python couldn't implement (like references
> to variables with $name), threw away a few things that we felt were
> only present for Perl 4 backwards compatibility (like some of the
> ambiguity between octal character references and backreferences), and
> added some Python specific features like named groups.  Hardly any new
> Perl regex features were added, but I can't guarantee that we didn't
> add anything.

Ok, that's a lot of help to me.

> Your best bet is to compare the reference docs for Python and Perl
> regexes.

Will try to defer this to the client, after giving
him the the above info.

Many thanks - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer@tismer.com  Thu Oct  3 00:13:57 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 02 Oct 2002 16:13:57 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>	<20021002221421.GA29958@thyrsus.com> <oqsmzo4fgu.fsf@carouge.sram.qc.ca>
Message-ID: <3D9B7DB5.4000702@tismer.com>

Fran=E7ois Pinard wrote:
> [Eric S. Raymond]
>=20
>=20
>>>1) Should future division return rationals rather than floats.
>>
>=20
>>+1 for returning rationals.  It's the right thing -- and if it fails,=20
>>it will fail noisily, right?

+1 here, too!

> While I agree with the theoretical arguments, I have the practical fear=
 that
> rationals could grow very big, rather quickly, in the course of a long
> computation involving them in various ways.  By big, I mean the numerat=
or
> and denominator of the fraction taken in isolation, not the number itse=
lf.
> Consider inversions of an integer matrices, approximations with truncat=
ed
> series, or worse things like, maybe, discrete Fourier transforms.

Yes, this fear is right.
I think this is just great. Let it grow! Let the user feel what
precision he's carrying around, and how much they throw away
when they reduce down to float.

No I think this is really of advantage. Exact is better than small,
and it is all in the user's hand. Make a float() and you're done.

> Bigger rationals are, slower they become, and more memory they take.  T=
he
> danger is that programmers may get surprised or hurt by Python performa=
nce
> degradation, raising frequent and recurrent questions here and elsewher=
e.
>=20
> On the other hand, I would love if Python was not loosing precision on
> non-truncating integer division, so let me try a bit to destroy my own
> fears.  On average, most programs will not use matrices of rational num=
bers,
> nor play with series.  Moreover, most programs do not use so many diffe=
rent
> numeric variables anyway, nor perform long computations involving them.
> Many programs do not go beyond adding or subtracting one, once in a whi=
le!

I don't think one was considering to go rationale, all the time,
just as the result of integer divide? To me, rationales would be
the natural superclass of integers.
Floats would stand somewhere else, really different animals.

> So, I would guess that _on the average_, using rational numbers might b=
e
> acceptable and go almost unnoticed by most people.  So it might be more
> worth accepting as a community to warn programmers who are more prone t=
o
> numerical algorithms of the intrinsic dangers of integer division in Py=
thon.
>=20
> But those feelings are no proof of anything.  How do we get the confirm=
ation
> that using rationals in Python would be easy going and innocuous in
> practice, beforehand?  It would surely be nice relying in such a featur=
e!

I'm all for it. Get correct results in the first place.
Cut precision explicitly when needed, or for optimization.

BTW., Mathematica did the same thing. Very handy if you are doing
symbolic operations on polynomials, and you can easily keep
exact coefficients for a long time.
Another point is that a long division would never give an overflow.
Overflow since floats are too limited is an effect I don't find funny.

ciao - chris

--=20
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From guido@python.org  Thu Oct  3 01:18:28 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 02 Oct 2002 20:18:28 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Wed, 02 Oct 2002 18:14:21 EDT."
 <20021002221421.GA29958@thyrsus.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
Message-ID: <200210030018.g930IT006079@pcp02138704pcs.reston01.va.comcast.net>

> python-pep@ccraig.org <python-pep@ccraig.org>:
> > I just uploaded a reference implementation of how rationals might look
> > in Python as patch 617779 [1].  I do have some new issues for
> > discussion that I'd like to get some comments on before I change the
> > PEP.
> > 
> > 1) Should future division return rationals rather than floats.  I had
> > sort of assumed this would happen, but I just had a discussion with
> > Kirby Urner and couldn't convince him it was a good idea, so I guess
> > it isn't so clear.
> > 
> >    Arguments for:
> >      - you don't lose precision on divides
> >      - It provides a really nice way to specify rationals (i.e. 1/3)
> >      - It allows you to eventually unify int/long/rationals so that
> >        rationals with a denominator of 1 are automagically upcast.
> > 
> >    Arguments against:
> >      - people who have already changed their code to expect floats will
> >        have to change it again 
> >      - rationals are slow

[ESR]
> +1 for returning rationals.  It's the right thing -- and if it fails, 
> it will fail noisily, right?

Not clear at all.  ABC did this, and we found that a common problem
was that a program doing numeric stuff would run very slowly (i.e. the
opposite of failing noisily).

How are you going to print rationals?  If str(1/3) (and hence print
1/3) or repr(1/3) will return "1/3", that is surely going to cause a
huge amount of breakage.

> > 2) Should floats compare equal with rationals only when they are
> > equal, or whenever the are the closest float?  (i.e. will .2 
> > compare equal to rational(1, 5))
> >
> > 3) Should rationals try to hash the same as floats?  My leaning on
> > this is that it will be decided by (2).  If they compare equal when
> > 'close enough' then they should hash the same, if not then they should
> > only hash the same when both are integral.  I would rather not see .5
> > hash with rational(1, 2) but not .2 with rational(1, 5).
> 
> APL faced this problem twenty-five years ago.  I like its solution; 
> a `fuzz' variable defining the close-enough-for-equality range.

It's not at all clear that that's the right solution.  The problem
with a fuzz variable is that there's only one, and library modules may
end up fighting over the right value for what they are trying to
accomplish.  Making it a per-module variable causes the opposite set
of problems.

I'm all for adding rationals to the language -- but I'd like them
segregated until we have a lot more experience with how they behave.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct  3 01:20:11 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 02 Oct 2002 20:20:11 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Wed, 02 Oct 2002 18:27:22 EDT."
 <yu99r8f8lbn9.fsf@europa.research.att.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com>
 <yu99r8f8lbn9.fsf@europa.research.att.com>
Message-ID: <200210030020.g930KBT06090@pcp02138704pcs.reston01.va.comcast.net>

[Andrew Koenig]
> Much as I like APL, I'd rather use Scheme's numeric model.

I've heard that before, but I've also heard criticism of Scheme's
numeric model.  "It works in Scheme" doesn't give me the warm fuzzy
feeling that it's been tried in real life.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct  3 01:22:52 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 02 Oct 2002 20:22:52 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Wed, 02 Oct 2002 18:56:49 EDT."
 <oqsmzo4fgu.fsf@carouge.sram.qc.ca>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com>
 <oqsmzo4fgu.fsf@carouge.sram.qc.ca>
Message-ID: <200210030022.g930Mrl06112@pcp02138704pcs.reston01.va.comcast.net>

> But those feelings are no proof of anything.  How do we get the
> confirmation that using rationals in Python would be easy going and
> innocuous in practice, beforehand?

By adding them to the language but as an isolated type.  The right
conversions should happen when you mix rationals with other types of
numbers (int/long -> rational -> float -> complex), but no operations
should return rationals unless a rational goes in.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Thu Oct  3 01:42:56 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 12:42:56 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030022.g930Mrl06112@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>

Guido:

> The right conversions should happen when you mix rationals with other
> types of numbers (int/long -> rational -> float -> complex), but no
> operations should return rationals unless a rational goes in.

Maybe there should be a separate operator for rational
division?

  1/3 --> float
  1//3 --> int
  1///3 --> rational

(Okay, a 3-char operator is a bit verbose, but I can't
think of anything else that looks division-like just at
the moment.)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From ark@research.att.com  Thu Oct  3 02:10:52 2002
From: ark@research.att.com (Andrew Koenig)
Date: Wed, 2 Oct 2002 21:10:52 -0400 (EDT)
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030020.g930KBT06090@pcp02138704pcs.reston01.va.comcast.net>
 (message from Guido van Rossum on Wed, 02 Oct 2002 20:20:11 -0400)
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com>
 <yu99r8f8lbn9.fsf@europa.research.att.com> <200210030020.g930KBT06090@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210030110.g931AqP07405@europa.research.att.com>

Guido> [Andrew Koenig]
>> Much as I like APL, I'd rather use Scheme's numeric model.

Guido> I've heard that before, but I've also heard criticism of
Guido> Scheme's numeric model.  "It works in Scheme" doesn't give me
Guido> the warm fuzzy feeling that it's been tried in real life.

...and "It works in APL" does?

More seriously, there aren't that many languages with
infinite-precision rationals, which means there aren't
all that many precedents.

I find the partial ordering among Python's types interesting.
If we use "<" to mean "is a strict subset of", then

   int < long < rational  (except perhaps on machines with 64-bit int,
			   which opens a different can of worms entirely)

   int < float < rational

   float < complex

Excluding complex, then, adding rational to the numeric types makes
the numeric types a lattice.  We could make all of the numeric types
a lattice by adding a "complex rational" type:


                  complex rational
                        |      \___
                        |          \
                     rational    complex
                      /    \   ____/
                     /      \ /
                  long     float
                    \     __/
                     \   /     
                      \ /
                      int


What's nice about a lattice is that for any two types T1 and T2, there
is a unique minimum type T of which T1 and T2 are both subsets (not
necessarily proper subsets, because T1 could be a subset of T2 or vice
versa).


From ark@research.att.com  Thu Oct  3 02:17:30 2002
From: ark@research.att.com (Andrew Koenig)
Date: 02 Oct 2002 21:17:30 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <mailman.1033600509.17328.python-list@python.org>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
 <oqsmzo4fgu.fsf@carouge.sram.qc.ca>
 <mailman.1033600509.17328.python-list@python.org>
Message-ID: <yu99d6qspbh1.fsf@europa.research.att.com>

By the way, has anyone considered the idea of making operations
on rationals faster by never storing any trailing zeroes in the
numerator or denominator?  Instead, strip them and store the
(signed) difference between the number of zeroes you've stripped
from the numerator and the number you've stripped from the denominator.

In other words, instead of storing (num, denom) and having the number
be the (exact) value of num/demon, store (num, denom, exp) and have the
number be the (exact) value of (2**exp)*num/denom.


-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From tim.one@comcast.net  Thu Oct  3 02:21:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 02 Oct 2002 21:21:54 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation
 and new issues
In-Reply-To: <200210030020.g930KBT06090@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGOBIAB.tim.one@comcast.net>

[Andrew Koenig]
> Much as I like APL, I'd rather use Scheme's numeric model.

[Guido]
> I've heard that before, but I've also heard criticism of Scheme's
> numeric model.  "It works in Scheme" doesn't give me the warm fuzzy
> feeling that it's been tried in real life.

We've been thru this before too <wink>, but it doesn't even work in
Scheme -- the Scheme std is too permissive in what it allows conforming
implementations to get away (rationals aren't required; unbounded ints
aren't required; ints *period* aren't required; while an "exact" flag is
required, it has no portable mandatory semantics outside the (also
undefined) range of numbers needed to index vectors; etc).  Real
number-crunchers have no use for it even in a full implementation, as it
doesn't have a way to force precision-vs-space tradeoffs without extending
the language.  There's a reason the NumPy folks never bug you for Scheme
features <wink>.



From ark@research.att.com  Thu Oct  3 02:22:51 2002
From: ark@research.att.com (Andrew Koenig)
Date: 02 Oct 2002 21:22:51 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
Message-ID: <yu998z1gpb84.fsf@europa.research.att.com>

Greg>   1/3 --> float
Greg>   1//3 --> int
Greg>   1///3 --> rational

Greg> (Okay, a 3-char operator is a bit verbose, but I can't
Greg> think of anything else that looks division-like just at
Greg> the moment.)

I can't resist:

        A one-l lama is a priest,
        A two-l llama is a beast,
        But I'll bet you a silk pyjama
        There isn't any three-l lllama.


                                --Ogden Nash


PS: When I first saw this poem, it was accompanied by a claim (in a
    footnote) that a three-l lllama is a substantial conflagration.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From tim.one@comcast.net  Thu Oct  3 02:31:11 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 02 Oct 2002 21:31:11 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation
 and new issues
In-Reply-To: <200210030018.g930IT006079@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEHABIAB.tim.one@comcast.net>

[Guido]
> ...
> Not clear at all.  ABC did this, and we found that a common problem
> was that a program doing numeric stuff would run very slowly (i.e. the
> opposite of failing noisily).

Time for my yearly repetition of that I believe that, at least for me, many
(perhaps most, but not all) of the surprises in ABC were due to that
"floating-point literals" (like 6.02e23) were also treated as exact
rationals.

> ...
> The problem with a fuzz variable is that there's only one, and
> library modules may end up fighting over the right value for what
> they are trying to accomplish.  Making it a per-module variable
> causes the opposite set of problems.

Knuth defines a much more sophisticated notion of "fuzzy comparison" for
floats (TAoCP Vol 2).  That never caught on either.  I'm never clear on what
people think they're *solving* with suggestions like these -- binary
floating-point is so horrendously at odds with "common sense" regardless
that it solves nothing.  In APL there was a particular reason for it, in
order to create arrays of booleans from whole-array comparison operators
efficiently, but that didn't make it a useful *scalar* gimmick there either,
and in my brief APL days the damn fuzz destroyed more careful numeric
algorithms than it helped for that reason.

> I'm all for adding rationals to the language -- but I'd like them
> segregated until we have a lot more experience with how they behave.

They behave most like precocious children with voracious appetite <wink>.



From tim.one@comcast.net  Thu Oct  3 02:44:27 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 02 Oct 2002 21:44:27 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <yu99d6qspbh1.fsf@europa.research.att.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net>

[Andrew Koenig]
> By the way, has anyone considered the idea of making operations
> on rationals faster by never storing any trailing zeroes in the
> numerator or denominator?  Instead, strip them and store the
> (signed) difference between the number of zeroes you've stripped
> from the numerator and the number you've stripped from the denominator.
>
> In other words, instead of storing (num, denom) and having the number
> be the (exact) value of num/demon, store (num, denom, exp) and have the
> number be the (exact) value of (2**exp)*num/denom.

How many trailing zeroes do you expect to save this way?  "On average", a
random int has exactly n (>= 0) trailing zero bits with probability
1/2**(n+1).  Then how expensive is it for operations to keep track of them
appropriately (esp. + and -, where the split representation seems offhand
(to me) unnatural to work with)?  My FixedPoint.py does a variation of this
for a form of unnormalized rational, except the "trailing zeroes" there are
wrt base 10.  But it wasn't to save time (heh) in that context, it was to
create a faithful illusion of exact decimal arithmetic.



From ark@research.att.com  Thu Oct  3 03:16:07 2002
From: ark@research.att.com (Andrew Koenig)
Date: Wed, 2 Oct 2002 22:16:07 -0400 (EDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> (message from
 Tim Peters on Wed, 02 Oct 2002 21:44:27 -0400)
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net>
Message-ID: <200210030216.g932G7707727@europa.research.att.com>

Tim> How many trailing zeroes do you expect to save this way?

The real point is to have small, efficient rational representations
for floating-point numbers, even those with large exponents.


From greg@cosc.canterbury.ac.nz  Thu Oct  3 03:18:59 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 14:18:59 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <yu998z1gpb84.fsf@europa.research.att.com>
Message-ID: <200210030218.g932IxH20780@oma.cosc.canterbury.ac.nz>

Andrew Koenig <ark@research.att.com>:

>         A one-l lama is a priest,
>         A two-l llama is a beast,
>         But I'll bet you a silk pyjama
>         There isn't any three-l lllama.
> 
>                                 --Ogden Nash
> 
> PS: When I first saw this poem, it was accompanied by a claim (in a
>     footnote) that a three-l lllama is a substantial conflagration.

:-) !!!

I have a vision of an obscure corner of Tibet featuring
an order of special firefighting lamas, ready at a moment's
notice to jump on their llamas and race off to do battle
with a lllama...

And whenever I see "1/3" now I'm going to want to
pronouce it "one lama three". What have you done to
me?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Thu Oct  3 03:22:30 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 14:22:30 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030216.g932G7707727@europa.research.att.com>
Message-ID: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz>

Andrew Koenig <ark@research.att.com>:

> The real point is to have small, efficient rational representations
> for floating-point numbers, even those with large exponents.

But how long are they going to *stay* small and efficient,
once you start doing arithmetic on them?

The problem as I see it is not the magnitude of the number, but that
the number of significant bits grows without bound, if you never throw
any of them away.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From ark@research.att.com  Thu Oct  3 03:25:07 2002
From: ark@research.att.com (Andrew Koenig)
Date: 02 Oct 2002 22:25:07 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz>
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz>
Message-ID: <yu994rc4p8cc.fsf@europa.research.att.com>

Greg> But how long are they going to *stay* small and efficient,
Greg> once you start doing arithmetic on them?

Depends on what kind of arithmetic.

Greg> The problem as I see it is not the magnitude of the number, but
Greg> that the number of significant bits grows without bound, if you
Greg> never throw any of them away.

So I'm suggesting that maybe it won't get bad as quickly if
you keep only the significant bits :-)

Seriously, I don't know whether it would help in practice.
It might be that normalizing rationals from time to time would
be enough.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From esr@thyrsus.com  Thu Oct  3 03:41:21 2002
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 2 Oct 2002 22:41:21 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030218.g932IxH20780@oma.cosc.canterbury.ac.nz>
References: <yu998z1gpb84.fsf@europa.research.att.com> <200210030218.g932IxH20780@oma.cosc.canterbury.ac.nz>
Message-ID: <20021003024121.GA31589@thyrsus.com>

Greg Ewing <greg@cosc.canterbury.ac.nz>:
> Andrew Koenig <ark@research.att.com>:
> 
> >         A one-l lama is a priest,
> >         A two-l llama is a beast,
> >         But I'll bet you a silk pyjama
> >         There isn't any three-l lllama.
> > 
> >                                 --Ogden Nash
> > 
> > PS: When I first saw this poem, it was accompanied by a claim (in a
> >     footnote) that a three-l lllama is a substantial conflagration.
> 
> :-) !!!
> 
> I have a vision of an obscure corner of Tibet featuring
> an order of special firefighting lamas, ready at a moment's
> notice to jump on their llamas and race off to do battle
> with a lllama...

:-)

By the way, the footnote read as follows:

"The author's attention has been called to a type of conflagration known
as a three-alarmer. Pooh."
 
> And whenever I see "1/3" now I'm going to want to
> pronouce it "one lama three". What have you done to
> me?

Oh, ghods.  

You realize this is turning into a classic Jargon File entry, don't you?

@hd{lama} @g{n.} [Python] The division operator /; thus 1/3 is pronounced
  "one lama three".  Coined during and October 2002 on the Python-development
  mailing list.  Someone proposed that /, //, and /// should stand for
  integer-, rational-, and float-valued division, and someone else quoted
  Ogden Nash's poem @uref{http://www.cs.rice.edu/~ssiyer/minstrels/poems/1080.html, 
  The Lama}.  Amidst talk of firefighting llamas in Tibet, the analogy stuck.

Of course, I can't actually add this.  The last three words aren't true.  

Yet.,,
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>


From python@rcn.com  Thu Oct  3 03:50:56 2002
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 2 Oct 2002 22:50:56 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <t1wn0pwwkwy.fsf@kermit.wreck.org><20021002221421.GA29958@thyrsus.com> <oqsmzo4fgu.fsf@carouge.sram.qc.ca>
Message-ID: <009401c26a87$b2e0c780$b2e97ad1@othello>

[François Pinard]
> While I agree with the theoretical arguments, I have the practical fear that
> rationals could grow very big, rather quickly, in the course of a long
> computation involving them in various ways.  By big, I mean the numerator
> and denominator of the fraction taken in isolation, not the number itself.
> Consider inversions of an integer matrices, approximations with truncated
> series, or worse things like, maybe, discrete Fourier transforms.
>
> Bigger rationals are, slower they become, and more memory they take.  The
> danger is that programmers may get surprised or hurt by Python performance
> degradation, raising frequent and recurrent questions here and elsewhere.

There should be a builtin variable (overriddable within some inner scope)
for a maximum denominator magnitude.  It should default to some value
where performance tanks.  If set to None, then no limit would apply.

The HP32SII calculator implements a useful rational model using
flags and a maximum denominator register.  If the first flag is clear,
fractions are have denominators upto the maximum value.  If only
the first flag is set, fractions always use the maximum denominator
as the denominator and are then reduced (i.e. if the max is 8, then
.5 is represented as 1/2 and .1 is represented as 1/8).



[Christopher A. Craig]
> > 3) Should rationals try to hash the same as floats?  My leaning on
> > this is that it will be decided by (2).  If they compare equal when
> > 'close enough' then they should hash the same, if not then they should
> > only hash the same when both are integral.  I would rather not see .5
> > hash with rational(1, 2) but not .2 with rational(1, 5).

[Eric Raymond]
> APL faced this problem twenty-five years ago.  I like its solution;
> a `fuzz' variable defining the close-enough-for-equality range.

Instead of a global fuzz variable, I would prefer a fuzzy compare
function with a specifiable fuzz factor and a reasonable default
setting.  This approach is more explicit in leaving == as an exact
compare, specifying nearlyequal(a,b) when that is what is meant,
and providing a locally specifiable factor on each compare, for
example, nearlyequal(a,b, absolute=1e-7).

Also, a fuzzy compare function could be set to use absolute or
relative differences when needed.  For example:
   if nearlyequal(a,b, relative=.01):  # are a and b within 1%



Raymond Hettinger



From tismer@tismer.com  Thu Oct  3 03:55:25 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 02 Oct 2002 19:55:25 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com>
Message-ID: <3D9BB19D.4000401@tismer.com>

Andrew Koenig wrote:
> Greg> But how long are they going to *stay* small and efficient,
> Greg> once you start doing arithmetic on them?
> 
> Depends on what kind of arithmetic.

I doubt very much that this would help in more than few
simple cases. Floats with high exponents are usually
incorrect, since the number of bits of the exact
representation is already truncated to the float limitation.
With an exact rational representation all the bits are
preserved, and it is unlikely that many of the lower bits
simply become zero.

> Greg> The problem as I see it is not the magnitude of the number, but
> Greg> that the number of significant bits grows without bound, if you
> Greg> never throw any of them away.
> 
> So I'm suggesting that maybe it won't get bad as quickly if
> you keep only the significant bits :-)

Yes. And this is what a rounding to float would do for you.

> Seriously, I don't know whether it would help in practice.
> It might be that normalizing rationals from time to time would
> be enough.

Ok, not just zeros, but normalizing the fraction to no
common denominator. That makes sense if lots of small
numbers/fractions were multiplied and prime factors
pile up.
Not so with addition. This creates completely different
prime factors all the time, and addition becomes much
more expensive than multiplication when the common
denominator must always be reduced to the minimum.

gotta-be-expensive  -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From python@rcn.com  Thu Oct  3 04:02:58 2002
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 2 Oct 2002 23:02:58 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com> <3D9BB19D.4000401@tismer.com>
Message-ID: <00ac01c26a89$6117aac0$b2e97ad1@othello>

From: "Christian Tismer" <tismer@tismer.com>

> > Greg> The problem as I see it is not the magnitude of the number, but
> > Greg> that the number of significant bits grows without bound, if you
> > Greg> never throw any of them away.
> > 
> > So I'm suggesting that maybe it won't get bad as quickly if
> > you keep only the significant bits :-)
> 
> Yes. And this is what a rounding to float would do for you.

That is unless you wanted more precision than comes with a float.
Normalization to some arbitrary maximum denominator magnitude
would be a more general purpose solution.


Raymond Hettinger



From greg@cosc.canterbury.ac.nz  Thu Oct  3 04:07:19 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 15:07:19 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <009401c26a87$b2e0c780$b2e97ad1@othello>
Message-ID: <200210030307.g9337JW21043@oma.cosc.canterbury.ac.nz>

Raymond Hettinger <python@rcn.com>:

> The HP32SII calculator implements a useful rational model using
> flags and a maximum denominator register.  If the first flag is clear,
> fractions are have denominators upto the maximum value.  If only
> the first flag is set, fractions always use the maximum denominator
> as the denominator and are then reduced (i.e. if the max is 8, then
> .5 is represented as 1/2 and .1 is represented as 1/8).

I can see that being useful when you're doing calculations
with, e.g. measurements in 64ths of an inch and you don't
care if anything smaller than that isn't quite exact.

But I get the impression that the folks who want rationals
in Python want them precisely because they're *always*
exact. If you couldn't rely on them to always be exact,
it would defeat the purpose.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fschaef@ces.clemson.edu  Thu Oct  3 04:12:35 2002
From: fschaef@ces.clemson.edu (Frank S.)
Date: Wed, 2 Oct 2002 23:12:35 -0400 (EDT)
Subject: [Python-Dev] RFC: Option Parsing Libraries
Message-ID: <Pine.GSO.4.33.0210022237190.14016-100000@maple.ces.clemson.edu>

After releasing the GetPot 1.0 Version I did some research on other
packages and I found the the Option Parsing RFC. I totally missed the
discussion. How come that nobody contacted me about the GetPot package ?
To get a python doc out should not be a problem at all. Not one single
person, though, complained until now, since the C++ doc is totally
equivalent.

Take a look at http://getpot.sourceforge.net for new features of the 1.0
Version. Except for the automated documenting feature (which may be a
little intrusive in its format) I don't really see what Optik brings
new on the table.

Guys, even though GetPot is designed for many languages, I'd like
to join the command line parsing discussion.

Best Regards,

Frank Schaefer.





From greg@cosc.canterbury.ac.nz  Thu Oct  3 04:14:24 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 15:14:24 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021003024121.GA31589@thyrsus.com>
Message-ID: <200210030314.g933EOb21057@oma.cosc.canterbury.ac.nz>

"Eric S. Raymond" <esr@thyrsus.com>:

> Someone proposed that /, //, and /// should stand for
> integer-, rational-, and float-valued division,
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

actually,

  float-, integer- and rational-

I'm not suggesting the existing ones should be changed!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tismer@tismer.com  Thu Oct  3 04:14:34 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 02 Oct 2002 20:14:34 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <t1wn0pwwkwy.fsf@kermit.wreck.org><20021002221421.GA29958@thyrsus.com> <oqsmzo4fgu.fsf@carouge.sram.qc.ca> <009401c26a87$b2e0c780$b2e97ad1@othello>
Message-ID: <3D9BB61A.5070902@tismer.com>

Raymond Hettinger wrote:
...

> There should be a builtin variable (overriddable within some inner scope)
> for a maximum denominator magnitude.  It should default to some value
> where performance tanks.  If set to None, then no limit would apply.
> 
> The HP32SII calculator implements a useful rational model using
> flags and a maximum denominator register.  If the first flag is clear,
> fractions are have denominators upto the maximum value.  If only
> the first flag is set, fractions always use the maximum denominator
> as the denominator and are then reduced (i.e. if the max is 8, then
> .5 is represented as 1/2 and .1 is represented as 1/8).

Makes very much sense to me.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From ark@research.att.com  Thu Oct  3 04:14:12 2002
From: ark@research.att.com (Andrew Koenig)
Date: Wed, 2 Oct 2002 23:14:12 -0400 (EDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
In-Reply-To: <3D9BB19D.4000401@tismer.com> (message from Christian Tismer on
 Wed, 02 Oct 2002 19:55:25 -0700)
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com> <3D9BB19D.4000401@tismer.com>
Message-ID: <200210030314.g933ECr08046@europa.research.att.com>

>> So I'm suggesting that maybe it won't get bad as quickly if
>> you keep only the significant bits :-)

Christian> Yes. And this is what a rounding to float would do for you.

Except that doing so would throw away some significant bits too.


From python@rcn.com  Thu Oct  3 04:13:16 2002
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 2 Oct 2002 23:13:16 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <200210030307.g9337JW21043@oma.cosc.canterbury.ac.nz>
Message-ID: <00fb01c26a8b$91f5d200$b2e97ad1@othello>

From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
> > The HP32SII calculator implements a useful rational model using
> > flags and a maximum denominator register.  If the first flag is clear,
> > fractions are have denominators upto the maximum value.  If only
> > the first flag is set, fractions always use the maximum denominator
> > as the denominator and are then reduced (i.e. if the max is 8, then
> > .5 is represented as 1/2 and .1 is represented as 1/8).
> 
> I can see that being useful when you're doing calculations
> with, e.g. measurements in 64ths of an inch and you don't
> care if anything smaller than that isn't quite exact.
> 
> But I get the impression that the folks who want rationals
> in Python want them precisely because they're *always*
> exact. If you couldn't rely on them to always be exact,
> it would defeat the purpose.

For that, I propose maxdenom=None to let the rationals
grow without bound.


Raymond Hettinger



From greg@cosc.canterbury.ac.nz  Thu Oct  3 04:22:50 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 15:22:50 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021003024121.GA31589@thyrsus.com>
Message-ID: <200210030322.g933MoR21076@oma.cosc.canterbury.ac.nz>

"Eric S. Raymond" <esr@thyrsus.com>:

> By the way, the footnote read as follows:
> 
> "The author's attention has been called to a type of conflagration known
> as a three-alarmer. Pooh."

This is getting weird. I did a Google search for "lllama"
and found this, apparently related to some sort of roleplaying
game:

  From: Alex Yeager (YeagerAW@maritz.com)
  Date: Mon, 25 May 1998

  Type: Plot
  Name: Three-L Lllama
  Graphic: A building on fire.
  Quote: Ogden Nash was wrong! They DO exist!
  
  Text: Disaster! This is an Instant Attack to Destroy any Place. Its
  Power is 12, increased by four for every undrawn Plot you throw onto
  the blaze. If the Place is Coastal, it may spend its token to defend
  itself. If the attack succeeds, the target is Devastated. If it
  succeeds by more than 6, the target is destroyed!  Quote2: So where do
  I claim those silk pajamas?...  Requirements: none

  Illamanated Conspiracy

  Proposed by "Yeager, Alex" (YeagerAW@Maritz.com) on Mon, 25 May 1998.

So, was this Lllama spell inspired by the "three-alarmer" pun, or
is it just a strange coincidence? Or is it some secret plot by
the PSU to stop us sleeping at nights? Inquiring minds probably
don't really want to know...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Thu Oct  3 04:24:16 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 15:24:16 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <00ac01c26a89$6117aac0$b2e97ad1@othello>
Message-ID: <200210030324.g933OGx21083@oma.cosc.canterbury.ac.nz>

Raymond Hettinger <python@rcn.com>:

> That is unless you wanted more precision than comes with a float.
> Normalization to some arbitrary maximum denominator magnitude
> would be a more general purpose solution.

Wouldn't a float type with specifiable precision do
just as well for that, and be more efficient?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Thu Oct  3 04:27:51 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Oct 2002 15:27:51 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <00fb01c26a8b$91f5d200$b2e97ad1@othello>
Message-ID: <200210030327.g933Rpw21328@oma.cosc.canterbury.ac.nz>

Raymond Hettinger <python@rcn.com>:

> For that, I propose maxdenom=None to let the rationals
> grow without bound.

A global setting for this would have all the same problems
we've just gone over in relation to the "fuzz" setting.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Thu Oct  3 05:00:51 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 00:00:51 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <yu994rc4p8cc.fsf@europa.research.att.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEHLBIAB.tim.one@comcast.net>

[Andrew Koenig]
> ...
> Seriously, I don't know whether it would help in practice.
> It might be that normalizing rationals from time to time would
> be enough.

Do check out Val=E9rie M=E9nissier-Morain's work on exact rational ar=
ithmetic in
Caml; I've referenced it several times before, and this discussion is=
 always
the same <wink>.  In real apps, the time it takes to normalize ration=
als can
be a disaster (the gcd of two random ints is most likely to be 1, and
longint gcds are very expensive).  In other real apps, not normalizin=
g
rationals can be a disaster.  There isn't a one-size-fits-all policy =
for
this, and a serious implementation has to take that seriously (sorry,=
 Scheme
<wink>).




From Anthony Baxter <anthony@interlink.com.au>  Thu Oct  3 05:01:28 2002
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Thu, 03 Oct 2002 14:01:28 +1000
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030322.g933MoR21076@oma.cosc.canterbury.ac.nz>
Message-ID: <200210030401.g9341Sb16764@localhost.localdomain>

>>> Greg Ewing wrote
> This is getting weird. I did a Google search for "lllama"
> and found this, apparently related to some sort of roleplaying
> game:
> 
>   From: Alex Yeager (YeagerAW@maritz.com)
>   Date: Mon, 25 May 1998
> 
>   Type: Plot
>   Name: Three-L Lllama
>   Graphic: A building on fire.
>   Quote: Ogden Nash was wrong! They DO exist!

This is a card for Steve Jackson's Illuminati: New World Order collectible
card game. A lot of fun, but don't play it with people who take badly to 
being brutally betrayed and backstabbed (that's kinda the point of the 
game).

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From oren-py-d@hishome.net  Thu Oct  3 06:28:29 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 01:28:29 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030216.g932G7707727@europa.research.att.com>
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com>
Message-ID: <20021003052829.GA59708@hishome.net>

On Wed, Oct 02, 2002 at 10:16:07PM -0400, Andrew Koenig wrote:
> Tim> How many trailing zeroes do you expect to save this way?
> 
> The real point is to have small, efficient rational representations
> for floating-point numbers, even those with large exponents.

But why would you want to convert floating point numbers to rationals in
the first place?  Floating point numbers are generally inexact. Converting 
them to exact rationals can give a false sense of exactitude. 

Any calculation involving a floating point number should taint the result
with inexactitude and yield a floating point result. 

	Oren


From aleax@aleax.it  Thu Oct  3 06:56:50 2002
From: aleax@aleax.it (Alex Martelli)
Date: Thu, 3 Oct 2002 07:56:50 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
Message-ID: <E17wyys-0007ei-00@mail.python.org>

On Thursday 03 October 2002 02:42 am, Greg Ewing wrote:
	...
>   1///3 --> rational
>
> (Okay, a 3-char operator is a bit verbose, but I can't
> think of anything else that looks division-like just at
> the moment.)

1\3 ... ?


Alex


From tim.one@comcast.net  Thu Oct  3 07:01:49 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 02:01:49 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <20021003052829.GA59708@hishome.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEIDBIAB.tim.one@comcast.net>

[Oren Tirosh]
> But why would you want to convert floating point numbers to rationals
> in the first place?

For one thing, to continue an exact calculation that has grown too large to
continue in floating-point without losing information, but where the extreme
expense of using rationals instead of floating point pushes you toward using
fp just as long as you can.

> Floating point numbers are generally inexact.

The IEEE-754 standard mandates an "inexact flag" (and your HW has one) for
this very kind of purpose.  Your FPU knows for certain whether you've ever
lost a bit of information when doing basic fp operations; unfortunately,
it's still extremely clumsy to get at this info.

> Converting them to exact rationals can give a false sense of
> exactitude.

So can printing more digits than warranted <wink>.

> Any calculation involving a floating point number should taint
> the result with inexactitude and yield a floating point result.

How old are you?  See -- integers can be inexact too.  Representation has
nothing to do with exactness, which is one part of this story Scheme got
straight.



From Anthony Baxter <anthony@interlink.com.au>  Thu Oct  3 07:03:20 2002
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Thu, 03 Oct 2002 16:03:20 +1000
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <E17wyys-0007ei-00@mail.python.org>
Message-ID: <200210030603.g9363Kd17878@localhost.localdomain>

>>> Alex Martelli wrote
> On Thursday 03 October 2002 02:42 am, Greg Ewing wrote:
> 	...
> >   1///3 --> rational
> >
> > (Okay, a 3-char operator is a bit verbose, but I can't
> > think of anything else that looks division-like just at
> > the moment.)
> 
> 1\3 ... ?

a = """ 1
        -
        3"""




From oren-py-d@hishome.net  Thu Oct  3 07:10:45 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 02:10:45 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEHLBIAB.tim.one@comcast.net>
References: <yu994rc4p8cc.fsf@europa.research.att.com> <LNBBLJKPBEHFEDALKOLCIEHLBIAB.tim.one@comcast.net>
Message-ID: <20021003061045.GB59708@hishome.net>

On Thu, Oct 03, 2002 at 12:00:51AM -0400, Tim Peters wrote:
> [Andrew Koenig]
> > ...
> > Seriously, I don't know whether it would help in practice.
> > It might be that normalizing rationals from time to time would
> > be enough.
> 
> Do check out Valérie Ménissier-Morain's work on exact rational arithmetic in
> Caml; I've referenced it several times before, and this discussion is always
> the same <wink>.  In real apps, the time it takes to normalize rationals can
> be a disaster (the gcd of two random ints is most likely to be 1, and
> longint gcds are very expensive).  In other real apps, not normalizing
> rationals can be a disaster.  There isn't a one-size-fits-all policy for
> this, and a serious implementation has to take that seriously (sorry, Scheme
> <wink>).

Here is a quick attempt at a one-size-fits-all policy:

Rationals have a denormalized internal representation but always appear to 
be normalized. Rationals may be normalized in-place and still be considered 
immutable because this is invisible to the user.

Rationals are normalized in-place in the following cases:

1. When the string representation of the rational number is required
2. When accessing the numerator or denominator explicitly
3. Occasionally, according to some magic heuristics to prevent growth

The price for trying to please everybody is complexity but performance 
does not have to suffer too much. 

	Oren


From oren-py-d@hishome.net  Thu Oct  3 08:21:47 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 03:21:47 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEIDBIAB.tim.one@comcast.net>
References: <20021003052829.GA59708@hishome.net> <LNBBLJKPBEHFEDALKOLCIEIDBIAB.tim.one@comcast.net>
Message-ID: <20021003072147.GC59708@hishome.net>

On Thu, Oct 03, 2002 at 02:01:49AM -0400, Tim Peters wrote:
> [Oren Tirosh]
> > But why would you want to convert floating point numbers to rationals
> > in the first place?
> 
> For one thing, to continue an exact calculation that has grown too large to
> continue in floating-point without losing information, but where the extreme
> expense of using rationals instead of floating point pushes you toward using
> fp just as long as you can.
> 
> > Floating point numbers are generally inexact.
> 
> The IEEE-754 standard mandates an "inexact flag" (and your HW has one) for
> this very kind of purpose.  Your FPU knows for certain whether you've ever
> lost a bit of information when doing basic fp operations; unfortunately,
> it's still extremely clumsy to get at this info.

The IEEE-754 standard also mandates NaNs, Infs, rounding modes and several
other features. These are not supported by Python in a consistent and
portable manner nor it is likely that they can be supported on all Python
platforms in the foreseeable future because of the limitations of the
underlying C libraries.

> > Converting them to exact rationals can give a false sense of
> > exactitude.
> 
> So can printing more digits than warranted <wink>.

Yes, my material strength teacher drove this point home by taking off 
points for results with either too many or too few digits.

> > Any calculation involving a floating point number should taint
> > the result with inexactitude and yield a floating point result.
> 
> How old are you?  See -- integers can be inexact too.  Representation has
> nothing to do with exactness, which is one part of this story Scheme got
> straight.

I am approaximate 61131/76 fortnights old :-)

A integer is always exact and integer arithmetic is always exact with the 
exception of division. The mapping of this mathematical integer to some 
real-world meaning may be inexact, though. I won't sue anyone because the 
pack of rubber bands doesn't contain exactly 1000 pieces. 

A double precision floating point number may represent some number 
exactly and produce exact results for some arithmetic operations but in
practice if I type 0.3 and get 0.29999999999999999 the numbers aren't 
exact in the first place and if I have no access to the flags that tell me 
if the result is exact or not I must treat all results as inexact.

As usual, your arguments are flawless - in an academic sort of way. In 
practice, though, I think my original point about inexactness of of floats 
and tainting of results still stands. 

	Oren



From mwh@python.net  Thu Oct  3 10:14:09 2002
From: mwh@python.net (Michael Hudson)
Date: 03 Oct 2002 10:14:09 +0100
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Guido van Rossum's message of "Wed, 02 Oct 2002 20:20:11 -0400"
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com> <yu99r8f8lbn9.fsf@europa.research.att.com> <200210030020.g930KBT06090@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <2mlm5fuboe.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> [Andrew Koenig]
> > Much as I like APL, I'd rather use Scheme's numeric model.
> 
> I've heard that before, but I've also heard criticism of Scheme's
> numeric model.  "It works in Scheme" doesn't give me the warm fuzzy
> feeling that it's been tried in real life.

Could try aping Common Lisp.  It's much more tightly specified, and
actually has had some exposure to the real world.

Here would seem to be a start:

    http://www.lispworks.com/reference/HyperSpec/Body/12_ac.htm

Cheers,
M.

-- 
  how am I expected to quit smoking if I have to deal with NT
  every day                                                -- Ben Raia


From esteban@ccpgames.com  Thu Oct  3 11:09:04 2002
From: esteban@ccpgames.com (Esteban U.C.. Castro)
Date: Thu, 3 Oct 2002 10:09:04 -0000
Subject: [Python-Dev] re: *very* revised proposal for interfaces
Message-ID: <25A39AFEB31B06408C675CEF28199E5B073CAC@postur.ccp.cc>

> For my purposes, I'm not sure the error checking is really a Good
Thing
> after some of the things Esteban Castro pointed out.  If you fail to
> implement the nesessary methods, that's your own problem.  This
approach
> is a lot less work for me, and handles strange cases of late binding
> more gracefully.  Of course a warning would be helpful in the 95% of
> cases where the assumptions made for error checking are correct.

I think you misinterpret me here. I am not against checking an
interface.=20
Indeed, checking would be the only way to detect implicit interface=20
compliance (one of my main points).

>>> from interfaces import ListInterface
>>> a =3D [1,2,3]
>>> implements(a, ListInterface)
1
>>> implements(a[0], ListInterface)
0
>>> import my
>>> b =3D my.SuperDuperFakeList()
>>> implements(b, ListInterface)
1

It should be possible to define ListInterface once and check it against
any arbitrary object, without having to fiddle with, or provide adapters

for the type/class of each object that does (de facto) implement it.=20

For example, I don't want to have to do __bindings__ on the list type,=20
UserList, any extension types designed to be used as lists, wrappers or=20
proxies on lists that are not instances of UserList, etc.=20



Declaring that a class implements an interface (this is, it promises
that=20
all its instances will satisfy it) still makes sense:


* For documentation: declaring compliance with an interface gives a very

  useful hint on the purpose and usage of the class.


* For safety: you may be afraid that satisfying some method signatures
alone=20
  doesn't guarantee that an object is safe to use in contexts where your

  interface is required.=20

  For example, you have this my.Shape interface with this draw() method.
There=20
  are too many classes around that provide a draw() method and which
won't=20
  work for the purposes of my.Shape (and that's why a simple check for
draw()
  is not enough either). Implicit interface satisfaction is not a
concern here,=20
  because it doesn't make sense that anyone has implemented useful
my.Shapes=20
  before knowing about your module. So it makes full sense for you to
require=20
  that any objects that want to implement my.Shape explicitly declare
so. This=20
  does not necessarily replace checking for draw(). The author of each
specific=20
  interface must decide her policy on this.


* For performance: you may want to cache the results of class checks.
You may
  even do without checking at all, but I don't think this is a good
idea.


I have my own ideas on how the syntax should be but I'd rather bounce
them and=20
let them mature off this list. If anyone is willing to put up a mailing
list or=20
just discuss these things privately, please let me know.


Esteban.


From skip@pobox.com  Thu Oct  3 13:30:12 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 3 Oct 2002 07:30:12 -0500
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
In-Reply-To: <3D9BB19D.4000401@tismer.com>
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz>
 <yu994rc4p8cc.fsf@europa.research.att.com>
 <3D9BB19D.4000401@tismer.com>
Message-ID: <15772.14420.986230.807066@localhost.localdomain>

Danger Will Robinson!  I know next to nothing about numerical analysis.  I
don't even play a numerical analyst on TV.

    >> So I'm suggesting that maybe it won't get bad as quickly if you keep
    >> only the significant bits :-)

    Christian> Yes. And this is what a rounding to float would do for you.

Okay, we have a tension here.  On the one hand, float representation
provides the convenience of bounded storage at the expense of loss of
precision.  Rationals, on the other hand, have the potential to prevent loss
of precision, but at the cost of potentially unbounded storage (and greatly
increased computational cost).  Is there a middle ground where you can say,
in effect, "give me rational numbers which are truncated to provide
precision no worse than N-bit floating point numbers"?

I'm sure that's stated poorly (and incorrectly), so here's an example.  Pi
is an irrational number, represented in Python (at least on the platform
closest to hand) as 3.1415926535897931.  Suppose I needed pi to this
precision though: 3.14159265358979323846264.  Could I somehow parameterize
rational numbers to give me the required precision - that is, numerators
and/or denominators were automatically truncated to provide the desired
precision and prevent unbounded growth (space and computational)?

Skip


From sholden@holdenweb.com  Thu Oct  3 13:52:01 2002
From: sholden@holdenweb.com (Steve Holden)
Date: Thu, 3 Oct 2002 08:52:01 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com> <3D9BB19D.4000401@tismer.com>
Message-ID: <00bf01c26adb$abf530b0$8701a8c0@holdenweb.com>

Christian Tismer <tismer@tismer.com> wrote ...
> Andrew Koenig wrote:
[rational representations]

> > Seriously, I don't know whether it would help in practice.
> > It might be that normalizing rationals from time to time would
> > be enough.
>
> Ok, not just zeros, but normalizing the fraction to no
> common denominator. That makes sense if lots of small
> numbers/fractions were multiplied and prime factors
> pile up.
> Not so with addition. This creates completely different
> prime factors all the time, and addition becomes much
> more expensive than multiplication when the common
> denominator must always be reduced to the minimum.
>
> gotta-be-expensive  -- chris
>
Next someone will suggest that we store rationals as a sequence of
coefficients of the prime factors. This would make primes really easy to
recognise (since they'd be a sequence of zeros followed by a one with a
single one as denominator).

but-it-might-slow-down-other-operations-a-bit-ly y'rs  - steve
-----------------------------------------------------------------------
Steve Holden                                  http://www.holdenweb.com/
Python Web Programming                 http://pydish.holdenweb.com/pwp/
Previous .sig file retired to                    www.homeforoldsigs.com
-----------------------------------------------------------------------




From skip@pobox.com  Thu Oct  3 14:02:23 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 3 Oct 2002 08:02:23 -0500
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210030327.g933Rpw21328@oma.cosc.canterbury.ac.nz>
References: <00fb01c26a8b$91f5d200$b2e97ad1@othello>
 <200210030327.g933Rpw21328@oma.cosc.canterbury.ac.nz>
Message-ID: <15772.16351.770660.727862@localhost.localdomain>

    Greg> Raymond Hettinger <python@rcn.com>:
    >> For that, I propose maxdenom=None to let the rationals grow without
    >> bound.

    Greg> A global setting for this would have all the same problems we've
    Greg> just gone over in relation to the "fuzz" setting.

If it was a builtin, different modules could simply set their own value at
global or local levels to override the default for the arithmetic they do.
Would that make numerical stability problems nearly impossible to debug?

Skip



From ark@research.att.com  Thu Oct  3 14:05:03 2002
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 3 Oct 2002 09:05:03 -0400 (EDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021003052829.GA59708@hishome.net> (message from Oren Tirosh on
 Thu, 3 Oct 2002 01:28:29 -0400)
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net>
Message-ID: <200210031305.g93D53o12559@europa.research.att.com>

Oren> Any calculation involving a floating point number should taint
Oren> the result with inexactitude and yield a floating point result.

Even comparisons?


From list-python@ccraig.org  Thu Oct  3 14:09:42 2002
From: list-python@ccraig.org (Christopher A. Craig)
Date: 03 Oct 2002 09:09:42 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <00bf01c26adb$abf530b0$8701a8c0@holdenweb.com>
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz>
 <yu994rc4p8cc.fsf@europa.research.att.com>
 <3D9BB19D.4000401@tismer.com>
 <00bf01c26adb$abf530b0$8701a8c0@holdenweb.com>
Message-ID: <t1wadlvvfc9.fsf@kermit.wreck.org>

"Steve Holden" <sholden@holdenweb.com> writes:

> Next someone will suggest that we store rationals as a sequence of
> coefficients of the prime factors. This would make primes really easy to
> recognise (since they'd be a sequence of zeros followed by a one with a
> single one as denominator).

It would make the gcd really fast too, even for extremely large numbers. :-)

-- 
Christopher A. Craig <list-python@ccraig.org>
"It's the one thing I understand, I guess." Bill Gates, about BASIC


From David Abrahams" <david.abrahams@rcn.com  Thu Oct  3 13:34:42 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Thu, 3 Oct 2002 08:34:42 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net> <200210031305.g93D53o12559@europa.research.att.com>
Message-ID: <08bb01c26ad9$6f44e9f0$6701a8c0@boostconsulting.com>

From: "Andrew Koenig" <ark@research.att.com>


> Oren> Any calculation involving a floating point number should taint
> Oren> the result with inexactitude and yield a floating point result.
>
> Even comparisons?

Well, comparisons are tainted already in some sense, because you don't know
how much error there is in any given number.

FWIW, Boost just accepted an interval arithmetic library which does
computation with a closed range of floating point numbers, making it
possible to know just how fuzzy any given computation is.


-----------------------------------------------------------
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com




From python-pep@ccraig.org  Thu Oct  3 14:20:27 2002
From: python-pep@ccraig.org (Christopher A. Craig)
Date: 03 Oct 2002 09:20:27 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021002221421.GA29958@thyrsus.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
Message-ID: <t1w3crnveuc.fsf@kermit.wreck.org>

"Eric S. Raymond" <esr@thyrsus.com> writes:

> APL faced this problem twenty-five years ago.  I like its solution; 
> a `fuzz' variable defining the close-enough-for-equality range.

For what it's worth, the current implementation is nothing this
complicated.  Like every other numeric operation, compares of
rationals with floats coerce to floats.  So as a result
rational(x)==float(x) iff float(rational(x))==float(x).

That is, of course, much more permissive than if we coerced the other
way and had rational(x)==float(x) iff rational(x)==rational(float(x)).

-- 
Christopher A. Craig <python-pep@ccraig.org>
"You could shoot Microsoft Office off the planet and this country would
 run better. You would see everyone standing around saying, 'I've got
 so much time now.' "  Scott McNealy (CEO of Sun)


From paul@prescod.net  Thu Oct  3 14:49:20 2002
From: paul@prescod.net (Paul Prescod)
Date: Thu, 03 Oct 2002 06:49:20 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz> <E17wyys-0007ei-00@mail.python.org>
Message-ID: <3D9C4AE0.2060406@prescod.net>

Alex Martelli wrote:
> On Thursday 03 October 2002 02:42 am, Greg Ewing wrote:
> 	...
> 
>>  1///3 --> rational
>>
>>(Okay, a 3-char operator is a bit verbose, but I can't
>>think of anything else that looks division-like just at
>>the moment.)
> 
> 
> 1\3 ... ?

I predict this will be the Perl 6 solution if they run into this problem:

http://www.myfonts.com/Character00F7Style1150.html

Today, it has a certain impractical elegance, which is not usually a 
phrase I associate with Perl.

But a few years from now it may be practical.

  Paul Prescod



From ark@research.att.com  Thu Oct  3 15:40:38 2002
From: ark@research.att.com (Andrew Koenig)
Date: 03 Oct 2002 10:40:38 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <08bb01c26ad9$6f44e9f0$6701a8c0@boostconsulting.com>
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net>
 <200210030216.g932G7707727@europa.research.att.com>
 <20021003052829.GA59708@hishome.net>
 <200210031305.g93D53o12559@europa.research.att.com>
 <08bb01c26ad9$6f44e9f0$6701a8c0@boostconsulting.com>
Message-ID: <yu99u1k34mc9.fsf@europa.research.att.com>

>> Even comparisons?

David> Well, comparisons are tainted already in some sense, because you don't know
David> how much error there is in any given number.

Nevertheless, there can be serious problems when comparisons are
not consistent.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From ark@research.att.com  Thu Oct  3 15:45:27 2002
From: ark@research.att.com (Andrew Koenig)
Date: 03 Oct 2002 10:45:27 -0400
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <t1w3crnveuc.fsf@kermit.wreck.org>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org>
 <20021002221421.GA29958@thyrsus.com>
 <t1w3crnveuc.fsf@kermit.wreck.org>
Message-ID: <yu99ptur4m48.fsf@europa.research.att.com>

Christopher> For what it's worth, the current implementation is
Christopher> nothing this complicated.  Like every other numeric
Christopher> operation, compares of rationals with floats coerce to
Christopher> floats.  So as a result rational(x)==float(x) iff
Christopher> float(rational(x))==float(x).

Attractive as this strategy may be from an implementation viewpoint,
it means that comparisons do not have the order-relation properties
that algorithms such as sort expect.

A similar problem exists today for comparisons between float and long,
by the way.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From guido@python.org  Thu Oct  3 15:56:48 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 10:56:48 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Thu, 03 Oct 2002 12:42:56 +1200."
 <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
Message-ID: <200210031456.g93Eumx20160@odiug.zope.com>

After reading most of the discussion, I don't know what to do about
this.  There are a number of different ideas that should be reviewed
separately: "classic" rationals, fixed-point decimal, a new kind of
floats with settable precision...

I tend to think that rationals and superfloats are sufficiently
esoteric that they should probably be relegated to an extension or
library module.  Fixed-point decimals have some nice properties, but
apparently implementing it isn't easy -- Aahz has been sitting on a
prototype approximately forever.  I hear that Tim's FixedPoint package
is now a SF project.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct  3 16:15:35 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 11:15:35 -0400
Subject: [Python-Dev] RFC: Option Parsing Libraries
In-Reply-To: Your message of "Wed, 02 Oct 2002 23:12:35 EDT."
 <Pine.GSO.4.33.0210022237190.14016-100000@maple.ces.clemson.edu>
References: <Pine.GSO.4.33.0210022237190.14016-100000@maple.ces.clemson.edu>
Message-ID: <200210031515.g93FFZU20358@odiug.zope.com>

> After releasing the GetPot 1.0 Version I did some research on other
> packages and I found the the Option Parsing RFC.

What's the Option Parsing RFC?

> I totally missed the discussion. How come that nobody contacted me
> about the GetPot package ?

Because we didn't know you existed?

> To get a python doc out should not be a problem at all. Not one single
> person, though, complained until now, since the C++ doc is totally
> equivalent.
> 
> Take a look at http://getpot.sourceforge.net for new features of the
> 1.0 Version. Except for the automated documenting feature (which may
> be a little intrusive in its format) I don't really see what Optik
> brings new on the table.
> 
> Guys, even though GetPot is designed for many languages, I'd like
> to join the command line parsing discussion.

Too late.  We've settled on Optik (under a different name).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@alum.mit.edu  Thu Oct  3 16:35:20 2002
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 3 Oct 2002 11:35:20 -0400
Subject: [Python-Dev] RFC: Option Parsing Libraries
In-Reply-To: <Pine.GSO.4.33.0210022237190.14016-100000@maple.ces.clemson.edu>
References: <Pine.GSO.4.33.0210022237190.14016-100000@maple.ces.clemson.edu>
Message-ID: <15772.25528.921232.130849@slothrop.zope.com>

>>>>> "FS" == Frank S <fschaef@ces.clemson.edu> writes:

  FS> After releasing the GetPot 1.0 Version I did some research on
  FS> other packages and I found the the Option Parsing RFC. I totally
  FS> missed the discussion. How come that nobody contacted me about
  FS> the GetPot package ?

How come you didn't contact us about the GetPot package?

The discussion in February began when Greg Ward proposed Optik as a
standard Python module.  Guido suggested that there be a general call
on the python lists: If there were competing options packages, would
people let us know about them.  I had never heard of GetPot until I
received your mail.  (And if I had heard of it, I would have guessed
it was illegal except for medical purposes <wink>.)

So: There was general consensus that Optik should be added to the
standard library.  Is there a reason to include GetPot instead of
Optik?  Procedural issue: The (L)GPL license is unacceptable for
standard library modules.  I'm sure there are technical issues.

Jeremy



From mal@lemburg.com  Thu Oct  3 16:27:10 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 03 Oct 2002 17:27:10 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com>              <oqsmzo4fgu.fsf@carouge.sram.qc.ca> <200210030022.g930Mrl06112@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3D9C61CE.9010909@lemburg.com>

Guido van Rossum wrote:
>>But those feelings are no proof of anything.  How do we get the
>>confirmation that using rationals in Python would be easy going and
>>innocuous in practice, beforehand?
> 
> 
> By adding them to the language but as an isolated type.  The right
> conversions should happen when you mix rationals with other types of
> numbers (int/long -> rational -> float -> complex), but no operations
> should return rationals unless a rational goes in.

+1

Note that I started to work on mxNumber to get a feeling for
how well rationals et al. fit the existing world. Turns out
that having separate types is a goog thing. Here's the
coercion scheme I'm using:

                  mx.Number.Float
                        ^
                        |
        --------> Python float
       |                ^
       |                |
       |         mx.Number.Rational
       |                ^
       |                |
Python long --> mx.Number.Integer
       ^                ^
       |                |
        --------  Python integer

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Thu Oct  3 16:36:08 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 11:36:08 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Thu, 03 Oct 2002 17:27:10 +0200."
 <3D9C61CE.9010909@lemburg.com>
References: <t1wn0pwwkwy.fsf@kermit.wreck.org> <20021002221421.GA29958@thyrsus.com> <oqsmzo4fgu.fsf@carouge.sram.qc.ca> <200210030022.g930Mrl06112@pcp02138704pcs.reston01.va.comcast.net>
 <3D9C61CE.9010909@lemburg.com>
Message-ID: <200210031536.g93Fa8d22105@odiug.zope.com>

> Note that I started to work on mxNumber to get a feeling for
> how well rationals et al. fit the existing world. Turns out
> that having separate types is a goo[d] thing.

Any mxNumber users out there who have experience with mxRational?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From oren-py-d@hishome.net  Thu Oct  3 17:10:55 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 19:10:55 +0300
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210031305.g93D53o12559@europa.research.att.com>; from ark@research.att.com on Thu, Oct 03, 2002 at 09:05:03AM -0400
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net> <200210031305.g93D53o12559@europa.research.att.com>
Message-ID: <20021003191055.A14342@hishome.net>

On Thu, Oct 03, 2002 at 09:05:03AM -0400, Andrew Koenig wrote:
> Oren> Any calculation involving a floating point number should taint
> Oren> the result with inexactitude and yield a floating point result.
> 
> Even comparisons?

Comparisons could produce a third boolean value called 'Undetermined':

result = A>B
if result is Undetermined:
   print "Sorry, difference between A and B is below the error margin"
   print "I cannot determine which one is greater."
else:
   print "%s is greater than %s" % tuple(('BA','AB')[result])





[Taking off absurdly-purist-hat :-]

This actually gives me a serious idea: a floating-point type that helps
keep track of error magnitude. It's a pair of floats (like a complex
number) where the second number is an estimate or limit on the absolute 
value of errors in the first and is updated by all operators. An error
tracking replacement for the math module would also be required.

	Oren



From ark@research.att.com  Thu Oct  3 17:13:33 2002
From: ark@research.att.com (Andrew Koenig)
Date: Thu, 3 Oct 2002 12:13:33 -0400 (EDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021003191055.A14342@hishome.net> (message from Oren Tirosh on
 Thu, 3 Oct 2002 19:10:55 +0300)
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net> <200210031305.g93D53o12559@europa.research.att.com> <20021003191055.A14342@hishome.net>
Message-ID: <200210031613.g93GDXB13582@europa.research.att.com>

Oren> On Thu, Oct 03, 2002 at 09:05:03AM -0400, Andrew Koenig wrote:
Oren> Any calculation involving a floating point number should taint
Oren> the result with inexactitude and yield a floating point result.
>> 
>> Even comparisons?

Oren> Comparisons could produce a third boolean value called 'Undetermined':

I wonder what would be the definition of sort in the presence
of such comparisons?  Would it be possible to implement it
efficiently?


From David Abrahams" <david.abrahams@rcn.com  Thu Oct  3 16:58:27 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Thu, 3 Oct 2002 11:58:27 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net> <200210031305.g93D53o12559@europa.research.att.com> <20021003191055.A14342@hishome.net> <200210031613.g93GDXB13582@europa.research.att.com>
Message-ID: <098501c26af5$b7d1d8b0$6701a8c0@boostconsulting.com>

From: "Andrew Koenig" <ark@research.att.com>


> Oren> On Thu, Oct 03, 2002 at 09:05:03AM -0400, Andrew Koenig wrote:
> Oren> Any calculation involving a floating point number should taint
> Oren> the result with inexactitude and yield a floating point result.
> >>
> >> Even comparisons?
>
> Oren> Comparisons could produce a third boolean value called
'Undetermined':

It turns out that once you allow ternary logic there are too many
reasonable meanings for the comparison operators. We had this long
discussion around the Boost interval library. Eventually Douglas Gregor
came up with a solution which seemed to satisfy everyone. I can't find it
now but if anyone's interested I'll ask him to summarize the issues and
solution.

-----------------------------------------------------------
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com




From tim.one@comcast.net  Thu Oct  3 17:50:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 12:50:38 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <20021003191055.A14342@hishome.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEGDDLAA.tim.one@comcast.net>

[Oren Tirosh]
> Comparisons could produce a third boolean value called 'Undetermined':

Why?  What real numeric problem are you trying to solve?

> result = A>B
> if result is Undetermined:
>    print "Sorry, difference between A and B is below the error margin"
>    print "I cannot determine which one is greater."
> else:
>    print "%s is greater than %s" % tuple(('BA','AB')[result])

> [Taking off absurdly-purist-hat :-]
>
> This actually gives me a serious idea: a floating-point type that helps
> keep track of error magnitude. It's a pair of floats (like a complex
> number) where the second number is an estimate or limit on the absolute
> value of errors in the first and is updated by all operators.

It's called interval arithmetic; about 30 years ago I had to implement it
for a numerical analysis class assignment, in assembler for a Univac 1108;
the interface to the Fortran compiler indeed tricked it into storing the
bounds in complex numbrs; *efficient* support for interval arithmetic is why
the 754 std requires the directed (to-plus-inf, to-minus-inf) rounding
modes; it hasn't caught on, and the current incarnation of the 754 committee
is thinking about removing this requirement; in the hands of an expert it
can be very powerful; in the hands of a non-expert, the usual result is that
the error bounds grow so large as to render the computation useless.

> An error tracking replacement for the math module would also be required.

    http://www.uni-karlsruhe.de/~iam/html/language/pxsc.html

A single-rounding dot product proved crucial to their successes.



From oren-py-d@hishome.net  Thu Oct  3 17:56:02 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 12:56:02 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210031613.g93GDXB13582@europa.research.att.com>
References: <LNBBLJKPBEHFEDALKOLCOEHABIAB.tim.one@comcast.net> <200210030216.g932G7707727@europa.research.att.com> <20021003052829.GA59708@hishome.net> <200210031305.g93D53o12559@europa.research.att.com> <20021003191055.A14342@hishome.net> <200210031613.g93GDXB13582@europa.research.att.com>
Message-ID: <20021003165602.GA50630@hishome.net>

On Thu, Oct 03, 2002 at 12:13:33PM -0400, Andrew Koenig wrote:
> Oren> On Thu, Oct 03, 2002 at 09:05:03AM -0400, Andrew Koenig wrote:
> Oren> Any calculation involving a floating point number should taint
> Oren> the result with inexactitude and yield a floating point result.
> >> 
> >> Even comparisons?
> 
> Oren> Comparisons could produce a third boolean value called 'Undetermined':
> 
> I wonder what would be the definition of sort in the presence
> of such comparisons?  Would it be possible to implement it
> efficiently?

That depends on the exact constraints of this lossy ordering. But let's
take it even further!  Let's define fuzzy booleans! an a completely
Undetermined result is 0.5, meaning the comparison yielded zero
information about the order of the two inputs but if their intervals of
uncertainty are only partially overlapping we could get an estimate of
the probability that A is greater than B. No, wait! to get better
probability estimates we can define each floating point number as a CDF.
This will give us information not only about the interval of uncertainty
but about the actual distribution of errors. Uniform intervals are, after
all, just a special case - we might want to represent numbers with
gaussian errors, etc. I know it will be very useful for me in signal
processing to add the gaussian noise, run the signal through a simulation
of the error correction algorithm, have each undetermined comparison 
explore both branches in simulated parallel universes and sum the 
results according to their probabilities.

I am a bit concerned about the performance or simple operations like
addition when they require arbitrary precision convolutions of
distribution functions, though. 

	Oren



From oren-py-d@hishome.net  Thu Oct  3 18:43:38 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 3 Oct 2002 13:43:38 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEGDDLAA.tim.one@comcast.net>
References: <20021003191055.A14342@hishome.net> <BIEJKCLHCIOIHAGOKOLHMEGDDLAA.tim.one@comcast.net>
Message-ID: <20021003174338.GB50630@hishome.net>

On Thu, Oct 03, 2002 at 12:50:38PM -0400, Tim Peters wrote:
> [Oren Tirosh]
> > Comparisons could produce a third boolean value called 'Undetermined':
> 
> Why?  What real numeric problem are you trying to solve?

I guess the smiley was camouflaged too well as the closing bracket :-)

> It's called interval arithmetic; about 30 years ago I had to implement it
> for a numerical analysis class assignment, in assembler for a Univac 1108;
> the interface to the Fortran compiler indeed tricked it into storing the
> bounds in complex numbrs; *efficient* support for interval arithmetic is why
> the 754 std requires the directed (to-plus-inf, to-minus-inf) rounding
> modes; it hasn't caught on, and the current incarnation of the 754 committee
> is thinking about removing this requirement; in the hands of an expert it
> can be very powerful; in the hands of a non-expert, the usual result is that
> the error bounds grow so large as to render the computation useless.

I was thinking more of estimates based on the assumption that errors are 
normally distributed and track standard deviation. SD grows more slowly
(i.e. realistically) than true error bounds.  Have you ever tried measuring 
noise or bit error rates in a simulation that runs thousands of times 
slower than real time? (change a paremeter, rinse, repeat). Something that 
quickly estimates the error energy could be useful.

I guess that "everything is gaussian" is an approximation only an engineer 
could love and makes mathematicians shake their heads in disgust :-)

	Oren


From tismer@tismer.com  Thu Oct  3 19:20:54 2002
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 03 Oct 2002 11:20:54 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com> <3D9BB19D.4000401@tismer.com> <00ac01c26a89$6117aac0$b2e97ad1@othello>
Message-ID: <3D9C8A86.9020805@tismer.com>

Raymond Hettinger wrote:
> From: "Christian Tismer" <tismer@tismer.com>
> 
>>>Greg> The problem as I see it is not the magnitude of the number, but
>>>Greg> that the number of significant bits grows without bound, if you
>>>Greg> never throw any of them away.
>>>
>>>So I'm suggesting that maybe it won't get bad as quickly if
>>>you keep only the significant bits :-)
>>
>>Yes. And this is what a rounding to float would do for you.
> 
> 
> That is unless you wanted more precision than comes with a float.
> Normalization to some arbitrary maximum denominator magnitude
> would be a more general purpose solution.

Sure, this is right.
So rationals are seriously considered? Would be great.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tim.one@comcast.net  Thu Oct  3 19:55:46 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 14:55:46 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <15772.14420.986230.807066@localhost.localdomain>
Message-ID: <BIEJKCLHCIOIHAGOKOLHAEGJDLAA.tim.one@comcast.net>

[Skip Montanaro]
> ...
> Is there a middle ground where you can say, in effect, "give me rational
> numbers which are truncated to provide precision no worse than N-bit
> floating point numbers"?

Yes, so-called "fixed slash" and "floating slash" arithmetics do exactly
that.  BTW, they tend to give exactly the right results despite intermediate
roundoff errors when the *problem* domain is such that the correct results
are simple fractions.

But this is just another thing that gets mentioned every time this
discussion resurrects, so that's all I'm going to say about it this time
around.  There are literally hundreds of schemes for computer arithmetics
that have been tried and found genuinely useful for *some* things.  The only
ones on the table with any claim to broad usefulness are rationals and
decimal fp.



From tismer@tismer.com  Thu Oct  3 20:37:32 2002
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 03 Oct 2002 12:37:32 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210030222.g932MUd20787@oma.cosc.canterbury.ac.nz> <yu994rc4p8cc.fsf@europa.research.att.com> <3D9BB19D.4000401@tismer.com> <00bf01c26adb$abf530b0$8701a8c0@holdenweb.com>
Message-ID: <3D9C9C7C.7060609@tismer.com>

Steve Holden wrote:

[me about normalization]

> Next someone will suggest that we store rationals as a sequence of
> coefficients of the prime factors. This would make primes really easy to
> recognise (since they'd be a sequence of zeros followed by a one with a
> single one as denominator).

This makes of course pretty much sense if the only
operations you want are multiplication/division
which becomes very very cheap, while addition becomes
unaffordably expensive :-)

> but-it-might-slow-down-other-operations-a-bit-ly y'rs  - steve

will-not-suggest-it--tried-all-of-these-decades-ago - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tim.one@comcast.net  Thu Oct  3 20:53:16 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 15:53:16 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <20021003174338.GB50630@hishome.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHEEGNDLAA.tim.one@comcast.net>

[Oren Tirosh]
> ...
> I was thinking more of estimates based on the assumption that errors are
> normally distributed and track standard deviation. SD grows more slowly
> (i.e. realistically) than true error bounds.  Have you ever tried
> measuring noise or bit error rates in a simulation that runs thousands
> of times slower than real time? (change a paremeter, rinse, repeat).
> Something that quickly estimates the error energy could be useful.

If you run Emacs, get Dave Gillespie's woefully underappreciated calc-mode
and look up "error forms".  He did a pretty thorough implementation of the
concept, including for transcendentals and complex numbers.

> I guess that "everything is gaussian" is an approximation only an
> engineer  could love and makes mathematicians shake their heads
> in disgust :-)

Pure mathematicians, yes; applied mathematicians would chuckle at how
cautious you are <wink>.



From peter@norvig.com  Thu Oct  3 21:15:01 2002
From: peter@norvig.com (Peter Norvig)
Date: Thu, 03 Oct 2002 13:15:01 -0700
Subject: [Python-Dev] PEP239 (Rational Numbers) Reference Implementation and new issues
References: <20021003014507.930.92987.Mailman@mail.python.org>
Message-ID: <3D9CA545.8080601@norvig.com>

I agree with pinard, although I think it is a manageable problem. I went 
through this transition once, from Symbolics Lisp, in which integer 
division truncates, to Common Lisp, in which it produces a rational. 
 After only an hour or two of fiddling with my various programs, 
everything seemed to work; I was very happy.  But after a few days, the 
machine started to slow down mysteriously, especially on wake-up.  It 
took a while to discover the culprit was my screensaver program (which I 
hadn't checked) -- it was displaying fish on the screen, and the 
coordinates of the fish had become rationals quotients of thousand-digit 
numbers.

-Peter

pinard@iro.umontreal.ca writes:
 > While I agree with the theoretical arguments, I have the practical 
fear that
 > rationals could grow very big, rather quickly, in the course of a long
 > computation involving them in various ways.  By big, I mean the numerator
 > and denominator of the fraction taken in isolation, not the number 
itself.



From mmatus@dinha.acms.arizona.edu  Thu Oct  3 22:27:48 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 14:27:48 -0700
Subject: [Python-Dev] the not operator (and the __not__ special method)
Message-ID: <3D9CB654.8050306@acms.arizona.edu>

Hello:

The following test code:


class Hello:
    def __not__(self):
        return "calling Hello.__not__"

    def __neg__(self):
        return "calling Hello.__neg__"

a = Hello()
print -a
print not a


shows that python doesn't call the __not__ special method
in a 'not' operator statement.

The Modules/operator.c file, line 234, says:

spam2(neg,__neg__, "neg(a) -- Same as -a.")
spam2(pos,__pos__, "pos(a) -- Same as +a.")
spam2(abs,__abs__, "abs(a) -- Same as abs(a).")
spam2(inv,__inv__, "inv(a) -- Same as ~a.")
spam2(invert,__invert__, "invert(a) -- Same as ~a.")
spam2(lshift,__lshift__, "lshift(a, b) -- Same as a << b.")
spam2(rshift,__rshift__, "rshift(a, b) -- Same as a >> b.")
spam2(not_,__not__, "not_(a) -- Same as not a.")


so, one can expect that like "-a" is equivalent ot "a.__neg__()",
then  "not a" is equivalent to "a.__not__()",  right?


Another question, I notice than  "a or b" and "a and b" are
not equivalent to "a | b" and "a & b", since the last ones call
the methods __or__ and __and__ if they are defined, but
the "literal forms" never call them.  Is this intentional?, if
that is the case, I guess a big and clear note is needed
in the documentation.

Maybe the reason that the literal forms (and,or) are not equivalent to
the operator forms(&,|)  is preventing python to porperly
resolve the "not" operation.

And just for symmetry considerations, given that python has the
pairs (and, &) and (or, |), does anybody considering to introduce
the ! operator, so you can have the equivalent (not, !) too?



Marcelo






From marklists@mceahern.com  Thu Oct  3 22:36:01 2002
From: marklists@mceahern.com (Mark McEahern)
Date: Thu, 3 Oct 2002 16:36:01 -0500
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <3D9CB654.8050306@acms.arizona.edu>
Message-ID: <JHEOKEOOLIGLDHCMAHMOOEJDEDAA.marklists@mceahern.com>

http://www.python.org/doc/current/lib/module-operator.html

says

not_(o)
__not__(o)
Return the outcome of not o. (Note that there is no __not__() method for
object instances; only the interpreter core defines this operation. The
result is affected by the __nonzero__() and __len__() methods.)

// m



From guido@python.org  Thu Oct  3 22:40:27 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 17:40:27 -0400
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: Your message of "Thu, 03 Oct 2002 14:27:48 PDT."
 <3D9CB654.8050306@acms.arizona.edu>
References: <3D9CB654.8050306@acms.arizona.edu>
Message-ID: <200210032140.g93LeR930972@odiug.zope.com>

> The following test code:
> 
> 
> class Hello:
>     def __not__(self):
>         return "calling Hello.__not__"

The reference manual (chapter 3) does not define __not__ as a special
operator.

>     def __neg__(self):
>         return "calling Hello.__neg__"
> 
> a = Hello()
> print -a
> print not a
> 
> 
> shows that python doesn't call the __not__ special method
> in a 'not' operator statement.
> 
> The Modules/operator.c file, line 234, says:
> 
> spam2(neg,__neg__, "neg(a) -- Same as -a.")
> spam2(pos,__pos__, "pos(a) -- Same as +a.")
> spam2(abs,__abs__, "abs(a) -- Same as abs(a).")
> spam2(inv,__inv__, "inv(a) -- Same as ~a.")
> spam2(invert,__invert__, "invert(a) -- Same as ~a.")
> spam2(lshift,__lshift__, "lshift(a, b) -- Same as a << b.")
> spam2(rshift,__rshift__, "rshift(a, b) -- Same as a >> b.")
> spam2(not_,__not__, "not_(a) -- Same as not a.")

The operator module uses __not__ as a function name for symmetry with
its other function names, not because __not__ is a special operator.

> so, one can expect that like "-a" is equivalent ot "a.__neg__()",
> then  "not a" is equivalent to "a.__not__()",  right?

No.

> Another question, I notice than  "a or b" and "a and b" are
> not equivalent to "a | b" and "a & b", since the last ones call
> the methods __or__ and __and__ if they are defined, but
> the "literal forms" never call them.  Is this intentional?, if
> that is the case, I guess a big and clear note is needed
> in the documentation.

That all depends.  The non-equivalency between 'or' and '|' (and
between 'and' and '&') is quite well documented.  'or' and 'and'
cannot invoke an operator, because the right operand is not evaluated
at all when the left argument determines the outcome.

> Maybe the reason that the literal forms (and,or) are not equivalent to
> the operator forms(&,|)  is preventing python to porperly
> resolve the "not" operation.

No.

> And just for symmetry considerations, given that python has the
> pairs (and, &) and (or, |), does anybody considering to introduce
> the ! operator, so you can have the equivalent (not, !) too?

No, there's no shortcut possibility for not as there is for 'and' and
'or'.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From brian@sweetapp.com  Thu Oct  3 22:50:54 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Thu, 03 Oct 2002 14:50:54 -0700
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <3D9CB654.8050306@acms.arizona.edu>
Message-ID: <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>

> shows that python doesn't call the __not__ special method
> in a 'not' operator statement.

Python calls the special __nonzero__ method so check the truth value of
an object.

> Another question, I notice than  "a or b" and "a and b" are
> not equivalent to "a | b" and "a & b", since the last ones call
> the methods __or__ and __and__ if they are defined, but
> the "literal forms" never call them.  Is this intentional?, if
> that is the case, I guess a big and clear note is needed
> in the documentation.

"and" and "or" are logical operators, while "|" and "&" are bitwise
operators. 

>>> 13 & 14
12
>>> 13 and 14
14

> And just for symmetry considerations, given that python has the
> pairs (and, &) and (or, |), does anybody considering to introduce
> the ! operator, so you can have the equivalent (not, !) too?

There is a bitwise not operator: ~

>>> ~2
-3

Cheers,
Brian



From marklists@mceahern.com  Thu Oct  3 22:59:09 2002
From: marklists@mceahern.com (Mark McEahern)
Date: Thu, 3 Oct 2002 16:59:09 -0500
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>
Message-ID: <JHEOKEOOLIGLDHCMAHMOGEJGEDAA.marklists@mceahern.com>

> > And just for symmetry considerations, given that python has the
> > pairs (and, &) and (or, |), does anybody considering to introduce
> > the ! operator, so you can have the equivalent (not, !) too?
> 
> There is a bitwise not operator: ~
> 
> >>> ~2
> -3

Yeah, isn't ! reserved for shouting anyway?

// m


From guido@python.org  Thu Oct  3 23:03:45 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 18:03:45 -0400
Subject: [Python-Dev] perplexed by mro
In-Reply-To: Your message of "Tue, 01 Oct 2002 21:59:00 +0200."
 <01e101c26984$fc43da60$6d94fea9@newmexico>
References: <01e101c26984$fc43da60$6d94fea9@newmexico>
Message-ID: <200210032203.g93M3jr31166@odiug.zope.com>

> I'm trying to wrap my head around the mro computation in 2.2.

Thanks for doing this!  You've stumbled upon a place where I'm no
expert, and where my intuition about the equivalence between
algorithms turned out to be wrong.

> First issue: PEP 253 and the descrintro document describe an
> algorithm, typeobject.c implements a rather different
> one. Descrintro mentions that.
> 
> The algorithm described by descrintro, PEP253:
> 
> "Using the classic lookup rule, construct the list of classes that
> would be searched, including duplicates. Now for each class that
> occurs in the list multiple times, remove all occurrences except for
> the last."
> 
> which, if I understand correctly, means : do a left-right dfs of the
> class hierarchy and then discard all but the last occurence of each
> class.
> 
> Now descrintro says that the result of this and the real algorithm
> in typeobject are the same, unless
> 
> "two given base classes occur in a different order in the
> inheritance list of two different derived classes, and those derived
> classes are both inherited by yet another class"
> 
> But here is an example where the criterium does not apply but still
> the algorithms' results diverge:
> 
> >>> cpl.ex5(globals())
> class a(object): pass
> class b(object): pass
> class c(object): pass
> class x(a): pass
> class y(a): pass
> class z(x,b,y,c): pass
> >>> cpl.names(z.__mro__) # just the class names, real 2.2 mro algo
> ['z', 'x', 'y', 'a', 'b', 'c', 'object']
> >>> cpl.names(cpl.keeplast(cpl.lrdfs(z))) # descrintro simple algo
> ['z', 'x', 'b', 'y', 'a', 'c', 'object']

I wanted to say that the naive algorithm is "better" than the 2.2 algo
here because it doesn't reverse the order of b and y given by z's
bases.  But let's have a look at an example.

Suppose there's a method m that's defined in object, and overridden by
class y as well as in b.  What is z.m?  The 2.2 algo says y.m, the
naive algorithm says b.m.  Hard to decide which is better.  Now let's
assume m is *also* overridden in a, so that y.m overrides a.m
overrides object.m.  Now if we searched b before y (naive algo),
it is arguably strange if the 'super' call chain went from y.m via b.m
to a.m.

But I'm not sure how to formalize this intuition.

> I was not able to find a new rule. My intuition is that when the
> result of the real mro has something to do with the dfs, it is sure
> from the beginning of it not the end, so the relation of the two
> algos is unclear.
> 
> In general I have an hard time trying to predict the result of the
> real mro algo.
> 
> Does it a have a list of constraints it enforces/preserves, of the
> qualities it has?

The book "Putting Metaclasses To Work" that I always quote about this
has the algorithm and the rules it tries to enforce.  I don't have the
book handy here so I can't quote it.  I'm pretty sure that I
implemented their merge algo correctly, EXCEPT that I don't raise an
error or what they call "serious order conflicts".  There are no
serious order conflicts in this example though.

> I have tried to compare it to the algorithms used by Dylan and CLOS
> and others, described in this paper
> 
> http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
> 
> they are described in terms such as quality criteria and constraints
> they preserve.

Wish I could spend the time to study and understand all that.

> Let's consider this example:
> 
> >>> cpl.ex9(globals())
> class a(object): pass
> class b(object): pass
> class c(object): pass
> class d(object): pass
> class e(object): pass
> class k1(a,b,c): pass
> class k2(d,b,e): pass
> class k3(d,a): pass
> class z(k1,k2,k3): pass
> >>> cpl.names(z.__mro__)
> ['z', 'k1', 'k3', 'a', 'k2', 'd', 'b', 'c', 'e', 'object']

I think this one *does* have an order conflict, and in that case all
bets are off about the book's algo.  If you look at k1 and k2, clearly
d must come after a; yet k3 wants it after a.

> the thing that leaves me most perplexed is that k3 appears before
> k2. All other algortihms I know would put k2 before k3.  E.g. C3
> (see paper) ...
> 
> >>> cpl.names(cpl.c3_cpl(z))
> ['z', 'k1', 'k2', 'k3', 'd', 'a', 'b', 'c', 'e', 'object']
> 
> or the descrintro simple algo:
> 
> >>> cpl.names(cpl.keeplast(cpl.lrdfs(z)))
> ['z', 'k1', 'c', 'k2', 'b', 'e', 'k3', 'd', 'a', 'object']
> 
> This means that it does not preserve the local precedence lists
> (i.e. the inheritance list), it is not monotonic* because otherwise
> it would put d before a, it seems to preserve "some" aspects of the
> dfs traversal a,d vs d,a but still it puts k3 before k2.
> 
> The other hypothesis, is that in this case, the algorithm should
> fail, because it cannot merge the result of mro(k1) merged with
> mro(k2) with mro(k3), because the latter is inconsistent with what
> obtained so far?

Yes.  I've so far had a "don't do that then" attitude rather than a
"detect all errors" attitude here -- mostly because the order conflict
detection code is hairy to write in C.

> >>> l=cpl.names(k1.__mro__)
> >>> l
> ['k1', 'a', 'b', 'c', 'object']
> >>> r=cpl.names(k2.__mro__)
> >>> cpl.conservative_merge(l,r)
>  ...
> >>> l
> ['k1', 'a', 'k2', 'd', 'b', 'c', 'e', 'object']
> 
> vs.
> 
> >>> cpl.names(k3.__mro__)
> ['k3', 'd', 'a', 'object']
> 
> a<d is incosistent with d<a
> 
> But then this inconsistency rule is different/subtler than the one
> sketched in descrintro.
> 
> So can someone (Guido?) summarize the qualities of this rule? is
> there a paper that describes it? or should I buy the book?

Buy the book.  Last time I looked it was on sale (used) at Amazon.com.

> regards.
> 
> * the mro of a subclass is an extension without re-ordering of the
>   mros of the superclasses.

(I don't think your second post has anything else to which I need to
respond, right?)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mmatus@dinha.acms.arizona.edu  Thu Oct  3 23:30:34 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 15:30:34 -0700
Subject: [Python-Dev] the not operator (and the __not__ special method)
References: <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>
Message-ID: <3D9CC50A.2090208@acms.arizona.edu>

Thanks to everybody for your fast and helpful answer.

Clearly I was confused with the similarity between the names  'and': __and__
and  'or' : __or__, and I was thinking they were related, but that is 
not the case.

For now, I can implement my logical classes by using the bitwise 
operators '&,|,~' .

I forgot to say, but what I want to do is something like

    x = vfncs.Identity()                                 # a class instance
    b = x + 2                                                # another 
class  instance
    cond = (sin(x) <= 0.5 ) & ( b > 0.5 )    # another class instance
 
    print  x(0.3)           ==>   0.3
    print  b(0.3)          ==>   2.3
    print  cond(0.3)    ==>   1                    

where x, b and cond are "virtual functions" than can be evaluated  like
lambda functions after they are defined. The difference is that the are 
build from a C++
library, and they can be passed back to the C++ code where
they will be evaluated directly.

Anyway, given that now python has the "rich comparison" engine, where the
operator <=,>,<,>= can return anything you want (and thanks for that, 
because
I can do what I am doing now) and more boolean capabilities are been 
included
(I read that the boolean type is coming soon), I guess it will be also 
good to
have the special methods  for the logical operations 'and', 'or' and 'not',
(something like __land__, __lor__ and __lnot__ to distinguish them from
the bitwise versions __and__ and __or__).

So, if  you overload the proper logical methods, you can write something 
like

  cond = not (sin(x) > 0.5 ) and ( b <= 0.5 )

where 'not' and 'and', like '>' and '<=', wil return anything you want.


Marcelo


Brian Quinlan wrote:

>>shows that python doesn't call the __not__ special method
>>in a 'not' operator statement.
>>    
>>
>
>Python calls the special __nonzero__ method so check the truth value of
>an object.
>
>  
>
>>Another question, I notice than  "a or b" and "a and b" are
>>not equivalent to "a | b" and "a & b", since the last ones call
>>the methods __or__ and __and__ if they are defined, but
>>the "literal forms" never call them.  Is this intentional?, if
>>that is the case, I guess a big and clear note is needed
>>in the documentation.
>>    
>>
>
>"and" and "or" are logical operators, while "|" and "&" are bitwise
>operators. 
>
>  
>
>>>>13 & 14
>>>>        
>>>>
>12
>  
>
>>>>13 and 14
>>>>        
>>>>
>14
>
>  
>
>>And just for symmetry considerations, given that python has the
>>pairs (and, &) and (or, |), does anybody considering to introduce
>>the ! operator, so you can have the equivalent (not, !) too?
>>    
>>
>
>There is a bitwise not operator: ~
>
>  
>
>>>>~2
>>>>        
>>>>
>-3
>
>Cheers,
>Brian
>
>  
>









From barry@python.org  Fri Oct  4 00:02:54 2002
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 3 Oct 2002 19:02:54 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
References: <20021003174338.GB50630@hishome.net>
 <BIEJKCLHCIOIHAGOKOLHEEGNDLAA.tim.one@comcast.net>
Message-ID: <15772.52382.121534.278816@gargle.gargle.HOWL>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    TP> If you run Emacs, get Dave Gillespie's woefully
    TP> underappreciated calc-mode and look up "error forms".  He did
    TP> a pretty thorough implementation of the concept, including for
    TP> transcendentals and complex numbers.

calc-mode comes with XEmacs btw, and I still don't know what does and
doesn't come with FSF Emacs.  calc-mode is (and has been for years)
wonderful even to add a bunch of numbers!  It's nice to hear that Tim
had a hand in it.

-Barry


From greg@cosc.canterbury.ac.nz  Fri Oct  4 00:14:30 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Oct 2002 11:14:30 +1200 (NZST)
Subject: [Python-Dev] perplexed by mro
In-Reply-To: <200210032203.g93M3jr31166@odiug.zope.com>
Message-ID: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz>

Guido:

> Now let's assume m is *also* overridden in a, so that y.m overrides
> a.m overrides object.m.  Now if we searched b before y (naive algo),
> it is arguably strange if the 'super' call chain went from y.m via b.m
> to a.m.

I don't understand that. According to the diagram I just
scribbled, the mro from the documented algorithm comes out as

  z, x, b, y, a, c, object

so the super chain would be b.m, y.m, a.m, object.m.

(Which is *also* wrong according to my intuition, by the way --
I would expect a super call inside b.m to go to object.m,
and not depend on z's inheritance structure -- but that's a
separate issue.)

Anyway, why not just use the algorithm you said you were
using in the docs -- i.e. traverse the inheritance hierarchy
and build the mro afresh each time, rather than trying to
merge the mro's of the superclasses.

If it's an issue of which algorithm is the one that should
be used, I vote for the documented one, because it's
simple and easy to explain! If it leads to odd things
happening in some cases, at least it's clear what odd
things will happen and why -- as opposed to other odd
things happening for mysterious reasons.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Fri Oct  4 00:17:10 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Oct 2002 11:17:10 +1200 (NZST)
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <3D9CC50A.2090208@acms.arizona.edu>
Message-ID: <200210032317.g93NHAr29361@oma.cosc.canterbury.ac.nz>

Marcelo Matus <mmatus@dinha.acms.arizona.edu>:

> I guess it will be also good to have the special methods for the
> logical operations 'and', 'or' and 'not', (something like __land__,
> __lor__ and __lnot__ to distinguish them from the bitwise versions
> __and__ and __or__).

This is not easy, because the 'and' and 'or' operators
have to be capable of short-circuiting -- which is the
reason they don't already have __xxx__ methods.

But I have some ideas on how it could be done, which I
hope to get around to trying out one day.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From aahz@pythoncraft.com  Fri Oct  4 00:22:15 2002
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 3 Oct 2002 19:22:15 -0400
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>
References: <3D9CB654.8050306@acms.arizona.edu> <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>
Message-ID: <20021003232215.GA7735@panix.com>

On Thu, Oct 03, 2002, Brian Quinlan wrote:
>
> > shows that python doesn't call the __not__ special method
> > in a 'not' operator statement.
> 
> Python calls the special __nonzero__ method so check the truth value of
> an object.

Mark McEahern provided the full answer: Python uses __len__, too.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/


From mmatus@dinha.acms.arizona.edu  Fri Oct  4 00:45:56 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 16:45:56 -0700
Subject: [Python-Dev] PEP 285 (Adding a bool type)  extending it further
Message-ID: <3D9CD6B4.2050807@acms.arizona.edu>

Given my silly problems trying to write a class which overwrite logical 
operations
like 'and', 'or' and 'not', which is not possible by now

(see the discussion following

http://mail.python.org/pipermail/python-dev/2002-October/029122.html )


I would like to propose to extend the PEP 285
to add boolean operations that can be overloaded as the rest of the
operations in python.

The operations in question refers of course to  'and', 'or' and 'not'.

I guess this will involve to add a structure similar to PyNumberMethods
or PySequenceMethods, but for boolean methods, something like:

typedef struct {
    binaryfunc bl_and;
    binaryfunc bl_or;
    binaryfunc bl_not;
} PyBooleanMethods;

where the bl_and, bl_or and bl_not methods refer to the
boolean operations 'and', 'or' and 'not' of course.

This methods will be the one used to implement the
corresponding operations, and they could be overloaded
by special methods which names could be

__land__, __lor__ and __lnot__

to distinguish them from the numeric bitwise versions
 __and__ and __or__.
 

In the case that there is no logical special methods
defined in the class, the bolean methods must
behave as they do right now. However, if the
user overload them, they could return anything,
just like the "rich comparison methods"  >=, <=, < and >.


Therefore you can write code like this

class MyClass:
   def __land__(self, other):
      .....

   def __lor__(self, other):
      .....

   def _lnot__(self, other):
       ......


a = MyClass(1)
b = MyClass(2)
c = MyClass(3)

res = a >= b and  not (b < c)


and 'res' could be anything that in the "MyClass" makes sense.



Marcelo















From pinard@iro.umontreal.ca  Fri Oct  4 00:48:17 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Thu, 03 Oct 2002 19:48:17 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new
 issues
In-Reply-To: <15772.52382.121534.278816@gargle.gargle.HOWL> (barry@python.org's
 message of "Thu, 3 Oct 2002 19:02:54 -0400")
References: <20021003174338.GB50630@hishome.net>
 <BIEJKCLHCIOIHAGOKOLHEEGNDLAA.tim.one@comcast.net>
 <15772.52382.121534.278816@gargle.gargle.HOWL>
Message-ID: <oq3crn84ou.fsf@carouge.sram.qc.ca>

[Barry A. Warsaw]

> calc-mode comes with XEmacs btw, and I still don't know what does and
> doesn't come with FSF Emacs.

I read, a year ago or so, that Calc was going to be distributed with Emacs,
and maintained with it.

> calc-mode is (and has been for years) wonderful even to add a bunch of
> numbers!

Calc is wonderful, indeed.  Together with Gnus and PSGML, and surely many
other things, it makes Emacs a tool which is not easy to replace. :-)

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard



From pinard@iro.umontreal.ca  Fri Oct  4 01:19:36 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Thu, 03 Oct 2002 20:19:36 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new
 issues
In-Reply-To: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz> (Greg
 Ewing's message of "Thu, 03 Oct 2002 12:42:56 +1200 (NZST)")
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
Message-ID: <oqu1k36oo7.fsf@carouge.sram.qc.ca>

[Greg Ewing]

> Maybe there should be a separate operator for rational
> division?
>
>   1/3 --> float
>   1//3 --> int
>   1///3 --> rational
>
> (Okay, a 3-char operator is a bit verbose, but I can't
> think of anything else that looks division-like just at
> the moment.)

I much like Emacs Calc usage of 3:2 or 1:1:2 (no embedded space) for a
fraction constant, read or printed.  I find this more legible and
comfortable to read than 3/2 or `1 1/2'.  There is a Calc setting to decide
if division generates floats or fractions, but Guido is probably right in
choosing that `/' should not be magical enough to produce rationals.

One might presume that introducing `:' between numbers at the syntactic
level has just no chance to ever work.  But _maybe_ it could be more
tractable at the lexical level without bringing too much confusion, and for
numerical _constants_ only -- 3:2 is one number, has never been two.

Rationals could be got through a cast or constructor, or by introducing
rational objects into the computation, like rational constants would be.

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard



From guido@python.org  Fri Oct  4 01:19:43 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 20:19:43 -0400
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: Your message of "Thu, 03 Oct 2002 15:30:34 PDT."
 <3D9CC50A.2090208@acms.arizona.edu>
References: <002101c26b26$f2f1d720$df7e4e18@brianspiv1700>
 <3D9CC50A.2090208@acms.arizona.edu>
Message-ID: <200210040019.g940JhB08869@pcp02138704pcs.reston01.va.comcast.net>

> [...] I guess it will be also good to have the special methods for
> the logical operations 'and', 'or' and 'not', (something like
> __land__, __lor__ and __lnot__ to distinguish them from the bitwise
> versions __and__ and __or__).
> 
> So, if you overload the proper logical methods, you can write
> something like
> 
>   cond = not (sin(x) > 0.5 ) and ( b <= 0.5 )
> 
> where 'not' and 'and', like '>' and '<=', wil return anything you want.

Sorry, that won't be possible any time soon.  If I allowed these
operators to be overloaded, they would lose their shortcut properties,
which are the reason for their existence.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mmatus@dinha.acms.arizona.edu  Fri Oct  4 01:25:01 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 17:25:01 -0700
Subject: [Python-Dev] Re: PEP 285 (Adding a bool type) extending it further
Message-ID: <3D9CDFDD.2050807@acms.arizona.edu>

To solve the problem of short-circuiting 'and' and 'or',
the BooleanMethod class could be tested for non
zero method components, ie, you befine
the methods to null, like

PyBooleanMethods  {
    binaryfunc bl_and;
    binaryfunc bl_or;
    binaryfunc bl_not;
} PyBooleanMethods;


static PyBooleanMethods default_as_boolean = {

    0, /* bl_and */
    0, /* bl_or */
    0, /* bl_not */
};

and overwrite them only when the __land__, __lor__
or __lnot__ are defined.

Then, when you perfom the 'add' or 'or' operations, you
check for the proper boolean method in the first
argument only, which anyway  have to be evaluated.
If the boolean method exists, then you
force the evaluation of the second argument
and the apply the proper method. If not, you
do the usual, which will be the traditional short-circuiting
operation, ie:

    a and b

will be equivalent to

    if a.bl_and exist:
          return a.bl_and(b)
    else
        return a && b                   (traditional form with 
short-circuiting)

and similar for 'or'.

In this way, the implementation will be fully compatible with all
the previous code, allowing short-circuiting for the classes
that doesn't define the boolean methods (all of them by now),
but will allow you  to extend the operation for classes with
their own version of the logical operations.


Of course this requires that if you want to use your own
logical operations, both or at least the first
argument in the operation must always be one of your special
classes. But with proper documentations  and warnings about it,
I guess this alternative is better than nothing.



Marcelo


           
   







From jeremy@alum.mit.edu  Fri Oct  4 01:42:58 2002
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 3 Oct 2002 20:42:58 -0400
Subject: [Python-Dev] Re: PEP 285 (Adding a bool type) extending it further
In-Reply-To: <3D9CDFDD.2050807@acms.arizona.edu>
References: <3D9CDFDD.2050807@acms.arizona.edu>
Message-ID: <15772.58386.655293.791830@slothrop.zope.com>

Can't you just use & and | and leave it at that?

Jeremy



From guido@python.org  Fri Oct  4 01:52:40 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 20:52:40 -0400
Subject: [Python-Dev] PEP 285 (Adding a bool type) extending it further
In-Reply-To: Your message of "Thu, 03 Oct 2002 16:45:56 PDT."
 <3D9CD6B4.2050807@acms.arizona.edu>
References: <3D9CD6B4.2050807@acms.arizona.edu>
Message-ID: <200210040052.g940qe811749@pcp02138704pcs.reston01.va.comcast.net>

> Given my silly problems trying to write a class which overwrite
> logical operations like 'and', 'or' and 'not', which is not possible
> by now [...]  I would like to propose to extend the PEP 285 to add
> boolean operations that can be overloaded as the rest of the
> operations in python.

Sorry, PEP 285 is already accepted and implemented, and your proposal
has nothing to do with a Boolean type.  I've tried to explained before
why this can't be done.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct  4 01:55:13 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 20:55:13 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Thu, 03 Oct 2002 20:19:36 EDT."
 <oqu1k36oo7.fsf@carouge.sram.qc.ca>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
 <oqu1k36oo7.fsf@carouge.sram.qc.ca>
Message-ID: <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>

> One might presume that introducing `:' between numbers at the
> syntactic level has just no chance to ever work.  But _maybe_ it
> could be more tractable at the lexical level without bringing too
> much confusion, and for numerical _constants_ only -- 3:2 is one
> number, has never been two.

I think the slice notation e.g. x[1:4] kills that idea.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mmatus@dinha.acms.arizona.edu  Fri Oct  4 02:11:26 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 18:11:26 -0700
Subject: [Python-Dev] Re: PEP 285 (Adding a bool type) extending it further
References: <3D9CDFDD.2050807@acms.arizona.edu> <15772.58386.655293.791830@slothrop.zope.com>
Message-ID: <3D9CEABE.5020804@acms.arizona.edu>

Yes, I can use the &,|,~ operators,  and that is what I am doing right now.

But your question also apply to the boolean type:

   can't you just use an integer type and leave it like that?


So, my suggestion was originated because if  there are users
which consider relevant to have a boolean type,
there will be situations when you will want to overwrite the specific
'boolean operators' too.

Maybe the overwrite of the 'and' and 'or' is not the solution, maybe is 
better
to add the && and || operators directly, like there is the pair / and //,
so 'and' and 'or' will be always the old and untouched version, with the 
shortcut properties,
and && and || could be the one you can overwrite, but they will not warranty
the shortcutting.

I guess that could be a happy solution for all, then

   a and b

is the old and traditional version, and

    a && b

could means something like:

    if a.bl_and exist:
      return a.bl_and(b)
   else if b.bl_rand exist:
      return b.bl_rand(a)
   else
      return a and b


Anyway, if python allow you to do all this beatifull  things overwriting 
 operators
for all the other types, and don't see anything special about the 
boolean type
that forbid you to want to overwrite and extend its operators.

Also note that the 'and' and 'or' operation have an specific meaning in the
mathematical abstract sense, like '&' and '|', and the shortcut property
doesn't have anynothing to do with it. In the computational implementation
of them the shortcut property is usefull, because usually you apply 
these operators
to entities which are not boolean objects and you extend their meaning.

But like the operators '&' and '|' don't require the shorcut porperty,
boolean types and boolean algebra don't require it either. If it is 
there, good, if not,
good also.

So, after writting this email, I would like to change the proposed 
extension to the
inclusion of the "pure boolean" operator &&, || and !, which can be 
overwritten
but in any way they warranty the shortcut property, like the bitwise 
cousins !,&,~,
and leave the 'and', 'or' and 'not' operations untouched with their 
"extended
computational meanings".


Marcelo


Jeremy Hylton wrote:

>Can't you just use & and | and leave it at that?
>
>Jeremy
>
>  
>





From tim.one@comcast.net  Fri Oct  4 02:12:01 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 21:12:01 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELHBIAB.tim.one@comcast.net>

[Guido]
> I think the slice notation e.g. x[1:4] kills that idea.

Well, at least {'a':'b', 1:2:3:4:5:6} is unambiguous <wink>.

not-to-mention-attractive-ly y'rs  - tim


From greg@cosc.canterbury.ac.nz  Fri Oct  4 02:22:38 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Oct 2002 13:22:38 +1200 (NZST)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210040122.g941McN29809@oma.cosc.canterbury.ac.nz>

Guido:

> I think the slice notation e.g. x[1:4] kills that idea.

Maybe if ":" were a sliceobject-creating operator usable
anwyere, not just in [...], and you added arithmetic
methods to sliceobjects so you could use them as
rationals...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tdelaney@avaya.com  Fri Oct  4 02:50:55 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Fri, 4 Oct 2002 11:50:55 +1000
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implemen
 tation and new issues
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A618@natasha.auslabs.avaya.com>

> From: Greg Ewing [mailto:greg@cosc.canterbury.ac.nz]
> 
> > I think the slice notation e.g. x[1:4] kills that idea.
> 
> Maybe if ":" were a sliceobject-creating operator usable
> anwyere, not just in [...], and you added arithmetic
> methods to sliceobjects so you could use them as
> rationals...

The sad thing is this is actually tempting to me, given the beauty of using
the "ratio" notation ``x:y`` to denote rational literals ...

Actually, following through my thought processes, ``x:y`` is only legal in
the context of a sequence slice i.e. inside ``[]``. Therefore ``x:y``
*could* be used outside of a sequence context for rationals. There would be
no ambiguity if rationals could not be used in such a context, just as
floats can't be (which would be consistent).

e.g.

``a[1:2]`` is *always* a slice.

``b:2`` is always a rational.

The problem of course is that the ``:`` operator then becomes overloaded
depending on its context. Not a good thing.

Damn. Just realised of course that a rational could be a dictionary index.
Perhaps in that case slice takes precedence, meaning that to use a rational
expression as an index it would need to be disambiguated as:

``d[(1:2)]``

This boils down to 2 rules:

1. In an indexing context, ``:`` denotes a slice expression;

2. In a non-indexing context, ``:`` denotes a rational expression.

with the corollary:

1. In an indexing context, a rational expression must be disambiguated using
parentheses.

Tim Delaney


From guido@python.org  Fri Oct  4 02:54:51 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 03 Oct 2002 21:54:51 -0400
Subject: [Python-Dev] perplexed by mro
In-Reply-To: Your message of "Fri, 04 Oct 2002 11:14:30 +1200."
 <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz>
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz>
Message-ID: <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net>

> > Now let's assume m is *also* overridden in a, so that y.m overrides
> > a.m overrides object.m.  Now if we searched b before y (naive algo),
> > it is arguably strange if the 'super' call chain went from y.m via b.m
> > to a.m.
> 
> I don't understand that. According to the diagram I just
> scribbled, the mro from the documented algorithm comes out as
> 
>   z, x, b, y, a, c, object
> 
> so the super chain would be b.m, y.m, a.m, object.m.

Sorry, I misread my own scribbled diagram.  That's indeed the correct
order using the naive algorithm.  And I now agree that this is a fine
call chain.

> (Which is *also* wrong according to my intuition, by the way --
> I would expect a super call inside b.m to go to object.m,
> and not depend on z's inheritance structure -- but that's a
> separate issue.)

No, the whole point here is that the most inherited class's MRO
(i.e. z) can insert things in a base class's MRO.  Have a look at the
explanation of the diamond diagram in
http://python.org/2.2.1/descrintro.html

> Anyway, why not just use the algorithm you said you were
> using in the docs -- i.e. traverse the inheritance hierarchy
> and build the mro afresh each time, rather than trying to
> merge the mro's of the superclasses.

I would still construct a linearized MRO out of it -- many places
benefit from having it as a list rather than having to recurse down
the inheritance lattice.  But I agree that the naive algorithm seems
to have some nice properties -- at the very least, it is easy to
explain. :-)

If Samuele agrees that the naive algorithm works better, I'll try to
make it so in 2.3.

> If it's an issue of which algorithm is the one that should
> be used, I vote for the documented one, because it's
> simple and easy to explain! If it leads to odd things
> happening in some cases, at least it's clear what odd
> things will happen and why -- as opposed to other odd
> things happening for mysterious reasons.

Yes.  Here's how this happened: I had read and implemented the book's
algorithm without really visualizing what it did in more complex
cases.  In the few cases that I cared about it worked fine.  When I
had to document it, I realized that the description in the book is
very complicated, and I tried to come up with a simpler description.
That's how I stumbled upon the naive algorithm -- at first I believed
that it was really the same thing, but later I found a counter-example
with an order disagreements, so I added that.

Now Samuele has found another counter-example that does not involve an
order disagreement.  It does have a somewhat strange smell to it, 
but I can't pinpoint the quality of that smell, and it certainly
doesn't have an order disagreement.

I've updated the descrintro.html page to acknowledge the problems.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mmatus@dinha.acms.arizona.edu  Fri Oct  4 03:29:25 2002
From: mmatus@dinha.acms.arizona.edu (Marcelo Matus)
Date: Thu, 03 Oct 2002 19:29:25 -0700
Subject: [Python-Dev] PEP 285 (Adding a bool type) extending it further
References: <3D9CD6B4.2050807@acms.arizona.edu> <200210040052.g940qe811749@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3D9CFD05.6050009@acms.arizona.edu>

Guido van Rossum wrote:

>>Given my silly problems trying to write a class which overwrite
>>logical operations like 'and', 'or' and 'not', which is not possible
>>by now [...]  I would like to propose to extend the PEP 285 to add
>>boolean operations that can be overloaded as the rest of the
>>operations in python.
>>    
>>
>
>Sorry, PEP 285 is already accepted and implemented, and your proposal
>has nothing to do with a Boolean type.  I've tried to explained before
>why this can't be done.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)
>  
>
I am sorry if this is not the right forum for this topic, in that case 
please tell me.

Also, I am sorry because it seems there were some crossover of mails, 
and I've got
your answer after sending the request to extend the PEP 285. And when
I ask to extend the PEP 285 I am not that literal, the proposed changes
could be included later, maybe in another PEP.

I said this proposal is an extension of the PEP 285 because it is 
related to the
boolean support in python.

And as you could see in one of the other emails, I understood the problem
and the sugestions were well taken, so now what I am proposing is not to 
change or
modify the 'and' or 'or' operators, because of the expected shortcut 
property.

Instead what I am proposing now is the addition of the pure boolean 
operators
&&, || and !, that like their bitwise versions &,| and ~, they don't 
require nor warranty
any shortcut property, but they could be overloaded, and in the default 
case,
they call the traditional 'and', 'or' and 'not' versions.

Why I insist on this instead of only use the bitwise versions &,|, ~ and 
just "leave it at that"?,
let me show you why :


By now my classes are overloading the the &,|, ~ versions as replacement
of the logical versions, but this have to be done with extreme carefull. 
Just
a simple example show the problem:

Assume I have a  set of classes which overload the bitwise operation
as a replacement of the logical ones, and then I write the following 
function

def cond(a, b, c):
    return  (~a | b) & ~c      # for meaning (not a or b) and  not c


well, this function will work with my classes, but if I want to mix my 
classes
with the real boolean and integer types, then what will be the result?.
If you, for example, set a=1; b=1 and c=1 then

print ~a | b & ~c                      ==> -2      (which means true)

print (not a or b) and  not c    ==> 0       (which means false)


therefore, replacing the 'and', 'or' and 'not' by '&,|,^'  is clearly 
dangerous
and at the end wrong because you can warranty the proper behaviour.

Now, maybe this proposal "has nothing to do with a Boolean type" as you
said, but once python starts to support the Boolean type, one could 
expect that this
means more than just an uniform way to name and/or  manage the True and 
False
values. Real booelan operators and, as for almost all the other types in 
python, the
real capability to overload them, is also important.

Finally, as you can  see, the shortcut property in the 'and', 'or' and 
 'not' operators
is really important in the "computational  sense", but to implement a real
boolean algebra, like in the bitwise operations "&,|,^", that is secondary.

Therefore, the addition of the pure boolean operators "&&,||,!" seems to
be  Salomonic solution. They do not interfere with the "computationaly 
oriented"
'and', 'or' and 'not', but they allow you to implement new (and correct)
boolean relations between you own types.


Marcelo








From greg@cosc.canterbury.ac.nz  Fri Oct  4 03:29:27 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Oct 2002 14:29:27 +1200 (NZST)
Subject: [Python-Dev] perplexed by mro
In-Reply-To: <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>

Guido:

> No, the whole point here is that the most inherited class's MRO
> (i.e. z) can insert things in a base class's MRO.

It worries me that when I write a super call, I'll
never be sure which method is going to be called,
because someone might inherit me and mess with
my mro.

But I can see that there are cases where this
behaviour is desirable. I guess it's okay as long
as I have the option of explicitly specifying
the base class. (You're not going to take that
away, are you?)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Fri Oct  4 03:34:00 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 03 Oct 2002 22:34:00 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <B43D149A9AB2D411971300B0D03D7E8BF0A618@natasha.auslabs.avaya.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOELMBIAB.tim.one@comcast.net>

[Delaney, Timothy]
> ...
> Actually, following through my thought processes, ``x:y`` is only
> legal in the context of a sequence slice i.e. inside ``[]``.

Dicts are worse.  {1:2:3} would be hopelessly ambiguous.


From esr@thyrsus.com  Fri Oct  4 03:35:08 2002
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 3 Oct 2002 22:35:08 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implemen tation and new issues
In-Reply-To: <B43D149A9AB2D411971300B0D03D7E8BF0A618@natasha.auslabs.avaya.com>
References: <B43D149A9AB2D411971300B0D03D7E8BF0A618@natasha.auslabs.avaya.com>
Message-ID: <20021004023508.GA8292@thyrsus.com>

Delaney, Timothy <tdelaney@avaya.com>:
> 1. In an indexing context, ``:`` denotes a slice expression;
> 
> 2. In a non-indexing context, ``:`` denotes a rational expression.
> 
> with the corollary:
> 
> 1. In an indexing context, a rational expression must be disambiguated using
> parentheses.

-1.  That way lies madness^H^H^H^H^H^H^H^Perl.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>


From pedronis@bluewin.ch  Fri Oct  4 03:39:30 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 04:39:30 +0200
Subject: [Python-Dev] perplexed by mro
References: <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>
Message-ID: <041201c26b4f$444bc540$6d94fea9@newmexico>

From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
> Guido:
>
> > No, the whole point here is that the most inherited class's MRO
> > (i.e. z) can insert things in a base class's MRO.
>
> It worries me that when I write a super call, I'll
> never be sure which method is going to be called,
> because someone might inherit me and mess with
> my mro.
>

it can add things yes, but here is were monotonicity property comes into play,
if a mro algo is monotonic you know that however your class is subclassed the
classes in its mro will be still encountered in the same order in the mros of
the subclasses.

Sadly the naive algorithm is not monotonic.

>>> cpl.ex6(globals())
class a_boat(object): pass
class b_day_boat(a_boat): pass
class c_wheel_boat(a_boat): pass
class d_engine_less(b_day_boat): pass
class e_small_multihull(b_day_boat): pass
class f_pedal_wheel_boat(d_engine_less,c_wheel_boat): pass
class g_small_catamaran(e_small_multihull): pass
class h_pedalo(f_pedal_wheel_boat,g_small_catamaran): pass
>>> cpl.names(cpl.keeplast(cpl.lrdfs(f_pedal_wheel_boat)))
['f_pedal_wheel_boat', 'd_engine_less', 'b_day_boat', 'c_wheel_boat', 'a_boat',
'object']

the above computes the mro of f_pedal_wheel_boat according to naive algo,
b_day_boat precedes c_wheel_boat,

but for the subclass h_pedalo

>>> cpl.names(cpl.keeplast(cpl.lrdfs(h_pedalo)))
['h_pedalo', 'f_pedal_wheel_boat', 'd_engine_less', 'c_wheel_boat',
'g_small_catamaran', 'e_small_multihull', 'b_day_boat', 'a_boat', 'object']

b_day_boat follows c_wheel_boat

regards.





From tdelaney@avaya.com  Fri Oct  4 03:44:24 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Fri, 4 Oct 2002 12:44:24 +1000
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implemen
 tation and new issues
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A619@natasha.auslabs.avaya.com>

> From: Tim Peters [mailto:tim.one@comcast.net]
> 
> [Delaney, Timothy]
> > ...
> > Actually, following through my thought processes, ``x:y`` is only
> > legal in the context of a sequence slice i.e. inside ``[]``.
> 
> Dicts are worse.  {1:2:3} would be hopelessly ambiguous.

{(1:2):3}
{1:(2:3)}

I'm trying to work out a way to get more parentheses in there without
surrounding the actual digits ;)

But it's another overloading that I missed ...

As I said, it's really tempting. The concept is very elegant. Unfortunately,
I don't think it's practical :( Too many cases where it would be necessary
to disambiguate - even having one exception is probably too many.

Do we have double-colons yet?

a::b (damn - I've forgotten what that means mathematically ... equivalent?)
Looks too much like C++ anyway ;)

Tim Delaney


From tdelaney@avaya.com  Fri Oct  4 03:46:26 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Fri, 4 Oct 2002 12:46:26 +1000
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implemen
 tation and new issues
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A61A@natasha.auslabs.avaya.com>

> From: Delaney, Timothy [mailto:tdelaney@avaya.com]
> 
> Do we have double-colons yet?
> 
> a::b

Of course we do ... (slices).

Tim Delaney


From pedronis@bluewin.ch  Fri Oct  4 03:53:23 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 04:53:23 +0200
Subject: [Python-Dev] perplexed by mro
References: <01e101c26984$fc43da60$6d94fea9@newmexico>  <200210032203.g93M3jr31166@odiug.zope.com>
Message-ID: <049201c26b51$34abea00$6d94fea9@newmexico>

more on 2.2 algo:

> > >>> cpl.ex5(globals())
> > class a(object): pass
> > class b(object): pass
> > class c(object): pass
> > class x(a): pass
> > class y(a): pass
> > class z(x,b,y,c): pass
> > >>> cpl.names(z.__mro__) # just the class names, real 2.2 mro algo
> > ['z', 'x', 'y', 'a', 'b', 'c', 'object']

now b is there, but a simple substitution will change things radically:

>>> cpl.ex5perturb(globals())
class a(object): pass
class b(object): pass
class c(object): pass
class x(a): pass
class y(a): pass
class y_b(b,y): pass
class z(x,y_b,c): pass
>>> cpl.names(z.__mro__)
['z', 'x', 'y_b', 'b', 'y', 'a', 'c', 'object']

instead of inheriting directly from b and y, z inherits from y_b that inherits
from both. Now b is between x and a.
> > >>> cpl.ex9(globals())
> > class a(object): pass
> > class b(object): pass
> > class c(object): pass
> > class d(object): pass
> > class e(object): pass
> > class k1(a,b,c): pass
> > class k2(d,b,e): pass
> > class k3(d,a): pass
> > class z(k1,k2,k3): pass
> > >>> cpl.names(z.__mro__)
> > ['z', 'k1', 'k3', 'a', 'k2', 'd', 'b', 'c', 'e', 'object']
>
> I think this one *does* have an order conflict, and in that case all
> bets are off about the book's algo.  If you look at k1 and k2, clearly
> d must come after a; yet k3 wants it after a.
>

yes but for class z(k3,k1,k2) the conflict would go away. It's not intuitive
nor stable.

regards.



From skip@pobox.com  Fri Oct  4 04:00:37 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 3 Oct 2002 22:00:37 -0500
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210040122.g941McN29809@oma.cosc.canterbury.ac.nz>
References: <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
 <200210040122.g941McN29809@oma.cosc.canterbury.ac.nz>
Message-ID: <15773.1109.273430.632460@localhost.localdomain>

    Greg> Guido:
    >> I think the slice notation e.g. x[1:4] kills that idea.

    Greg> Maybe if ":" were a sliceobject-creating operator usable anwyere,
    Greg> not just in [...], and you added arithmetic methods to
    Greg> sliceobjects so you could use them as rationals...

Does x[1:4] return a slice out of x or x[rational(1,4)]?

Skip



From martin@v.loewis.de  Fri Oct  4 06:24:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Oct 2002 07:24:40 +0200
Subject: [Python-Dev] perplexed by mro
In-Reply-To: <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>
References: <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>
Message-ID: <m3smzm3hev.fsf@mira.informatik.hu-berlin.de>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> It worries me that when I write a super call, I'll
> never be sure which method is going to be called,
> because someone might inherit me and mess with
> my mro.

That's why it is called a "cooperative super call". Derived classes
can *always* mess it up (be it methods or state), but this approach
provides a pattern for the use case "invoke all of the base methods,
and each of them once".

There are cases where this is useful, and cases where this is not
useful. In cases where it is useful, you have to trust derived classes
to follow the pattern. Since the pattern is easy to follow, this
should not be a problem.

Regards,
Martin



From martin@v.loewis.de  Fri Oct  4 06:56:28 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Oct 2002 07:56:28 +0200
Subject: [Python-Dev] PEP 285 (Adding a bool type) extending it further
In-Reply-To: <3D9CFD05.6050009@acms.arizona.edu>
References: <3D9CD6B4.2050807@acms.arizona.edu>
 <200210040052.g940qe811749@pcp02138704pcs.reston01.va.comcast.net>
 <3D9CFD05.6050009@acms.arizona.edu>
Message-ID: <m3ofaa3fxv.fsf@mira.informatik.hu-berlin.de>

Marcelo Matus <mmatus@dinha.acms.arizona.edu> writes:

> I am sorry if this is not the right forum for this topic, in that case
> please tell me.

The forum is right, but you really have to read carefully all
responses you get to earlier messages. You are also supposed to do
loads of research before speaking up.

> Also, I am sorry because it seems there were some crossover of
> mails, and I've got your answer after sending the request to extend
> the PEP 285. And when I ask to extend the PEP 285 I am not that
> literal, the proposed changes could be included later, maybe in
> another PEP.

That is unfortunate, indeed. However, this specific issue has been
discussed many times before, so you could have known.

> I said this proposal is an extension of the PEP 285 because it is
> related to the boolean support in python.

We understand why you said this. Please also understand why Guido said
that PEP 285 is closed and done.

> By now my classes are overloading the the &,|, ~ versions as
> replacement of the logical versions, but this have to be done with
> extreme carefull. Just a simple example show the problem:

If you really insist on having such a feature, you'll have to give a
better rationale.

It is unclear to me why you have to override those operators in the
first place. I would assume that every problem that involves
boolean-ness of some object can be solved by implementing __nonzero__.

> Now, maybe this proposal "has nothing to do with a Boolean type" as you
> said, but once python starts to support the Boolean type, one could
> expect that this
> means more than just an uniform way to name and/or  manage the True
> and False
> values. Real booelan operators and, as for almost all the other types
> in python, the
> real capability to overload them, is also important.

Can you please break you lines equally?

> Finally, as you can see, the shortcut property in the 'and', 'or'
> and 'not' operators is really important in the "computational
> sense", but to implement a real boolean algebra, like in the bitwise
> operations "&,|,^", that is secondary.

It so happens that the semantics is considered part of the Python
language, and people rely on it. Applications would break if you took
it away. In software development, the computational meaning of a
program is important, not the algebraic one - even in functional
languages, people could never settle on the issue of strictness.


> Therefore, the addition of the pure boolean operators "&&,||,!"
> seems to be Salomonic solution.

I find that solution very ugly. It would add more line noise to the
language, and confuse newcomers.

Regards,
Martin


From martin@v.loewis.de  Fri Oct  4 06:59:24 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Oct 2002 07:59:24 +0200
Subject: [Python-Dev] Re: PEP 285 (Adding a bool type) extending it further
In-Reply-To: <3D9CEABE.5020804@acms.arizona.edu>
References: <3D9CDFDD.2050807@acms.arizona.edu>
 <15772.58386.655293.791830@slothrop.zope.com>
 <3D9CEABE.5020804@acms.arizona.edu>
Message-ID: <m3k7ky3fsz.fsf@mira.informatik.hu-berlin.de>

Marcelo Matus <mmatus@dinha.acms.arizona.edu> writes:

> Yes, I can use the &,|,~ operators,  and that is what I am doing right now.
> 
> But your question also apply to the boolean type:
> 
>    can't you just use an integer type and leave it like that?

I'm uncertain what you are trying to achieve. If you want to convince
people, hoping that they will just see the light rarely
works. Instead, you have to proceed in a diplomatic manner. If you
feel resistance, analyse where this resistance comes from, and try to
soften it by means of negotiation.

Giving snippy responses probably won't help your cause.

Regards,
Martin


From mmclay@comcast.net  Fri Oct  4 08:46:47 2002
From: mmclay@comcast.net (Michael McLay)
Date: Fri, 04 Oct 2002 03:46:47 -0400
Subject: [Python-Dev] Offtopic: Microsoft Altair BASIC legend likes Python
Message-ID: <200210040346.47705.mmclay@comcast.net>

Some good press for Python:
	http://www.theregister.co.uk/content/archive/18909.html

Twenty six years ago the microprocessor revolution found a software catalyst - 
a tiny BASIC interpreter that ran in 4K of memory. You've probably heard of 
two of its three authors - Paul Allen and Bill Gates, who'd incorporated the 
company 'Micro-Soft' in Albuquerque the same year. The third man, Monte 
Davidoff, isn't nearly as famous. You'll search in vain for an interview on 
the web with Monte. So we figured we'd set that right. 
[...]
His other passion, he tells us, is Python.

"Hats off to them. It's an extremely well designed language. It's object 
orientated from the get-go. They've really succeeded there," he says, and 
commends it as the ideal teaching language. That used to be BASIC, of course. 
[...]
Monte was wary about drawing generalizations about the [free software] 
process, but pointed out that where one or a small number of people was in 
charge - citing Linus and the kernel, or Guido von Rossum and Python, or 
Apache - the end result was demonstrably better.




From mal@lemburg.com  Fri Oct  4 08:57:06 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 04 Oct 2002 09:57:06 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>              <oqu1k36oo7.fsf@carouge.sram.qc.ca> <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3D9D49D2.7040604@lemburg.com>

[discussion on the syntax to create rationals]

Are you sure that you really want a special syntax for this
rather than just a simply constructor like rat(2,3) ?

Just think of how difficult it would be to explain how the
following would work (assuming that you use @ as the magical
operator):

	1 @ 2.3
	1.5 @ 1.3
	1j+3.4 @ 2

Note that intution isn't going to help here because you are
missing a precision indicator. mxNumber has a constructor called
FareyRational() which converts floats to rationals:

    FareyRational(value, maxden)
	 Returns a Rational-object reflecting the given value
	 and using maxden as maximum denominator

Here's the algorithm:

/* Farey Function

    This is a GNU MP implementation of the following function which
    Scott David Daniels posted to the Python Cookbook;
    http://www.activestate.com/ASPN/Python/Cookbook/Recipe/52317 :

     def farey(v, lim):
	'''Named after James Farey, an English surveyor.
	No error checking on args -- lim = max denominator,
	results are (numerator, denominator), (1,0) is infinity
	'''
	if v < 0:
	  n,d = farey(-v, lim)
	  return -n,d
	z = lim-lim   # get 0 of right type for denominator
	lower, upper = (z,z+1), (z+1,z)
	while 1:
	  mediant = (lower[0] + upper[0]), (lower[1]+upper[1])
	  if v * mediant[1] > mediant[0]:
	      if lim < mediant[1]: return upper
	      lower = mediant
	  elif v * mediant[1] == mediant[0]:
	      if lim >= mediant[1]: return mediant
	      if lower[1] < upper[1]: return lower
	      return upper
	  else:
	      if lim < mediant[1]: return lower
	      upper = mediant

    A nice proof of the algorithm can be found at "Cut the Knot":
    http://www.cut-the-knot.com/blue/Farey.html

*/

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From oren-py-d@hishome.net  Fri Oct  4 11:43:54 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Fri, 4 Oct 2002 06:43:54 -0400
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: <20021003232215.GA7735@panix.com>
References: <3D9CB654.8050306@acms.arizona.edu> <002101c26b26$f2f1d720$df7e4e18@brianspiv1700> <20021003232215.GA7735@panix.com>
Message-ID: <20021004104354.GA2137@hishome.net>

On Thu, Oct 03, 2002 at 07:22:15PM -0400, Aahz wrote:
> On Thu, Oct 03, 2002, Brian Quinlan wrote:
> >
> > > shows that python doesn't call the __not__ special method
> > > in a 'not' operator statement.
> > 
> > Python calls the special __nonzero__ method so check the truth value of
> > an object.
> 
> Mark McEahern provided the full answer: Python uses __len__, too.

This reminds me of an asymmetry between the bool type and other built-in
types: the lack of a __bool__ method.

Perhaps a __bool__ method should be added to objects, just like __int__,
__str__, __long__, __float__, __complex__ and __unicode__.

If __bool__ is not defined bool() would fall back to using __nonzero__ 
and __len__, just like str() falls back to using __repr__ when __str__ is 
not defined.

Would this be done in CPython with a new tp_bool slot or special-cased
like __unicode__ and __complex__?  Binary compatibility aside, I think it 
should be a slot.  Boolean evaluation is a pretty basic operation.

	Oren


From mwh@python.net  Fri Oct  4 11:52:24 2002
From: mwh@python.net (Michael Hudson)
Date: 04 Oct 2002 11:52:24 +0100
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: Oren Tirosh's message of "Fri, 4 Oct 2002 06:43:54 -0400"
References: <3D9CB654.8050306@acms.arizona.edu> <002101c26b26$f2f1d720$df7e4e18@brianspiv1700> <20021003232215.GA7735@panix.com> <20021004104354.GA2137@hishome.net>
Message-ID: <2md6qqbhnb.fsf@starship.python.net>

Oren Tirosh <oren-py-d@hishome.net> writes:

> On Thu, Oct 03, 2002 at 07:22:15PM -0400, Aahz wrote:
> > On Thu, Oct 03, 2002, Brian Quinlan wrote:
> > >
> > > > shows that python doesn't call the __not__ special method
> > > > in a 'not' operator statement.
> > > 
> > > Python calls the special __nonzero__ method so check the truth value of
> > > an object.
> > 
> > Mark McEahern provided the full answer: Python uses __len__, too.
> 
> This reminds me of an asymmetry between the bool type and other built-in
> types: the lack of a __bool__ method.
> 
> Perhaps a __bool__ method should be added to objects, just like __int__,
> __str__, __long__, __float__, __complex__ and __unicode__.
> 
> If __bool__ is not defined bool() would fall back to using __nonzero__ 
> and __len__, just like str() falls back to using __repr__ when __str__ is 
> not defined.
> 
> Would this be done in CPython with a new tp_bool slot or special-cased
> like __unicode__ and __complex__?  Binary compatibility aside, I think it 
> should be a slot.  Boolean evaluation is a pretty basic operation.

I'd say just pretend "nonzero" is spelled "bool". (So "__nonzero__" ==
"__bool__", "tp_as_number->nb_nonzero" == "tp_bool").

It would be nice to have done this differently from the start, but it
seems to me to be too much hassle to change now.

Cheers,
M.

-- 
  QNX... the OS that walks like a duck, quacks like a duck, but is,
  in fact, a platypus. ... the adventures of porting duck software 
  to the platypus were avoidable this time.
                                 -- Chris Klein, alt.sysadmin.recovery


From pinard@iro.umontreal.ca  Fri Oct  4 13:06:30 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Fri, 04 Oct 2002 08:06:30 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new
 issues
In-Reply-To: <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Thu, 03 Oct 2002 20:55:13 -0400")
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
 <oqu1k36oo7.fsf@carouge.sram.qc.ca>
 <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <oqwuoy4ddl.fsf@titan.progiciels-bpi.ca>

[Guido van Rossum]

>> But _maybe_ it could be more tractable at the lexical level without
>> bringing too much confusion, and for numerical _constants_ only -- 3:2 is
>> one number, has never been two.
>
> I think the slice notation e.g. x[1:4] kills that idea.

And it kills it dead indeed, at least, in my opinion.

[M.-A. Lemburg]

> Are you sure that you really want a special syntax for this
> rather than just a simply constructor like rat(2,3) ?

This is what I currently do whenever I need rationals.  It might not be as
elegant as the `:' would have been, but it works well in practice.  I guess
I would prefer ^rat(2, 3)" over any non-elegant or non-natural notation for
rational constants.  We do not necessarily ought to have a special notation
hardwired in Python syntax.

If Guido was adding complex numbers today instead of long ago, I wonder if
he would allow a special notation for them, or just suggest a constructor.

> Note that intution isn't going to help here because you are missing a
> precision indicator.  mxNumber has a constructor called FareyRational()
> which converts floats to rationals: [...]

Interesting, I'll save it.  I use continued fraction expansion to get the
"best" rational fitting a float within a tolerance, and wonder how Farey
will be similar/different.  Tim will surely tell us, out of his head! :-)

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From mal@lemburg.com  Fri Oct  4 13:09:22 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 04 Oct 2002 14:09:22 +0200
Subject: [Python-Dev] How to add an encoding alias?
References: <200209261913.g8QJDFe01575@odiug.zope.com> <3D936369.3000908@lemburg.com> <200209262003.g8QK36P01952@odiug.zope.com>              <3D936AA4.1080302@lemburg.com> <200209262027.g8QKRlO02176@odiug.zope.com>
Message-ID: <3D9D84F2.4090501@lemburg.com>

Guido van Rossum wrote:
>>>Would it make sense to change the lookup function to convert *all*
>>>punctuation to underscores before doing the lookup?  (Then this one
>>>would actually have worked...)
>>
>>Codecs must currently use names as defined by the search function in the
>>encodings package:
>>
>>     Codec modules must have names corresponding to standard lower-case
>>     encoding names with hyphens mapped to underscores, e.g. 'utf-8' is
>>     implemented by the module 'utf_8.py'.
>>
>>We could extend this to:
>>
>>     Codec modules must have names corresponding to standard lower-case
>>     encoding names with all non-alphanumeric charactersmapped to
>>     underscores, e.g. 'utf-8' is implemented by the module 'utf_8.py'
>>     and 'ISO 639:1988' would be implemented as module 'iso_639_1988'.
>>
>>Note that the aliasing dictionary is consulted *after*
>>having applied this mapping.
> 
> 
> +1; +1 on backport to 2.2.2 also.
> 
> Note that this requires some changes to the dict in aliases.py.

Done.

Not backported to 2.2.2, though, since this is a new feature.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Fri Oct  4 13:11:28 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 08:11:28 -0400
Subject: [Python-Dev] perplexed by mro
In-Reply-To: Your message of "Fri, 04 Oct 2002 14:29:27 +1200."
 <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>
References: <200210040229.g942TRQ00012@oma.cosc.canterbury.ac.nz>
Message-ID: <200210041211.g94CBSX20029@pcp02138704pcs.reston01.va.comcast.net>

> > No, the whole point here is that the most inherited class's MRO
> > (i.e. z) can insert things in a base class's MRO.
> 
> It worries me that when I write a super call, I'll
> never be sure which method is going to be called,
> because someone might inherit me and mess with
> my mro.

But this is the whole *point* of the super() built-in.  Read up on
cooperative methods.

> But I can see that there are cases where this
> behaviour is desirable. I guess it's okay as long
> as I have the option of explicitly specifying
> the base class. (You're not going to take that
> away, are you?)

Not any time soon -- though I expect your fears are irrational. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct  4 13:39:21 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 08:39:21 -0400
Subject: [Python-Dev] the not operator (and the __not__ special method)
In-Reply-To: Your message of "Fri, 04 Oct 2002 06:43:54 EDT."
 <20021004104354.GA2137@hishome.net>
References: <3D9CB654.8050306@acms.arizona.edu> <002101c26b26$f2f1d720$df7e4e18@brianspiv1700> <20021003232215.GA7735@panix.com>
 <20021004104354.GA2137@hishome.net>
Message-ID: <200210041239.g94CdMJ20907@pcp02138704pcs.reston01.va.comcast.net>

> This reminds me of an asymmetry between the bool type and other
> built-in types: the lack of a __bool__ method.
> 
> Perhaps a __bool__ method should be added to objects, just like
> __int__, __str__, __long__, __float__, __complex__ and __unicode__.
> 
> If __bool__ is not defined bool() would fall back to using
> __nonzero__ and __len__, just like str() falls back to using
> __repr__ when __str__ is not defined.

Apart from being a slightly odd name, __nonzero__ does this; I see no
reason to rename it to __bool__, since that would add nothing new --
it would only be cosmetics.

> Would this be done in CPython with a new tp_bool slot or
> special-cased like __unicode__ and __complex__?  Binary
> compatibility aside, I think it should be a slot.  Boolean
> evaluation is a pretty basic operation.

I think it's actually a bad idea to allow types to define their
booleanness independent from their numeric value (for numbers) or
length (for containers).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct  4 13:54:20 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 08:54:20 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Fri, 04 Oct 2002 08:06:30 EDT."
 <oqwuoy4ddl.fsf@titan.progiciels-bpi.ca>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz> <oqu1k36oo7.fsf@carouge.sram.qc.ca> <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
 <oqwuoy4ddl.fsf@titan.progiciels-bpi.ca>
Message-ID: <200210041254.g94CsKG21170@pcp02138704pcs.reston01.va.comcast.net>

> If Guido was adding complex numbers today instead of long ago, I
> wonder if he would allow a special notation for them, or just
> suggest a constructor.

At the time I believe there was heavy pressure from the Numeric crowd
to allow a special notation.  I'm not so sure if I should have given
in though.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Fri Oct  4 14:49:58 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 15:49:58 +0200
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz>  <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <037201c26bac$ede8d040$6d94fea9@newmexico>

[GvR]
> I would still construct a linearized MRO out of it -- many places
> benefit from having it as a list rather than having to recurse down
> the inheritance lattice.  But I agree that the naive algorithm seems
> to have some nice properties -- at the very least, it is easy to
> explain. :-)
>
> If Samuele agrees that the naive algorithm works better, I'll try to
> make it so in 2.3.
>

after cooking up the substitution example, I'm quite conviced that 2.2 mro is
the worst of our options: it produces easely counterintuitive results, and if
the conflict notion it needs is the one I think, it is the most  prone to
reject hierarchies or in case we ignore conflict to produce very odd results.

So IMO our options are:
*) use the naive algo:  it is easy to explain (the algorithm) but we should
live with the fact that it is not monotonic and I don't think it produces the
most natural results (because it considers what is encoutered at the end of a
dfs traversal, instead of what is at the beginning)

*) adopt C3:
(1) C3 is monotonic (it takes and merges respecting them the mros of the
superclasses, but in parallel, non sequentially like the current 2.2 mro algo)
(2) it respects also the inheritance lists of all superclasses; the paper I
cited claims that is something intuitively expected, and I tend to agree.
(3) it has another property, that basically enforces that, when this does not
conflicts with (1) and (2), the dfs order is respected.

How it works: one takes the mros of the direct superclasses and the inheritance
list of the considered class cl, and merges them as ordered sequences, in case
of a tie the mros of the classes appearing earlier in the inheritance list are
favored (this is what induces (3))

The intuitive difference with the current 2.2 mro is that C3 uses (3) only in
case of a tie, while the current 2.2 mro algo tries to enforce a a mix of (1)
and (3) at the same time.

>From the examples this does not seem not to have any evident benefit (unless we
consider the placement of b in ex5 such), and makes it the most complicated of
all algorithms from the point of view of understanding the outcomes.

regards.










From neal@metaslash.com  Fri Oct  4 14:59:48 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 4 Oct 2002 09:59:48 -0400
Subject: [Python-Dev] tests failing in 2.2.2
Message-ID: <20021004135948.GA30482@epoch.metaslash.com>

Is anyone else having problems with 2.2.2 tests?

minidom & sax fail for me on Linux:

test_sax
test test_sax skipped -- cannot import name quoteattr
test_minidom
Traceback (most recent call last):
  File "Lib/test/test_minidom.py", line 636, in ?
    func()
  File "Lib/test/test_minidom.py", line 208, in testAAA
    "setAttribute() sets ownerDocument")
  File "Lib/test/test_minidom.py", line 22, in confirm
    raise Exception
Exception

I'm off for a long weekend.  I'll be back late Monday.

Neal


From guido@python.org  Fri Oct  4 16:00:33 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 11:00:33 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: Your message of "Fri, 04 Oct 2002 09:59:48 EDT."
 <20021004135948.GA30482@epoch.metaslash.com>
References: <20021004135948.GA30482@epoch.metaslash.com>
Message-ID: <200210041500.g94F0XV25271@pcp02138704pcs.reston01.va.comcast.net>

[Neal Norwitz]
> Is anyone else having problems with 2.2.2 tests?
> 
> minidom & sax fail for me on Linux:
> 
> test_sax
> test test_sax skipped -- cannot import name quoteattr
> test_minidom
> Traceback (most recent call last):
>   File "Lib/test/test_minidom.py", line 636, in ?
>     func()
>   File "Lib/test/test_minidom.py", line 208, in testAAA
>     "setAttribute() sets ownerDocument")
>   File "Lib/test/test_minidom.py", line 22, in confirm
>     raise Exception
> Exception
> 
> I'm off for a long weekend.  I'll be back late Monday.

Both tests work fine for me.  I expect there's a problem with your
expat installation.  Maybe Fred has a suggestion?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct  4 16:15:53 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 11:15:53 -0400
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
In-Reply-To: Your message of "Fri, 04 Oct 2002 15:49:58 +0200."
 <037201c26bac$ede8d040$6d94fea9@newmexico>
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net>
 <037201c26bac$ede8d040$6d94fea9@newmexico>
Message-ID: <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>

[Guido]
> > If Samuele agrees that the naive algorithm works better, I'll try
> > to make it so in 2.3.

[Samuele]
> after cooking up the substitution example, I'm quite conviced that
> 2.2 mro is the worst of our options: it produces easely
> counterintuitive results, and if the conflict notion it needs is the
> one I think, it is the most prone to reject hierarchies or in case
> we ignore conflict to produce very odd results.

I don't disagree.

> So IMO our options are:

> *) use the naive algo: it is easy to explain (the algorithm) but we
> should live with the fact that it is not monotonic and I don't think
> it produces the most natural results (because it considers what is
> encoutered at the end of a dfs traversal, instead of what is at the
> beginning)

Sadly, true.

> *) adopt C3:

> (1) C3 is monotonic (it takes and merges respecting them the mros of
>     the superclasses, but in parallel, non sequentially like the
>     current 2.2 mro algo)

> (2) it respects also the inheritance lists of all superclasses; the
>     paper I cited claims that is something intuitively expected, and
>     I tend to agree.

> (3) it has another property, that basically enforces that, when this
>     does not conflicts with (1) and (2), the dfs order is respected.

> How it works: one takes the mros of the direct superclasses and the
> inheritance list of the considered class cl, and merges them as
> ordered sequences, in case of a tie the mros of the classes
> appearing earlier in the inheritance list are favored (this is what
> induces (3))
> 
> The intuitive difference with the current 2.2 mro is that C3 uses
> (3) only in case of a tie, while the current 2.2 mro algo tries to
> enforce a a mix of (1) and (3) at the same time.
> 
> From the examples this does not seem not to have any evident benefit
> (unless we consider the placement of b in ex5 such), and makes it
> the most complicated of all algorithms from the point of view of
> understanding the outcomes.

I'll have to read the paper describing C3:

http://www.webcom.com/haahr/dylan/linearization-oopsla96.html

but I expect I'll have to agree.  Thanks for your help!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Fri Oct  4 17:27:02 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 18:27:02 +0200
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net>              <037201c26bac$ede8d040$6d94fea9@newmexico>  <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <05d401c26bc2$df1f67c0$6d94fea9@newmexico>

> I'll have to read the paper describing C3:
>
> http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
>
>but I expect I'll have to agree.  Thanks for your help!

You're welcome,

btw, I just discovered (reading the source and then the documentation, oh well,
the last time I checked out of curisioty I could not find this info and did not
think to look at the source) that Goo

http://www.ai.mit.edu/~jrb/goo/index.htm

by Jonathan Bachrach et al. at MIT  is using C3 too

http://www.ai.mit.edu/~jrb/goo/manual.43/goomanual_55.html



From tim@zope.com  Fri Oct  4 17:33:01 2002
From: tim@zope.com (Tim Peters)
Date: Fri, 4 Oct 2002 12:33:01 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <oqwuoy4ddl.fsf@titan.progiciels-bpi.ca>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEOCBIAB.tim@zope.com>

[Fran=E7ois Pinard]
> ...
> Interesting, I'll save it.  I use continued fraction expansion to get
> the "best" rational fitting a float within a tolerance, and wonder how
> Farey will be similar/different.  Tim will surely tell us, out of his
> head! :-)

Your wish is granted <wink>:  Moshe Zadka's prototype implementation of
rationals is still sitting in Python CVS nondist/sandox/rational/.  Its
_trim function is one I worked on with him, and uses c.f. expansion to fi=
nd
"the best" rational approximating a given rational, among all rationals w=
ith
denominator no larger than argument max_d.  The Farey method is almost
identical, except potentially much slower.  It's almost what "the usual"
continued fraction algorithm would do if you couldn't use integer divisio=
n
to do a whole bunch of steps in one gulp; e.g., whenever the c.f. expansi=
on
gets a partial quotient of N, the Farey method does N distinct steps.  Ab=
out
"almost identical":  c.f. expansion produces a sequence of rationals
alternately smaller and larger than the target, each one (much) closer th=
an
the last.  The Farey method also looks at rationals "between" those; _tri=
m
does too, but only at the endpoint, when max_d is between the denominator=
s
of two successive c.f. convergents.

Moshe's package also has an _approximate function, to find "the smallest"
rational within a given absolute distance of an input rational; that's
probably closest to what you're doing now.  _trim answers questions like
"what's the best approximation to pi with denominator no greater than 6?".
Neither of the adjacent convergents 3/1 and 22/7 is the correct answer to
that; 19/6 is correct.  Note that the c.f. expansion gets 22/7 because th=
e
previous two convergents were 1/0 and 3/1, the next partial quotient is 7=
,
and then the next convergent is

    1 + 3*7   22
    ------- =3D --
    0 + 1*7    7

If the partial quotient *had* been 6, it would have given 19/6 instead.
That's what the tail end of _trim deduces.

The Farey method does this one step at a time, going from (and skipping t=
o
then end of process)

   1/0 3/1

as bounds to

   3/1 4/1

and then

   3/1 7/2

and then

   3/1 10/3

and then

   3/1 13/4

and then

   3/1 16/5

and then, finally

   3/1 19/6

Especially when coded in Python, it's much more efficient to deduce this =
in
one gulp (via exploiting division).



From guido@python.org  Fri Oct  4 17:52:01 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 12:52:01 -0400
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
In-Reply-To: Your message of "Fri, 04 Oct 2002 18:27:02 +0200."
 <05d401c26bc2$df1f67c0$6d94fea9@newmexico>
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net> <037201c26bac$ede8d040$6d94fea9@newmexico> <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>
 <05d401c26bc2$df1f67c0$6d94fea9@newmexico>
Message-ID: <200210041652.g94Gq1F27764@pcp02138704pcs.reston01.va.comcast.net>

> btw, I just discovered (reading the source and then the
> documentation, oh well, the last time I checked out of curisioty I
> could not find this info and did not think to look at the source)
> that Goo
> 
> http://www.ai.mit.edu/~jrb/goo/index.htm
> 
> by Jonathan Bachrach et al. at MIT  is using C3 too
> 
> http://www.ai.mit.edu/~jrb/goo/manual.43/goomanual_55.html

Could someone translate that last page into Python for me?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Fri Oct  4 17:51:59 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 18:51:59 +0200
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net> <037201c26bac$ede8d040$6d94fea9@newmexico> <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>              <05d401c26bc2$df1f67c0$6d94fea9@newmexico>  <200210041652.g94Gq1F27764@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <067401c26bc6$5b4d89a0$6d94fea9@newmexico>

From: "Guido van Rossum" <guido@python.org>>
>
> Could someone translate that last page into Python for me?

I can <wink>, during my exploration I have coded all known (to me) algorithms
(the mysterious cpl module).
A bit of polishing and I will post it.

regards.



From mal@lemburg.com  Fri Oct  4 18:04:48 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 04 Oct 2002 19:04:48 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <LNBBLJKPBEHFEDALKOLCAEOCBIAB.tim@zope.com>
Message-ID: <3D9DCA30.6010809@lemburg.com>

Tim Peters wrote:
> [Fran=E7ois Pinard]
>=20
>>...
>>Interesting, I'll save it.  I use continued fraction expansion to get
>>the "best" rational fitting a float within a tolerance, and wonder how
>>Farey will be similar/different.  Tim will surely tell us, out of his
>>head! :-)
>=20
>=20
> Your wish is granted <wink>:  Moshe Zadka's prototype implementation of
> rationals is still sitting in Python CVS nondist/sandox/rational/.  Its
> _trim function is one I worked on with him, and uses c.f. expansion to =
find
> "the best" rational approximating a given rational, among all rationals=
 with
> denominator no larger than argument max_d.  The Farey method is almost
> identical, except potentially much slower.  It's almost what "the usual=
"
> continued fraction algorithm would do if you couldn't use integer divis=
ion
> to do a whole bunch of steps in one gulp; e.g., whenever the c.f. expan=
sion
> gets a partial quotient of N, the Farey method does N distinct steps.=20

But isn't division much more costly than addition and multiplication
if you have long integers to deal with ? (I can't tell, because the
works are done by GMP in mxNumber)

> About
> "almost identical":  c.f. expansion produces a sequence of rationals
> alternately smaller and larger than the target, each one (much) closer =
than
> the last.  The Farey method also looks at rationals "between" those; _t=
rim
> does too, but only at the endpoint, when max_d is between the denominat=
ors
> of two successive c.f. convergents.
>=20
> Moshe's package also has an _approximate function, to find "the smalles=
t"
> rational within a given absolute distance of an input rational; that's
> probably closest to what you're doing now.  _trim answers questions lik=
e
> "what's the best approximation to pi with denominator no greater than 6=
?".
> Neither of the adjacent convergents 3/1 and 22/7 is the correct answer =
to
> that; 19/6 is correct.  Note that the c.f. expansion gets 22/7 because =
the
> previous two convergents were 1/0 and 3/1, the next partial quotient is=
 7,
> and then the next convergent is
>=20
>     1 + 3*7   22
>     ------- =3D --
>     0 + 1*7    7
>=20
> If the partial quotient *had* been 6, it would have given 19/6 instead.
> That's what the tail end of _trim deduces.
>=20
> The Farey method does this one step at a time, going from (and skipping=
 to
> then end of process)
>=20
>    1/0 3/1
>=20
> as bounds to
>=20
>    3/1 4/1
>=20
> and then
>=20
>    3/1 7/2
>=20
> and then
>=20
>    3/1 10/3
>=20
> and then
>=20
>    3/1 13/4
>=20
> and then
>=20
>    3/1 16/5
>=20
> and then, finally
>=20
>    3/1 19/6
>=20
> Especially when coded in Python, it's much more efficient to deduce thi=
s in
> one gulp (via exploiting division).

How useful are .trim() and .approximate() in practice ? If they
are, then I could put them on the TODO list for mxNumber.

--=20
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From pedronis@bluewin.ch  Fri Oct  4 18:31:54 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Fri, 4 Oct 2002 19:31:54 +0200
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net> <037201c26bac$ede8d040$6d94fea9@newmexico> <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>              <05d401c26bc2$df1f67c0$6d94fea9@newmexico>  <200210041652.g94Gq1F27764@pcp02138704pcs.reston01.va.comcast.net> <067401c26bc6$5b4d89a0$6d94fea9@newmexico>
Message-ID: <06ac01c26bcb$eeb704a0$6d94fea9@newmexico>

[me]
> A bit of polishing and I will post it.
>

def pmerge(seqs):
    res = []
    while 1:
      for seq in seqs:
          if seq: break
      else: # all sequences are empty
          return res
      cands = []
      for seq in seqs: # find merge candidates among seq heads
          if seq:
              cand = seq[0]
              for seq in seqs:
                  if cand in seq[1:]: break
              else: # cand is not in any sequences' tail => constraint-free
                  cands.append(cand)
      if not cands:
          raise "inconsistent precedences"
      next = cands[0]
      # append next to result and remove it from sequences
      res.append(next)
      for seq in seqs:
          if seq and seq[0] == next:
              del seq[0]

def c3_cpl(cl): # compute class precedence list (mro) of class cl according to
C3
    inputs = [[cl]]
    for base in cl.__bases__:
        cpl = c3_cpl(base)
        inputs.append(cpl)
    inputs.append(list(cl.__bases__)) # C3: considers directly also inheritance
list of cl
    return pmerge(inputs)



From tim@zope.com  Fri Oct  4 18:36:59 2002
From: tim@zope.com (Tim Peters)
Date: Fri, 4 Oct 2002 13:36:59 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <3D9DCA30.6010809@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEOKBIAB.tim@zope.com>

[M.-A. Lemburg]
> ...
> But isn't division much more costly than addition and multiplication
> if you have long integers to deal with ? (I can't tell, because the
> works are done by GMP in mxNumber)

There's no real bound on how large partial quotients can get, and rationals
can grow extremely large.  Dividing once to get, e.g., 10000, is enormously
cheaper than going around a Python loop 10000 times, and creating and
destroying several times that many temporary longs, just to avoid one
relatively fast C-speed longint division with a small quotient.

It's essentially the same as figuring out "the fastest" way to code a gcd,
and that's a very tricky problem at the Python level.  Partial quotients are
*usually* 1, and then subtraction is cheaper, and it's also possible to meld
both approaches to exploit that.

> ...
> How useful are .trim() and .approximate() in practice ?

You're going to get as many responses to that as Guido got to his query
about how mxNumber users like its type hierarchy <wink>.

> If they are, then I could put them on the TODO list for mxNumber.

They're useful for people who want to mix rationals with approximation, and
that's an unlikely intersection outside of expert use.  _trim is essentially
what fixed-slash and floating-slash arithmetics use under the covers to keep
rigorous bounds on memory use, in exchange for losing information.  How
useful is that in practice?  Beats me; it depends so much on whose practice
we're talking about <wink>.



From dave@boost-consulting.com  Fri Oct  4 18:36:09 2002
From: dave@boost-consulting.com (David Abrahams)
Date: Fri, 4 Oct 2002 13:36:09 -0400
Subject: [Python-Dev] Python with MinGW-2.0?
Message-ID: <02c601c26bcc$b1338800$6701a8c0@boostconsulting.com>

Sorry if this is only mildly on-topic. If anyone here has successfully used
MinGW-2.0 to build an extension module, please contact me.

I'd really like to support this platform for the upcoming Boost.Python
release, but it appears that MinGW-2.0 is unable to link to imported data
(e.g. PyType_Type) from a stock Win32 import library such as the one that
comes with the regular Windows install of Python.

TIA,
Dave

-----------------------------------------------------------
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com





From haering_python@gmx.de  Fri Oct  4 18:58:36 2002
From: haering_python@gmx.de (Gerhard Haering)
Date: Fri, 4 Oct 2002 19:58:36 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
Message-ID: <20021004175835.GB95201@gargamel.ghaering.test>

Trying to get around to my mingw32 port again. I currently don't have Visual
C++ installed, but why is this nonstandard _alloca needed? Can't it simply be
replaced by alloca? Doesn't MSVC have alloca?

For the moment, I'm as far as building posixmodule.c, which I succeeded by
doing a

#define _alloca alloca

If there's a way to kill MSVC peculiarities, could this please be done?

-- Gerhard


From mal@lemburg.com  Fri Oct  4 19:16:57 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 04 Oct 2002 20:16:57 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <LNBBLJKPBEHFEDALKOLCEEOKBIAB.tim@zope.com>
Message-ID: <3D9DDB19.3020502@lemburg.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>...
>>But isn't division much more costly than addition and multiplication
>>if you have long integers to deal with ? (I can't tell, because the
>>works are done by GMP in mxNumber)
> 
> 
> There's no real bound on how large partial quotients can get, and rationals
> can grow extremely large.  Dividing once to get, e.g., 10000, is enormously
> cheaper than going around a Python loop 10000 times, and creating and
> destroying several times that many temporary longs, just to avoid one
> relatively fast C-speed longint division with a small quotient.

Well, I'm working with GMP here, so temporary longs are not that
expensive (plus they reuse already allocated memory). That's
why I was asking.

> It's essentially the same as figuring out "the fastest" way to code a gcd,
> and that's a very tricky problem at the Python level.  Partial quotients are
> *usually* 1, and then subtraction is cheaper, and it's also possible to meld
> both approaches to exploit that.

I see. Thanks.

>>...
>>How useful are .trim() and .approximate() in practice ?
> 
> 
> You're going to get as many responses to that as Guido got to his query
> about how mxNumber users like its type hierarchy <wink>.

:-)

>>If they are, then I could put them on the TODO list for mxNumber.
> 
> 
> They're useful for people who want to mix rationals with approximation, and
> that's an unlikely intersection outside of expert use.  _trim is essentially
> what fixed-slash and floating-slash arithmetics use under the covers to keep
> rigorous bounds on memory use, in exchange for losing information.  How
> useful is that in practice?  Beats me; it depends so much on whose practice
> we're talking about <wink>.

Point taken.

I just added the Farey algorithm to mxNumber
because it seemed like a nice way of limiting the size of the
integers involved in the rational representation of floats.

It sometimes even helps to e.g. backpatch rounding/representation
errors in floating point calculations when you know that your
dealing with small denominator rationals:

 >>> 1/3.0
0.33333333333333331
 >>> FareyRational(1/3.0, 100)
1/3

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Fri Oct  4 20:31:13 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 15:31:13 -0400
Subject: [Python-Dev] Re: my proposals about mros (was: perplexed by mro)
In-Reply-To: Your message of "Fri, 04 Oct 2002 18:27:02 +0200."
 <05d401c26bc2$df1f67c0$6d94fea9@newmexico>
References: <200210032314.g93NEUL29325@oma.cosc.canterbury.ac.nz> <200210040154.g941spF32473@pcp02138704pcs.reston01.va.comcast.net> <037201c26bac$ede8d040$6d94fea9@newmexico> <200210041515.g94FFrw25471@pcp02138704pcs.reston01.va.comcast.net>
 <05d401c26bc2$df1f67c0$6d94fea9@newmexico>
Message-ID: <200210041931.g94JVDN29297@pcp02138704pcs.reston01.va.comcast.net>

> > I'll have to read the paper describing C3:
> >
> > http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
> >
> >but I expect I'll have to agree.  Thanks for your help!

I've read this now and I agree: we should adopt C3.  I *think* that
the current MRO algo is the same as the algorithm called C*[LOOPS] in
that paper; for all cases where they show how C*[LOOPS] differs from
Dylan or from C3, Python gives the same result as C*[LOOPS].

I wish I could find a clue in the Metaclasses book, but it's hard to
search a 300-page book for an occurrence of reference [7]. :-)

BTW for people coming late to this discussion, please read up on MRO
(Method Resolution Order) first here:

  http://www.python.org/2.2.1/descrintro.html

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct  4 20:40:50 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 15:40:50 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: Your message of "Fri, 04 Oct 2002 19:58:36 +0200."
 <20021004175835.GB95201@gargamel.ghaering.test>
References: <20021004175835.GB95201@gargamel.ghaering.test>
Message-ID: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>

> Trying to get around to my mingw32 port again. I currently don't
> have Visual C++ installed, but why is this nonstandard _alloca
> needed? Can't it simply be replaced by alloca? Doesn't MSVC have
> alloca?

It seems that it does.  But I guess _alloca is more politically
correct, since alloca is not standard C.

> For the moment, I'm as far as building posixmodule.c, which I
> succeeded by doing a
> 
> #define _alloca alloca
> 
> If there's a way to kill MSVC peculiarities, could this please be done?

I'd be happy to do a global subst of _alloca -> alloca.

Mark, do you see any reason why this might *not* work?

Could it break other compilers?

A conservative approach would be to add #ifdef CYGWIN around the
#define you propose.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Fri Oct  4 20:42:48 2002
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 4 Oct 2002 15:42:48 -0400
Subject: [Python-Dev] Re: PEP 285 (Adding a bool type) extending it further
In-Reply-To: <3D9CEABE.5020804@acms.arizona.edu>
References: <3D9CDFDD.2050807@acms.arizona.edu> <15772.58386.655293.791830@slothrop.zope.com> <3D9CEABE.5020804@acms.arizona.edu>
Message-ID: <20021004194248.GA28850@panix.com>

On Thu, Oct 03, 2002, Marcelo Matus wrote:
>
> Yes, I can use the &,|,~ operators,  and that is what I am doing right now.
> 
> But your question also apply to the boolean type:
> 
>   can't you just use an integer type and leave it like that?

Because Python is a delicate balancing act.  We try to make it easy to
learn *and* use for newcomers to programming; we try to make it easy to
learn *and* use for experienced programmers; we try to keep the core
language small; we try to make it powerful.  Start up interactive Python
and type "import this" for the extended version of this philosophy.

So, specifics: Guido resisted adding a boolean type for a long time,
believing that it would clutter things.  Ten years of experience shows,
though, that almost every medium-to-large application defines some kind
of boolean type internally.  Aggregate clutter is reduced by putting
Booleans into the language.

You want to make a base-level change like you're proposing, you need to
put in the legwork to convince people that it's a net improvement.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/


From haering_python@gmx.de  Fri Oct  4 21:23:48 2002
From: haering_python@gmx.de (Gerhard Haering)
Date: Fri, 4 Oct 2002 22:23:48 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
References: <20021004175835.GB95201@gargamel.ghaering.test> <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021004202348.GA96145@gargamel.ghaering.test>

* Guido van Rossum <guido@python.org> [2002-10-04 15:40 -0400]:
> > Trying to get around to my mingw32 port again. I currently don't
> > have Visual C++ installed, but why is this nonstandard _alloca
> > needed? Can't it simply be replaced by alloca? Doesn't MSVC have
> > alloca?
> 
> It seems that it does.  But I guess _alloca is more politically
> correct, since alloca is not standard C.

I don't see how this applies, as neither form is standard C, but in practise,
alloca is supported by all compilers I use, be it Windows or Linux. And I don't
care about P. C. ;-)

> > For the moment, I'm as far as building posixmodule.c, which I
> > succeeded by doing a
> > 
> > #define _alloca alloca
> > 
> > If there's a way to kill MSVC peculiarities, could this please be done?
> 
> I'd be happy to do a global subst of _alloca -> alloca.
> 
> Mark, do you see any reason why this might *not* work?
> 
> Could it break other compilers?

No, as it's only used in platform-specific code as seen below:

$ find . -name *.[ch]|xargs grep -w _alloca
./Modules/posixmodule.c:		s1 = (char *)_alloca(i);
./Modules/posixmodule.c:			s2 = (char *)_alloca(x);
./Modules/posixmodule.c:			s2 = (char *)_alloca(x);
./Python/pythonrun.c:		/* _alloca throws a stack overflow exception if there's
./Python/pythonrun.c:		_alloca(PYOS_STACK_MARGIN * sizeof(void*));

$ find . -name *.[ch]|xargs grep -w alloca
./Modules/mpzmodule.c:** 			alloca with arg < 0 (when casted to a signed
./PC/_winreg.c:#include "malloc.h" /* for alloca */
./PC/_winreg.c:	retBuf = (char *)alloca(len);
./PC/_winreg.c:	retValueBuf = (char *)alloca(retValueSize);
./PC/_winreg.c:	retDataBuf = (char *)alloca(retDataSize);
./PC/_winreg.c:	retBuf = (char *)alloca(bufSize);
./PC/_winreg.c:	retBuf = (char *)alloca(bufSize);
./PC/import_nt.c:#include "malloc.h" /* for alloca */
./PC/import_nt.c:	/* alloca == no free required, but memory only local to fn,
./PC/import_nt.c:	moduleKey = alloca(bufSize); 
./PC/os2vacpp/getpathp.c:#include "malloc.h" // for alloca - see comments below!
./PC/os2vacpp/getpathp.c:	// alloca == no free required, but memory only local to fn.
./PC/os2vacpp/getpathp.c:	keyBuf = alloca(sizeof(keyPrefix)-1 + versionLen + sizeof(keySuffix)); // chars only, plus 1 NULL.

So, alloca and _alloca are only used in platform specific Windows and OS/2
code. The MSVC specific code is a little inconsistent, it uses both _alloca and
alloca forms depending on the source file.

> A conservative approach would be to add #ifdef CYGWIN around the
> #define you propose.

Not even more #ifdefs, please. I'd suggest the global replace _alloca ->
alloca.

-- Gerhard


From martin@v.loewis.de  Fri Oct  4 21:35:31 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Oct 2002 22:35:31 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
References: <20021004175835.GB95201@gargamel.ghaering.test>
 <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3ofaaeycs.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Doesn't MSVC have alloca?

> It seems that it does.

Without checking: It probably has this only if __STDC__ is not
defined. MSVC hides all non-standard symbols if __STDC__ is defined -
either by compiler switches, or in some application header.

In that sense, _alloca is the Microsoft name for this extension. They
may have been misguided by using that scheme, though, since _alloca
isn't any more reserved than alloca.

However, all uses of _alloca in posixmodule.c are MS specific, so I
really see no reason to change that.

Regards,
Martin



From guido@python.org  Fri Oct  4 21:52:30 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 16:52:30 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: Your message of "Fri, 04 Oct 2002 22:23:48 +0200."
 <20021004202348.GA96145@gargamel.ghaering.test>
References: <20021004175835.GB95201@gargamel.ghaering.test> <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
 <20021004202348.GA96145@gargamel.ghaering.test>
Message-ID: <200210042052.g94KqUj30872@pcp02138704pcs.reston01.va.comcast.net>

> Not even more #ifdefs, please. I'd suggest the global replace _alloca ->
> alloca.

Submit a patch.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mhammond@skippinet.com.au  Sat Oct  5 00:34:16 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 5 Oct 2002 09:34:16 +1000
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>

> > Trying to get around to my mingw32 port again. I currently don't
> > have Visual C++ installed, but why is this nonstandard _alloca
> > needed? Can't it simply be replaced by alloca? Doesn't MSVC have
> > alloca?
> 
> It seems that it does.  But I guess _alloca is more politically
> correct, since alloca is not standard C.
> 
> > For the moment, I'm as far as building posixmodule.c, which I
> > succeeded by doing a
> > 
> > #define _alloca alloca
> > 
> > If there's a way to kill MSVC peculiarities, could this please be done?
> 
> I'd be happy to do a global subst of _alloca -> alloca.
> 
> Mark, do you see any reason why this might *not* work?

None at all.

Mark.



From gerhard.haering@gmx.de  Sat Oct  5 00:50:39 2002
From: gerhard.haering@gmx.de (Gerhard Haering)
Date: Sat, 5 Oct 2002 01:50:39 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>
References: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net> <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>
Message-ID: <20021004235039.GA97477@gargamel.ghaering.test>

* Mark Hammond <mhammond@skippinet.com.au> [2002-10-05 09:34 +1000]:
> > > [me: change _alloca to alloca]
> > [Guido:]
> > Mark, do you see any reason why this might *not* work?
> 
> None at all.

As per suggestion of Guido, I've submitted a patch. The patch, however, doesn't
make any sense standalone, so it's part of #1 of many mingw related patches.
These will ultimately make it possible to build Python on win32 using the
autoconf/make toolchain.

See http://python.org/sf/618791

I've assigned it to Guido for no good reason except being not sure if Mark
knows about autoconf/makesetup, which my patch touches.

But I'd also be glad if anybody else applies it ;-)

-- Gerhard


From guido@python.org  Sat Oct  5 01:18:15 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 04 Oct 2002 20:18:15 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: Your message of "Sat, 05 Oct 2002 01:50:39 +0200."
 <20021004235039.GA97477@gargamel.ghaering.test>
References: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net> <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>
 <20021004235039.GA97477@gargamel.ghaering.test>
Message-ID: <200210050018.g950IFO32089@pcp02138704pcs.reston01.va.comcast.net>

> As per suggestion of Guido, I've submitted a patch. The patch,
> however, doesn't make any sense standalone, so it's part of #1 of
> many mingw related patches.

So I should ignore the patch until you're done submitting those other
patches?

> These will ultimately make it possible
> to build Python on win32 using the autoconf/make toolchain.

I don't understand.  How many Unix emulations on Windows do we need?
We've already absorbed a sheer endless set of patches to make it work
on CYGWIN.  How does Mingw differ?

> See http://python.org/sf/618791
> 
> I've assigned it to Guido for no good reason except being not sure
> if Mark knows about autoconf/makesetup, which my patch touches.
> 
> But I'd also be glad if anybody else applies it ;-)

Me too.  I may reduce the priority.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Oct  5 02:05:53 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 04 Oct 2002 21:05:53 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBMBJAB.tim.one@comcast.net>

[Guido]
>> I'd be happy to do a global subst of _alloca -> alloca.
>>
>> Mark, do you see any reason why this might *not* work?

[Mark]
> None at all.

In the interest of universal harmony, I've done this and will check it in
soon (when the tests complete).

no-point-arguing-over-non-issues-ly y'rs  - tim



From tismer@tismer.com  Sat Oct  5 02:16:04 2002
From: tismer@tismer.com (Christian Tismer)
Date: Fri, 04 Oct 2002 18:16:04 -0700
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
References: <20021004175835.GB95201@gargamel.ghaering.test>	<200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net> <m3ofaaeycs.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3D9E3D54.6020403@tismer.com>

Martin v. Loewis wrote:
> Guido van Rossum <guido@python.org> writes:
> 
> 
>>>Doesn't MSVC have alloca?
>>
> 
>>It seems that it does.
> 
> 
> Without checking: It probably has this only if __STDC__ is not
> defined. MSVC hides all non-standard symbols if __STDC__ is defined -
> either by compiler switches, or in some application header.

Hmm. I had this problem (no alloca, no idea why) and defined
it by a macro in this case.
Is it cleaner to undefine __STDC__ instead?

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tim.one@comcast.net  Sat Oct  5 02:23:12 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 04 Oct 2002 21:23:12 -0400
Subject: [Python-Dev] test_pep277 vs Win98
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBNBJAB.tim.one@comcast.net>

The newish test_pep277 fails on Win98SE, then leaves behind a directory
named @test that causes 4 later tests to fail for bogus reasons.

Anyone know whether this test is supposed to work <wink>?  Mark, Martin?

C:\Code\python\PCbuild>python ../lib/test/test_pep277.py
test_directory (__main__.UnicodeFileTests) ... ERROR
test_failures (__main__.UnicodeFileTests) ... ERROR
test_listdir (__main__.UnicodeFileTests) ... ERROR
test_open (__main__.UnicodeFileTests) ... ERROR
test_rename (__main__.UnicodeFileTests) ... ERROR

======================================================================
ERROR: test_directory (__main__.UnicodeFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 31, in setUp
    f = open(name, 'w')
IOError: [Errno 2] No such file or directory: '@test\\Ge??-sa?'

======================================================================
ERROR: test_failures (__main__.UnicodeFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 31, in setUp
    f = open(name, 'w')
IOError: [Errno 2] No such file or directory: '@test\\Ge??-sa?'

======================================================================
ERROR: test_listdir (__main__.UnicodeFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 31, in setUp
    f = open(name, 'w')
IOError: [Errno 2] No such file or directory: '@test\\Ge??-sa?'

======================================================================
ERROR: test_open (__main__.UnicodeFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 31, in setUp
    f = open(name, 'w')
IOError: [Errno 2] No such file or directory: '@test\\Ge??-sa?'

======================================================================
ERROR: test_rename (__main__.UnicodeFileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 31, in setUp
    f = open(name, 'w')
IOError: [Errno 2] No such file or directory: '@test\\Ge??-sa?'

----------------------------------------------------------------------
Ran 5 tests in 0.270s

FAILED (errors=5)
Traceback (most recent call last):
  File "../lib/test/test_pep277.py", line 106, in ?
    test_main()
  File "../lib/test/test_pep277.py", line 103, in test_main
    run_suite(suite)
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 187, in run_suite
    raise TestFailed(msg)
test.test_support.TestFailed: errors occurred; run in verbose mode for
details

C:\Code\python\PCbuild>



From gerhard.haering@gmx.de  Sat Oct  5 02:26:47 2002
From: gerhard.haering@gmx.de (Gerhard Haering)
Date: Sat, 5 Oct 2002 03:26:47 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <200210050018.g950IFO32089@pcp02138704pcs.reston01.va.comcast.net>
References: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net> <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au> <20021004235039.GA97477@gargamel.ghaering.test> <200210050018.g950IFO32089@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021005012647.GA98206@gargamel.ghaering.test>

* Guido van Rossum <guido@python.org> [2002-10-04 20:18 -0400]:
> > As per suggestion of Guido, I've submitted a patch. The patch,
> > however, doesn't make any sense standalone, so it's part of #1 of
> > many mingw related patches.
> 
> So I should ignore the patch until you're done submitting those other
> patches?

No, this can go in now. That's the point of my plan to submit several isolated
patches. I don't want to fork a Python tree, work several months on it, and
then submit one gigantic patch, that will probably never be applied because
there are problems reviewing it.

> > These will ultimately make it possible to build Python on win32 using the
> > autoconf/make toolchain.
> 
> I don't understand.  How many Unix emulations on Windows do we need?

mingw is _not_ a Unix emulation. It is the GNU Compiler suite targetting
_native_ Windows. The point is to be able to build Python on Windows with the
autoconf toolchain. And _without_ needing to buy Visual C++ just to do build
Python or to do serious Python extension development on Windows.

> We've already absorbed a sheer endless set of patches to make it work on
> CYGWIN.  How does Mingw differ?

mingw produces native win32 executables. No emulation whatsoever.

> > See http://python.org/sf/618791
> > 
> > I've assigned it to Guido for no good reason except being not sure
> > if Mark knows about autoconf/makesetup, which my patch touches.
> > 
> > But I'd also be glad if anybody else applies it ;-)
> 
> Me too.  I may reduce the priority.

The reason for setting a higher priority was that I hoped this could be applied
rather sooner than later, as I hoped this wasn't a controversial patch.

-- Gerhard


From gerhard.haering@gmx.de  Sat Oct  5 02:30:32 2002
From: gerhard.haering@gmx.de (Gerhard Haering)
Date: Sat, 5 Oct 2002 03:30:32 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <3D9E3D54.6020403@tismer.com>
References: <3D9E3D54.6020403@tismer.com>
Message-ID: <20021005013032.GB98206@gargamel.ghaering.test>

* Christian Tismer <tismer@tismer.com> [2002-10-04 18:16 -0700]:
> Martin v. Loewis wrote:
> >Guido van Rossum <guido@python.org> writes:
> >
> >
> >>>Doesn't MSVC have alloca?
> >>
> >
> >>It seems that it does.
> >
> >
> >Without checking: It probably has this only if __STDC__ is not
> >defined. MSVC hides all non-standard symbols if __STDC__ is defined -
> >either by compiler switches, or in some application header.
> 
> Hmm. I had this problem (no alloca, no idea why) and defined
> it by a macro in this case.

I've now installed MSVC6SP4 and verified that with my patch Python still builds
ok. Obviously, Python's compiler options for MSVC don't define __STDC__.

> Is it cleaner to undefine __STDC__ instead?

It'd be cleaner to use either _alloca or alloca, but not both like the current
M$VC specific code does.

-- Gerhard


From mhammond@skippinet.com.au  Sat Oct  5 02:44:32 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 5 Oct 2002 11:44:32 +1000
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEBNBJAB.tim.one@comcast.net>
Message-ID: <LCEPIIGDJPKCOIHOBJEPGEMKGLAA.mhammond@skippinet.com.au>

> The newish test_pep277 fails on Win98SE, then leaves behind a directory
> named @test that causes 4 later tests to fail for bogus reasons.
>
> Anyone know whether this test is supposed to work <wink>?  Mark, Martin?

It is not supposed to work on Win9x <wink/frown>

Not exactly sure what to do about this.  Maybe we should expose
posixmodule's unicode_file_names() function - some way of indicating if the
underlying file system supports Unicode.

Mark.



From martin@v.loewis.de  Sat Oct  5 02:45:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 03:45:56 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <3D9E3D54.6020403@tismer.com>
References: <20021004175835.GB95201@gargamel.ghaering.test>
 <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
 <m3ofaaeycs.fsf@mira.informatik.hu-berlin.de>
 <3D9E3D54.6020403@tismer.com>
Message-ID: <m3k7kx4q0b.fsf@mira.informatik.hu-berlin.de>

Christian Tismer <tismer@tismer.com> writes:

> Hmm. I had this problem (no alloca, no idea why) and defined
> it by a macro in this case.
> Is it cleaner to undefine __STDC__ instead?

No. I would always go with the Microsoft name. If the code also needs
to work on Unix, add a define for the functions you need.

Regards,
Martin



From mhammond@skippinet.com.au  Sat Oct  5 03:24:38 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 5 Oct 2002 12:24:38 +1000
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPGEMKGLAA.mhammond@skippinet.com.au>
Message-ID: <LCEPIIGDJPKCOIHOBJEPAEMMGLAA.mhammond@skippinet.com.au>

> > The newish test_pep277 fails on Win98SE, then leaves behind a directory
> > named @test that causes 4 later tests to fail for bogus reasons.
> >
> > Anyone know whether this test is supposed to work <wink>?  Mark, Martin?
>
> It is not supposed to work on Win9x <wink/frown>
>
> Not exactly sure what to do about this.  Maybe we should expose
> posixmodule's unicode_file_names() function - some way of
> indicating if the
> underlying file system supports Unicode.

OK - in the efforts of actually progressing something <wink>, would you
prefer a patch that:

* Added os._get_windows_version() as an alias for the Win32 GetVersion()
function. We then clone the test in posixmodule.c for test_pep277.py, and
life is good.  This function would be generally useful - it is almost a FAQ
on c.l.python and all the answers are generally unsatisfactory.

* Added os._unicode_file_apis(), which exposes the internal posixmodule
function unicode_file_names().  Currently this is simply a GetVersion()
call, but later it may grow additional checks.

* Did something else completely <wink>

Mark.



From martin@v.loewis.de  Sat Oct  5 03:58:10 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 04:58:10 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEBNBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEBNBJAB.tim.one@comcast.net>
Message-ID: <m3wuox383h.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> The newish test_pep277 fails on Win98SE, then leaves behind a directory
> named @test that causes 4 later tests to fail for bogus reasons.
> 
> Anyone know whether this test is supposed to work <wink>?  Mark, Martin?

To my knowledge, it can't work on Win9x. It would be good to skip that
on this system.

Regards,
Martin


From martin@v.loewis.de  Sat Oct  5 04:05:58 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 05:05:58 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPAEMMGLAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPAEMMGLAA.mhammond@skippinet.com.au>
Message-ID: <m3smzl37qh.fsf@mira.informatik.hu-berlin.de>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

> > Not exactly sure what to do about this.  Maybe we should expose
> > posixmodule's unicode_file_names() function - some way of
> > indicating if the underlying file system supports Unicode.

I think we already rejected this approach when developing the PEP. The
function name would be misleading: You can pass Unicode as a file name
on all systems, thanks to the file system default encoding.

> * Added os._get_windows_version() as an alias for the Win32 GetVersion()
> function. We then clone the test in posixmodule.c for test_pep277.py, and
> life is good.  This function would be generally useful - it is almost a FAQ
> on c.l.python and all the answers are generally unsatisfactory.

I would like this approach, although I'm uncertain why the function
name needs to start with an underscore (for that matter, I also wonder
why _winreg starts with an underscore).

os.windows_version() sounds good to me, although I could also accept
os.get_windows_version(). Even os.version might work.

> * Added os._unicode_file_apis(), which exposes the internal posixmodule
> function unicode_file_names().  Currently this is simply a GetVersion()
> call, but later it may grow additional checks.

You like underscores really much :-) See above. Also, I think the
likelyness of adding additional checks is very small: The only other
candidate system to provide a similar feature would be OS X. However,
macmodule.c lives elsewhere, to my knowledge.

> * Did something else completely <wink>

Perhaps there is already a way to find out the windows version you are
running? Perhaps by looking at the registry?

Regards,
Martin



From tim.one@comcast.net  Sat Oct  5 04:09:01 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 04 Oct 2002 23:09:01 -0400
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPAEMMGLAA.mhammond@skippinet.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCCECEBJAB.tim.one@comcast.net>

[Mark Hammond]
> OK - in the efforts of actually progressing something <wink>, would you
> prefer a patch that:
>
> * Added os._get_windows_version() as an alias for the Win32 GetVersion()
> function. We then clone the test in posixmodule.c for test_pep277.py,
> and life is good.  This function would be generally useful - it is
> almost a FAQ on c.l.python and all the answers are generally
> unsatisfactory.

I agree that would be (very) useful to have, but also agree with your
unstated but fiercly held belief that it's a hacky solution to the immediate
problem at hand <wink>.

> * Added os._unicode_file_apis(), which exposes the internal posixmodule
> function unicode_file_names().  Currently this is simply a GetVersion()
> call, but later it may grow additional checks.

This sounds more direct, but loses me.  Presumably the test is testing
something <wink> that *could* be a given an informative name.  Think like a
Tim here:  what does this test *do*?  Then why is it that Windows NT+ are
the only OSes on Earth where it's possible to do that thing?  Or are the
only OSes on Earth where it's necessary to do that thing?  A Tim-friendly
one-word answer to that question would give a nice name for a new boolean
attribute to test:

if not os.supports_FRIENDLY_NAME:
    rase TestSkipped, "test requires os.supports_FRIENDLY_NAME"

I don't know what FRIENDLY_NAME should be, but "_unicode_file_apis()"
doesn't seem likely to be it.



From tim.one@comcast.net  Sat Oct  5 04:11:26 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 04 Oct 2002 23:11:26 -0400
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <m3wuox383h.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECEBJAB.tim.one@comcast.net>

[Martin v. Loewis]
> To my knowledge, it can't work on Win9x. It would be good to skip that
> on this system.

Is this necessarily a different-flavors-of-Windows distinction, or is there
some more fundamental underlying capability that could (and should) be given
an OS-neutral name?  That is, if it's not necessarily a
different-flavors-of-Windows distinction for all time, testing a Windows OS
version number isn't the right answer to this.



From andymac@bullseye.apana.org.au  Sat Oct  5 01:38:34 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 5 Oct 2002 11:38:34 +1100 (edt)
Subject: [Python-Dev] Python with MinGW-2.0?
In-Reply-To: <02c601c26bcc$b1338800$6701a8c0@boostconsulting.com>
Message-ID: <Pine.OS2.4.32.0210051127220.389-100000@tenring.andymac.org>

On Fri, 4 Oct 2002, David Abrahams wrote:

> I'd really like to support this platform for the upcoming Boost.Python
> release, but it appears that MinGW-2.0 is unable to link to imported data
> (e.g. PyType_Type) from a stock Win32 import library such as the one that
> comes with the regular Windows install of Python.

My limited experience with MingW 1.1 and a DLL that contained data (and
code) in the import library as well as the imports suggests that the
import library has to be rebuilt to the .a format and split between static
(code+data objs) and dynamic (import) libraries.  This is messy & tedious
to do, but seems to work (I couldn't figure out a way to combine the
static objs & imports into the one .a library, though I wasn't able to
spend a great deal of time on it).

I gather MS "enhanced" the import library format somewhere along the line,
and MingW's ld doesn't deal with it (yet).

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From David Abrahams" <david.abrahams@rcn.com  Sat Oct  5 05:23:07 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 5 Oct 2002 00:23:07 -0400
Subject: [Python-Dev] Python with MinGW-2.0?
References: <Pine.OS2.4.32.0210051127220.389-100000@tenring.andymac.org>
Message-ID: <011001c26c27$0c6ba220$6701a8c0@boostconsulting.com>

From: "Andrew MacIntyre" <andymac@bullseye.apana.org.au>

> My limited experience with MingW 1.1 and a DLL that contained data (and
> code) in the import library as well as the imports suggests that the
> import library has to be rebuilt to the .a format and split between
static
> (code+data objs) and dynamic (import) libraries.  This is messy & tedious
> to do, but seems to work (I couldn't figure out a way to combine the
> static objs & imports into the one .a library, though I wasn't able to
> spend a great deal of time on it).
>
> I gather MS "enhanced" the import library format somewhere along the
line,
> and MingW's ld doesn't deal with it (yet).

Thanks, For some reason I didn't have to do any of that with MinGW 1.1, but
for MinGW-2.0, I did. Wasn't too painful, though: Gerhard sent a pointer to
these instructions which made everything work (after substantial code
massage):

http://www.python.org/doc/current/inst/non-ms-compilers.html#SECTION0003120
00000000000000


-----------------------------------------------------------
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com





From guido@python.org  Sat Oct  5 08:18:47 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 05 Oct 2002 03:18:47 -0400
Subject: [Python-Dev] Python 2.2.2 beta release on Monday!
Message-ID: <200210050718.g957IlF01399@pcp02138704pcs.reston01.va.comcast.net>

I plan to do a Python 2.2.2 beta release this Monday (if Fred's and
Tim's time is available to do their part).

Please finish your changes to 2.2.2 before the weekend is over!

Remember, use "cvs update -r release22-maint" to check out the 2.2.2
release branch.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Oct  5 08:25:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 05 Oct 2002 03:25:28 -0400
Subject: [Python-Dev] Python 2.2.2 beta release on Monday!
In-Reply-To: <200210050718.g957IlF01399@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCECNBJAB.tim.one@comcast.net>

[Guido[
> I plan to do a Python 2.2.2 beta release this Monday (if Fred's and
> Tim's time is available to do their part).

No problem here, and I see Fred's been working on the docs.

> Please finish your changes to 2.2.2 before the weekend is over!

Is there a list of outstanding issues to work on?  The NEWS file is often a
last-second mess <wink>.



From guido@python.org  Sat Oct  5 08:32:57 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 05 Oct 2002 03:32:57 -0400
Subject: [Python-Dev] Python 2.2.2 beta release on Monday!
In-Reply-To: Your message of "Sat, 05 Oct 2002 03:25:28 EDT."
 <LNBBLJKPBEHFEDALKOLCCECNBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCECNBJAB.tim.one@comcast.net>
Message-ID: <200210050732.g957WwD01579@pcp02138704pcs.reston01.va.comcast.net>

> Is there a list of outstanding issues to work on?  The NEWS file is
> often a last-second mess <wink>.

I'm not sure it's worth mentioning every backported fix in NEWS.
Backported features there aren't, except the new email package, for
which Barry already added a NEWS entry.  So I'm not sure there's much
to do.

But let's all read PEP 102 before it's too late... :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Sat Oct  5 10:26:45 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 11:26:45 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIECEBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIECEBJAB.tim.one@comcast.net>
Message-ID: <m3adltme22.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Is this necessarily a different-flavors-of-Windows distinction, or is there
> some more fundamental underlying capability that could (and should) be given
> an OS-neutral name?

There is an abstract underlying capability "supports arbitrary Unicode
file names"; the test more specifically relies on the capability
"supports those 8 file names being tested".

I doubt that this should be given a name at all, be it OS neutral or
not. Whatever name you give it, it is likely that the capability will
vary across systems, If you take a strict interpretation, you will
find that no system supports it (e.g. Windows doesn't allow \ in a
file name); if you take a loser definition, you find that all systems
support it.

> That is, if it's not necessarily a different-flavors-of-Windows
> distinction for all time, testing a Windows OS version number isn't
> the right answer to this.

The test should be run on Win NT (and sons) only. It tests the feature
that was just implemented, and that implementation is specific for
Windows NT.

Regards,
Martin


From martin@v.loewis.de  Sat Oct  5 10:32:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 11:32:43 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCECEBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCECEBJAB.tim.one@comcast.net>
Message-ID: <m365whmds4.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Think like a Tim here: what does this test *do*?

Currently, the test verifies that a certain piece of C code bracketed
by #ifdef MS_WINDOWS works correctly.

> Then why is it that Windows NT+ are the only OSes on Earth where
> it's possible to do that thing?

Testing the Windows code works can work only on Windows, by nature. It
so happens that it won't work on W9x, which leaves NT+.

> Or are the only OSes on Earth where it's necessary to do that thing?

No, users of other systems would want that feature as
well. Unfortunately, it can be correctly implemented only on OS X in
addition to NT+; a somewhat-flawed implementation would be possible
for Unix.

> I don't know what FRIENDLY_NAME should be, but "_unicode_file_apis()"
> doesn't seem likely to be it.

It is really hard to find a precise name, but whatever the answer, it
likely contains the words "unicode" and "file (name)".

Regards,
Martin


From tim.one@comcast.net  Sat Oct  5 19:19:11 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 05 Oct 2002 14:19:11 -0400
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <m3adltme22.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEEBBJAB.tim.one@comcast.net>

[martin@v.loewis.de]
> ...
> There is an abstract underlying capability "supports arbitrary Unicode
> file names"; the test more specifically relies on the capability
> "supports those 8 file names being tested".
>
> I doubt that this should be given a name at all, be it OS neutral or
> not. Whatever name you give it, it is likely that the capability will
> vary across systems, If you take a strict interpretation, you will
> find that no system supports it (e.g. Windows doesn't allow \ in a
> file name); if you take a loser definition, you find that all systems
> support it.

> ...
> It is really hard to find a precise name, but whatever the answer, it
> likely contains the words "unicode" and "file (name)".

This seems too much logic-chopping to me.  Python supports unbounded ints
too, although they're not really unbounded, and it's impossible to say
exactly how unbounded they are across platforms -- or even on one platform.
"Unbounded ints" remains a helpful description regardless.  I'm an expert on
such things, so I *could* quibble about "unbounded ints" endlessly, but it's
not helpful to do so.

Define os.path.supports_unicode_filenames as "supports the 8 specific
filenames tested by test_pep277.py", and then that's a precisely defined
lower bound that should have nothing to do with Windows specifically.  After
all, there is no Windows-specific code in this test!  A platform supports
what this test tries to do, or it doesn't.  If it does, fine, then by
definition it supports_unicode_filenames, and that's what the TestSkipped
logic can test for.

If this is thought to be a particularly stressful set of 8 specific file
names (I can't guess -- "ascii" is the only one I can read <wink>), then
change the set of file names to a more reasonable one.  test_long.py doesn't
try to create billion-digit integers, rather it restricts itself to
"unbounded ints" that any *reasonable* platform can handle.  Do likewise for
Unicode filename support?



From martin@v.loewis.de  Sat Oct  5 22:39:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Oct 2002 23:39:59 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEEBBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEEBBJAB.tim.one@comcast.net>
Message-ID: <m3adlslg40.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Define os.path.supports_unicode_filenames as "supports the 8 specific
> filenames tested by test_pep277.py", and then that's a precisely defined
> lower bound that should have nothing to do with Windows specifically.  

If you take this definition, it will be very difficult to determine
whether this should be True or False on Unix. On some systems, it will
depend on environment variables whether these are valid file names or
not.

Also, the test tests whether os.listdir(u".") returns Unicode file
names - which is currently never the case on Unix, even though they
may support the file names as arguments. So should I add
os.path.listdir_with_unicode_argument_returns_unicode as well?

Furthermore, how would I implement os.path.supports_unicode_filenames
for ntpath? It seems that something must be exported from os.

> If this is thought to be a particularly stressful set of 8 specific
> file names (I can't guess -- "ascii" is the only one I can read
> <wink>), then change the set of file names to a more reasonable one.
> test_long.py doesn't try to create billion-digit integers, rather it
> restricts itself to "unbounded ints" that any *reasonable* platform
> can handle.  Do likewise for Unicode filename support?

PEP 277 gives Python applications access to all file names on Windows
NT; this is a property unique to NT: on all other systems, you can
access all file names using byte strings. For the test to test that
feature, we need to chose a set of file names that are cannot all be
represented as byte strings, simultaneously.

Regards,
Martin


From tim.one@comcast.net  Sat Oct  5 23:17:31 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 05 Oct 2002 18:17:31 -0400
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <m3adlslg40.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEEJBJAB.tim.one@comcast.net>

[martin@v.loewis.de]
> If you take this definition, it will be very difficult to determine
> whether this should be True or False on Unix. On some systems, it will
> depend on environment variables whether these are valid file names or
> not.

Then set it to False without thinking on Unix, until Unix systems evolve to
the point where Unicode strings are generally useful file paths.

> Also, the test tests whether os.listdir(u".") returns Unicode file
> names - which is currently never the case on Unix, even though they
> may support the file names as arguments. So should I add
> os.path.listdir_with_unicode_argument_returns_unicode as well?

I would consider that more unhelpul logic-chopping <wink>.  I would hope
that intuitive listdir() behavior was part of what it *means* for Unicode
strings to be generally useful as file paths.  If no version of Unix
supports that (yet), fine, then no version of Unix can (yet) claim
supports_unicode_filenames in this sense.

> Furthermore, how would I implement os.path.supports_unicode_filenames
> for ntpath? It seems that something must be exported from os.

This sounds like, for now, whether a platform supports this or not is a
matter of hand-forcing a config symbol, where only a few Windows flavors are
currently justified in saying "yup, works here".

> ...
> PEP 277 gives Python applications access to all file names on Windows
> NT; this is a property unique to NT: on all other systems, you can
> access all file names using byte strings. For the test to test that
> feature, we need to chose a set of file names that are cannot all be
> represented as byte strings, simultaneously.

I still see nothing specific to Windows in this, apart from that NT+ appear
to be, for now, the only OSes that support Unicode strings as file paths in
a generally useful way.  The crux of PEP 277 from my POV is not that Windows
supports this, but that some flavors of Windows allow "direct access to all
file names, including those that are not representable using the current
locale", and using Unicode.  That's a Good Idea, and I doubt Microsoft has a
patent on it.  The PEP also mentions that it seems *plausible* to implement
this for Win9x using VFAT-32, so it's again not a "am I on Windows, and if
so which flavor?" issue at heart.



From martin@v.loewis.de  Sun Oct  6 01:43:44 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 02:43:44 +0200
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEEJBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEEJBJAB.tim.one@comcast.net>
Message-ID: <m3it0g75xb.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> > Furthermore, how would I implement os.path.supports_unicode_filenames
> > for ntpath? It seems that something must be exported from os.
> 
> This sounds like, for now, whether a platform supports this or not is a
> matter of hand-forcing a config symbol, where only a few Windows flavors are
> currently justified in saying "yup, works here".

Repeating my question: How can I hand-force a config symbol to True on
NT+ and to False on W9x?

Regards,
Martin



From tim.one@comcast.net  Sun Oct  6 02:05:14 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 05 Oct 2002 21:05:14 -0400
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <m3it0g75xb.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEEPBJAB.tim.one@comcast.net>

[martin@v.loewis.de]
> Repeating my question: How can I hand-force a config symbol to True on
> NT+ and to False on W9x?

It seemed like a rhetorical question in context <wink>, but if it's serious
then offhand I'm not sure.  If it's something we agree is desirable, then it
needs to be a runtime decision on Windows (given that Python supports this
stuff under NT+ but not (yet) under Win9x, and we only ship one Windows
binary).  One way to distinguish between NT+ and Win9x at runtime is in the
module initialization for winsound.c.  It's easy in C but exceedingly
non-obvious.  A more principled way would be to test for the existence of a
relevant system DLL function; I expect only Mark would know whether there is
such a beast in this case.



From mhammond@skippinet.com.au  Sun Oct  6 02:19:24 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sun, 6 Oct 2002 11:19:24 +1000
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <m3smzl37qh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LCEPIIGDJPKCOIHOBJEPIEOMGLAA.mhammond@skippinet.com.au>

Me:
> > * Added os._get_windows_version() as an alias for the Win32 GetVersion()
> > function.

Martin:
> I would like this approach, although I'm uncertain why the function
> name needs to start with an underscore (for that matter, I also wonder
> why _winreg starts with an underscore).

Actually, _winreg was named that was because there was some debate over a
higher level interface to the registry.  There wasn't general agreement on
the API, but there was agreement we could check-in a low-level "_winreg"
implementation, leaving "winreg" for some presumably PEP-designed interface.

> os.windows_version() sounds good to me, although I could also accept
> os.get_windows_version(). Even os.version might work.

And Tim later:

> I agree that would be (very) useful to have, but also agree with your
> unstated but fiercly held belief that it's a hacky solution to the
> immediate problem at hand <wink>.

Well spotted!  However, pragmatism generally gets in the way <wink/frown>.
I've opted for sys.getwindowsversion() - see http://www.python.org/sf/619108

Mark.



From mhammond@skippinet.com.au  Sun Oct  6 02:23:30 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sun, 6 Oct 2002 11:23:30 +1000
Subject: [Python-Dev] test_pep277 vs Win98
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIEOMGLAA.mhammond@skippinet.com.au>
Message-ID: <LCEPIIGDJPKCOIHOBJEPAEONGLAA.mhammond@skippinet.com.au>

> Well spotted!  However, pragmatism generally gets in the way <wink/frown>.
> I've opted for sys.getwindowsversion() - see
> http://www.python.org/sf/619108

Which I created immediately before seeing the rest of this thread!  Let's
continue the discussion in that patch, whatever form it ends up taking.

Mark.



From skip@pobox.com  Sun Oct  6 02:01:12 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 5 Oct 2002 20:01:12 -0500
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210041254.g94CsKG21170@pcp02138704pcs.reston01.va.comcast.net>
References: <200210030042.g930guG20522@oma.cosc.canterbury.ac.nz>
 <oqu1k36oo7.fsf@carouge.sram.qc.ca>
 <200210040055.g940tD711784@pcp02138704pcs.reston01.va.comcast.net>
 <oqwuoy4ddl.fsf@titan.progiciels-bpi.ca>
 <200210041254.g94CsKG21170@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15775.35672.406480.187688@Skip-Montanaros-Computer.local.>

    >> If Guido was adding complex numbers today instead of long ago, I
    >> wonder if he would allow a special notation for them, or just suggest
    >> a constructor.

    Guido> At the time I believe there was heavy pressure from the Numeric
    Guido> crowd to allow a special notation.  I'm not so sure if I should
    Guido> have given in though.

However, 1+4j can be peephole optimized into a compile-time constant whereas
complex(1,4) can't.  This is generally not a big deal, but to people who
deal with complex numbers a lot (and tend to be more sensitive to
optimization issues) it can be.  I believe when I tested my peephole
optimizer using pybench several years ago, the complex number tests showed
the most improvement because I could collapse constant expressions.

Of course, people using lots of complex numbers probably initialize their
complex constants outside of loops. ;-)

Skip



From tim.one@comcast.net  Sun Oct  6 05:37:12 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 00:37:12 -0400
Subject: [Python-Dev] Testing the 2.2.2 email pkg
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFKBJAB.tim.one@comcast.net>

Under current 2.2 CVS, running

python ../lib/test/regrtest.py test_email.py

from my test directory works fine.  But running test_email.py directly is a
disaster:

python ../lib/test/test_email.py

There's too much failure output to include here reasonably.  Bottom line is:

"""
........................FFFFFFFFFFFFFFFFFFFF...................E..FF.F.
.....F..F......E......E.....E....F......................F.F............
.F.F.F............F..F

...

Ran 164 tests in 0.930s

FAILED (failures=33, errors=4)
"""

I can't yet try it in an installation, as it appears the Windows installer
needs first to be taught about the new test directories under the email
package (so, under an installation, test_email doesn't work at all now, no
matter how it's run).



From tim.one@comcast.net  Sun Oct  6 06:00:07 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 01:00:07 -0400
Subject: [Python-Dev] 2.2 versus test_trace
Message-ID: <LNBBLJKPBEHFEDALKOLCAEFMBJAB.tim.one@comcast.net>

The CVS test_trace.py is failing for me when run under the debug build *and*
with -O.

"""
C:\Code\22\PCbuild>python_d -O ../lib/test/test_trace.py
Adding parser accelerators ...
Done.
test_1_basic (__main__.TraceTestCase) ... FAIL
test_2_arigo (__main__.TraceTestCase) ... FAIL
test_3_one_instr (__main__.TraceTestCase) ... FAIL
test_4_no_pop_blocks (__main__.TraceTestCase) ... FAIL
test_5_no_pop_tops (__main__.TraceTestCase) ... FAIL
test_6_call (__main__.TraceTestCase) ... FAIL
test_7_raise (__main__.TraceTestCase) ... FAIL
test_8_settrace_and_return (__main__.TraceTestCase) ... FAIL
test_9_settrace_and_raise (__main__.TraceTestCase) ... FAIL

======================================================================
FAIL: test_1_basic (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 173, in test_1_basic
    self.run_test(basic)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (1, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_2_arigo (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 175, in test_2_arigo
    self.run_test(arigo_example)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (2, 'line')
- (3, 'line')
- (3, 'line')
- (5, 'line')
- (5, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_3_one_instr (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 177, in test_3_one_instr
    self.run_test(one_instr_line)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (2, 'line')
- (3, 'line')
- (3, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_4_no_pop_blocks (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 179, in test_4_no_pop_blocks
    self.run_test(no_pop_blocks)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (1, 'line')
- (3, 'line')
- (3, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_5_no_pop_tops (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 181, in test_5_no_pop_tops
    self.run_test(no_pop_tops)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (2, 'line')
- (2, 'line')
- (3, 'line')
- (6, 'line')
- (2, 'line')
- (3, 'line')
- (4, 'line')
- (2, 'line')
- (2, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_6_call (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 183, in test_6_call
    self.run_test(call)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
  (-3, 'call')
- (-3, 'line')
- (-2, 'line')
- (-2, 'return')
?   ^

+ (-3, 'return')
?   ^

- (1, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_7_raise (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 185, in test_7_raise
    self.run_test(test_raise)
  File "../lib/test/test_trace.py", line 163, in run_test
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
  (0, 'call')
- (0, 'line')
- (1, 'line')
- (2, 'line')
  (-3, 'call')
- (-3, 'line')
- (-2, 'line')
- (-2, 'exception')
?   ^

+ (-3, 'exception')
?   ^

- (2, 'exception')
?  ^

+ (0, 'exception')
?  ^

- (3, 'line')
- (4, 'line')
- (4, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_8_settrace_and_return (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 188, in test_8_settrace_and_return
    self.run_test2(settrace_and_return)
  File "../lib/test/test_trace.py", line 170, in run_test2
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
- (1, 'return')
?  ^

+ (0, 'return')
?  ^


======================================================================
FAIL: test_9_settrace_and_raise (__main__.TraceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 190, in test_9_settrace_and_raise
    self.run_test2(settrace_and_raise)
  File "../lib/test/test_trace.py", line 170, in run_test2
    tracer.events, func.events)
  File "../lib/test/test_trace.py", line 154, in compare_events
    map(str, events))))
  File "C:\CODE\22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: events did not match expectation:
- (2, 'exception')
?  ^

+ (0, 'exception')
?  ^

- (3, 'line')
- (4, 'line')
- (4, 'return')
?  ^

+ (0, 'return')
?  ^


----------------------------------------------------------------------
Ran 9 tests in 0.220s

FAILED (failures=9)
Traceback (most recent call last):
  File "../lib/test/test_trace.py", line 216, in ?
    test_main()
  File "../lib/test/test_trace.py", line 212, in test_main
    test_support.run_unittest(TraceTestCase)
  File "../lib/test\test_support.py", line 180, in run_unittest
    run_suite(unittest.makeSuite(testclass), testclass)
  File "../lib/test\test_support.py", line 174, in run_suite
    raise TestFailed(msg)
test_support.TestFailed: errors occurred in __main__.TraceTestCase
[6768 refs]

C:\Code\22\PCbuild>
"""

It also fails under the release build with -O.  It does not fail in either
build if I leave -O off.  This smells familiar, and I think I fixed this
before.  I don't want to fix it again <wink>.



From skip@manatee.mojam.com  Sun Oct  6 13:00:24 2002
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 6 Oct 2002 07:00:24 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200210061200.g96C0OnL027526@manatee.mojam.com>

Bug/Patch Summary
-----------------

293 open / 2905 total bugs (+1)
112 open / 1719 total patches (+3)

New Bugs
--------

packages doc in the tutorial (2002-09-29)
	http://python.org/sf/616211
plat-linux2/IN.py FutureWarning's (2002-10-03)
	http://python.org/sf/618012
overflow error in calendar module (2002-10-03)
	http://python.org/sf/618146
Windows binary missing IPv6 support (2002-10-04)
	http://python.org/sf/618593
Connecting to IPv4 addrs from IPv6 socke (2002-10-04)
	http://python.org/sf/618594
list(xrange(sys.maxint/4)) again (2002-10-04)
	http://python.org/sf/618623
Use C3 MRO algorithm (2002-10-04)
	http://python.org/sf/618704

New Patches
-----------

textwrap: prefix short lines (2002-09-30)
	http://python.org/sf/616496
getframe hook (Psyco #1) (2002-10-01)
	http://python.org/sf/617309
Tiny profiling info (Psyco #2) (2002-10-01)
	http://python.org/sf/617311
debugger-controlled jumps (Psyco #3) (2002-10-01)
	http://python.org/sf/617312
Rational Reference Implementation (2002-10-02)
	http://python.org/sf/617779
gzip.py and files > 2G (2002-10-03)
	http://python.org/sf/618135
[mingw patches] alloca and posixmodule (2002-10-04)
	http://python.org/sf/618791
Fix test_pep277 on Win9x (2002-10-05)
	http://python.org/sf/619108

Closed Bugs
-----------

ImportError: No module named _socket (2002-06-07)
	http://python.org/sf/565710
NetBSD 1.4.3, a.out, shared modules (2002-08-17)
	http://python.org/sf/596576
socket.getfqdn() doesn't on Windows (2002-09-27)
	http://python.org/sf/615472
getdefaultlocale failure on OS X (2002-09-28)
	http://python.org/sf/616002

Closed Patches
--------------

test_commands test fails under Cygwin (2002-04-16)
	http://python.org/sf/544740
PEP 277: Unicode file name support (2002-08-12)
	http://python.org/sf/594001
ccompiler argument checking too strict (2002-09-02)
	http://python.org/sf/603831
build fixes for SCO (2002-09-26)
	http://python.org/sf/615069
acconfig.h out of date (2002-09-26)
	http://python.org/sf/615343


From barry@wooz.org  Sun Oct  6 13:17:34 2002
From: barry@wooz.org (Barry A. Warsaw)
Date: Sun, 6 Oct 2002 08:17:34 -0400
Subject: [Python-Dev] Re: Testing the 2.2.2 email pkg
References: <LNBBLJKPBEHFEDALKOLCKEFKBJAB.tim.one@comcast.net>
Message-ID: <15776.10718.561784.209566@gargle.gargle.HOWL>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    TP> Under current 2.2 CVS, running

    TP> python ../lib/test/regrtest.py test_email.py

    TP> from my test directory works fine.  But running test_email.py
    TP> directly is a disaster:

    TP> python ../lib/test/test_email.py

    TP> There's too much failure output to include here reasonably.
    TP> Bottom line is:

    TP> """
    TP> ........................FFFFFFFFFFFFFFFFFFFF...................E..FF.F.
    TP> .....F..F......E......E.....E....F......................F.F............
    TP> .F.F.F............F..F

    TP> ...

    TP> Ran 164 tests in 0.930s

    | FAILED (failures=33, errors=4)
    | """

    TP> I can't yet try it in an installation, as it appears the
    TP> Windows installer needs first to be taught about the new test
    TP> directories under the email package (so, under an
    TP> installation, test_email doesn't work at all now, no matter
    TP> how it's run).

It's gotta be a Windows thing:

% cat CVS/Tag
Trelease22-maint
% cvs -q up -P -d
% make
% ./python Lib/test/test_email.py
....................................................................................................................................................................
----------------------------------------------------------------------
Ran 164 tests in 1.179s

OK

This on RH7.3.  Could it be the "cvs transforms line endings" problem
we looked at a few weeks ago?  Does Python 2.3 cvs have the same
problem (it's got a nearly identical snapshot of the email package)?

If you want to send me the test output directly, feel free, although
I'm not sure how much time I'll have today to look at it.

-Barry


From guido@python.org  Sun Oct  6 14:29:24 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 09:29:24 -0400
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: Your message of "Sun, 06 Oct 2002 13:57:14 +0200."
 <20021006115713.GB22400@lysator.liu.se>
References: <20021006115713.GB22400@lysator.liu.se>
Message-ID: <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>

[Anders Qvist]
> Several machines in the snake farm fail to build libpython2.3 because
> they don't have ar in their path. ar, ld and the like live in
> /usr/ccs/bin, which seems to be some kind of standard (for generous
> values of standard) as both SunOS 5.8, AIX 4.3 and HP/UX 11.00 seems
> ot have this dir.
> 
> Should the configure script try with /usr/ccs/bin/ar and ld if none is
> explicitly supplied?

This may be a good idea, but I don't know zit about this area.  I'm
cc'ing python-dev, maybeo someone there understands this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Oct  6 15:11:33 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 10:11:33 -0400
Subject: [Python-Dev] Re: Testing the 2.2.2 email pkg
In-Reply-To: Your message of "Sun, 06 Oct 2002 08:17:34 EDT."
 <15776.10718.561784.209566@gargle.gargle.HOWL>
References: <LNBBLJKPBEHFEDALKOLCKEFKBJAB.tim.one@comcast.net>
 <15776.10718.561784.209566@gargle.gargle.HOWL>
Message-ID: <200210061411.g96EBXa18691@pcp02138704pcs.reston01.va.comcast.net>

There's one file in the email/test/data directory that contains
newlines when I check it out: msg_26.txt.

I find it hard to believe that alone would cause 33 failures and 4
errors, but who knows.

I note that I see a similar set of errors and failures with Python
2.3, so it appears to be a general problem with the new email
package's test suite on Windows, when invoked manually (python
Lib/test/test_email.py).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Oct  6 15:19:18 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 10:19:18 -0400
Subject: [Python-Dev] Re: Testing the 2.2.2 email pkg
In-Reply-To: Your message of "Sun, 06 Oct 2002 10:11:33 EDT."
 <200210061411.g96EBXa18691@pcp02138704pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEFKBJAB.tim.one@comcast.net> <15776.10718.561784.209566@gargle.gargle.HOWL>
 <200210061411.g96EBXa18691@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210061419.g96EJIu18736@pcp02138704pcs.reston01.va.comcast.net>

> I note that I see a similar set of errors and failures with Python
> 2.3, so it appears to be a general problem with the new email
> package's test suite on Windows, when invoked manually (python
> Lib/test/test_email.py).

Another thought just struck me.  Are we certain that the email test
suite is run when you use regrtest?  I think not!

[linux]$ time ./python ../Lib/test/test_email.py
....................................................................................................................................................................
----------------------------------------------------------------------
Ran 164 tests in 0.899s

OK

real    0m1.136s
user    0m1.020s
sys     0m0.070s
[linux]$ time ./python ../Lib/test/regrtest.py test_email
test_email
1 test OK.

real    0m0.248s
user    0m0.220s
sys     0m0.030s
[linux]$ 

The times of the second run suggest that no tests were run!

So the email regression test (or the email package!) probably has a
severe Unix dependency (or Windows deficiency :-) that has never been
detected before.

I'll mail the full test output from Windows box next.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sun Oct  6 16:06:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 11:06:28 -0400
Subject: [Python-Dev] Re: Testing the 2.2.2 email pkg
In-Reply-To: <200210061419.g96EJIu18736@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHABJAB.tim.one@comcast.net>

This gets too confusing too fast.  test_email fails under 2.3 CVS for me
this morning too, no matter how I run it -- via regrtest, by itself, doesn't
matter.

STRAIGHT REGREST
>python ../lib/test/regrtest.py test_email
test_email
test test_email failed -- errors occurred; run in verbose mode for details
1 test failed:
    test_email


DITTO BUT VERBOSE
>python ../lib/test/regrtest.py -v test_email
... lots of error output ...
Ran 164 tests in 1.150s
FAILED (failures=33, errors=4)
test test_email failed -- errors occurred; run in verbose mode for details
1 test failed:
    test_email

BY ITSELF
>python ../lib/test/test_email.py
... lots of error output ...
Ran 164 tests in 1.150s

FAILED (failures=33, errors=4)
Traceback (most recent call last):
  File "../lib/test/test_email.py", line 13, in ?
    test_main()
  File "../lib/test/test_email.py", line 10, in test_main
    run_suite(suite())
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 187, in run_suite
    raise TestFailed(msg)
test.test_support.TestFailed: errors occurred; run in verbose mode for
details

Why I'm getting a traceback from test_support despite not using regrtest
seems a bit of a mystery.

SKIPPING THE TEST DIRECTORY ENTIRELY
>python ../lib/email/test/test_email.py
... lots of error output ...
Ran 164 tests in 0.770s

FAILED (failures=33, errors=4)


> Another thought just struck me.  Are we certain that the email test
> suite is run when you use regrtest?  I think not!

As above, it certainly is for me, under current 2.3 CVS.

> ...
> So the email regression test (or the email package!) probably has a
> severe Unix dependency (or Windows deficiency :-) that has never been
> detected before.

Barry knows he has cross-platform line-end problems in his test data, but
hasn't wanted to "do something" about that so far.  I'm most unclear now on
why test_email ever worked on Windows; *something* changed very recently,
but don't know what.... lots of error output ...



From tim.one@comcast.net  Sun Oct  6 16:11:53 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 11:11:53 -0400
Subject: [Python-Dev] Re: Testing the 2.2.2 email pkg
In-Reply-To: <200210061411.g96EBXa18691@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHBBJAB.tim.one@comcast.net>

[Guido]
> ...
> I note that I see a similar set of errors and failures with Python
> 2.3, so it appears to be a general problem with the new email
> package's test suite on Windows, when invoked manually (python
> Lib/test/test_email.py).

This morning (but not yesterday) test_email fails for me here (under 2.3
CVS) even when run indirectly, via just running regrtest.  I assume some
checkin occurred I haven't located yet.



From tim.one@comcast.net  Sun Oct  6 16:28:41 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 11:28:41 -0400
Subject: [Python-Dev] RE: Testing the 2.2.2 email pkg
In-Reply-To: <15776.10718.561784.209566@gargle.gargle.HOWL>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEHDBJAB.tim.one@comcast.net>

[Barry]
> ...
> Could it be the "cvs transforms line endings" problem we looked at a
> few weeks ago?

The problem we looked at a few weeks ago was that WinZip transforms line
endings in text files, unless you disable that (it's a well-hidden WinZip
option).

CVS also transforms text-file line endings, which is something we did not
look at before, but which should have been obvious at the time.  I'm sure
it's the same problem.  It was hidden because regrtest.py didn't actually
run the email tests until Guido fixed that this morning.  Now test_email on
Windows fails no matter how it's run.

> Does Python 2.3 cvs have the same problem (it's got a nearly identical
> snapshot of the email package)?

Since Guido fixed regrtest-vs-test_email this morning, yes.

> If you want to send me the test output directly, feel free, although
> I'm not sure how much time I'll have today to look at it.

You already know what the problem is, so no need.



From paul@pfdubois.com  Sun Oct  6 17:14:01 2002
From: paul@pfdubois.com (Paul F Dubois)
Date: Sun, 6 Oct 2002 09:14:01 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021006120102.30351.31268.Mailman@mail.python.org>
Message-ID: <000001c26d53$64253b70$6701a8c0@NICKLEBY>

The question was raised as to why the Numeric community really wanted
the literal notation 1.0+2.0j for complex numbers.

The motivation was not speed particularly; nobody is going to construct
a large complex array out of literals. It is true that inside a loop a
constant might be a problem but you could always lift it by hand. I'd
say rather that the real motivation was ease of learning for those who
knew similar languages. There were essentially three ways this was done
in other languages that our users would already grok:

(1.0, 2.0) -- Fortran; a non-starter for Python
1.0 + 2.0i or 1.0 + 2.0j -- most (all?) existing tools for similar
purposes use one of these. 

We all felt that Python would be perceived as "inferior" to these other
programs if you had to do something clumsy.

I haven't followed the rational discussion but obviously there is no
other long-standing and highly used notation for rationals except
1.0/2.0, is there?  



From jacobs@penguin.theopalgroup.com  Sun Oct  6 18:13:30 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sun, 6 Oct 2002 13:13:30 -0400 (EDT)
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
Message-ID: <Pine.LNX.4.44.0210061231560.28570-100000@penguin.theopalgroup.com>

Hi Guido and python-dev,

Python 2.2.2 beta 1 is coming out, and I was hoping to float a final request
to have the fix for bug #537450 applied to the 2.2 maint branch (it is a
trivial backport, since the 2.3 patch applies just fine).  A back-port was
originally passed over, because it changes semantics from Python 2.2.1.  I
argue that the changed semantics are desirable and constitute a clear bug
fix, which closes holes that can allow very obscure errors.  It would be
very surprising to me if this change breaks any extant code, since it is
virtually impossible to take meaningful or safe advantage of the original
semantics.

Here is, in brief, an example of what I am talking about (the bug and
python-dev archives have more information on this topic).  Let's define a
class Foo:

class Foo(object):
  def __new__(cls):
    return [1,2,3]

print Foo()

Okay, this _is_ a contrived example, though it does reflects a real use-case
that appears in one of my large real-world apps.  Here is the output from
Python 2.2.1:

  >>> print Foo()
  []         # !!! not what we expected!

and from Python 2.3 CVS:

  >>> print Foo()
  [1, 2, 3]  # Just right!

This is because any object returned from __new__ has its constructor called,
regardless of its type in 2.2.1.  The new rule in Python 2.3 CVS is that the
constructor is only called when the type returned is an instance of the type
being built (including subtype instances).

The above example demonstrates the problems with the 2.2.1 behavior on lists,
though more problems are exposed when we consider other object types:

class Bar(object):
  def __init__(self, a, b, c):
    pass

class Foo(object):
  def __new__(cls,a):
    if a:
      return Bar(1,2,3)
    else:
      return super(Foo, cls).__new__(cls)

Here is what happens in Python 2.3 CVS:

  >>> print Foo(0)
  <__main__.Foo object at 0x402baf0c>
  >>> print Foo(1)
  <__main__.Bar object at 0x402bab6c>

All looks as it should, based on the discussions from when bug #537450 was
last discussed.

Here is what happens in Python 2.2.1 (and the current 2.2.2 CVS):

  >>> print Foo(0)
  <__main__.Foo object at 0x8174434>
  >>> print Foo(1)
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  TypeError: __init__() takes exactly 4 arguments (2 given)


In the first case where a=0, the super-class new is called, and a Foo
instance is returned.  All is fine, so far.  

However, the second case returns an initialized Bar instance, which Foo
tries to initialize again by calling Bar.__init__ with the same arguments
that were passed into Foo.__new__.  An arity error results, which is
actually a good thing, since arbitrarily calling another object's
constructor with arguments passed in for another object is a decidedly bad
thing to do in general.  The more severe problem occurs when the arguments
do conform, and an instance of an improperly constructed object is returned. 
Either way, Bar is initialized _twice_, once with the intended arguments,
another with arguments that were intended for Foo!

I assert that no sensible developer would ever write code that relied on the
pre-2.3 CVS semantics.  I've done a survey of much of the meta-class code
posted to or discussed on Python-dev and comp.lang.python, and none rely on
this behavior.  Plus, even if they did, we have already decided they will
have to change it when 2.3 comes out.

If you are not entirely convinced by the above argument, please consider a
compromise solution: put the fix in for 2.2.1 Beta 1.  If anyone has
complaints based on extant code, then we can always back it out for Beta 2,
or the final release.

Thanks for taking the time...

-Kevin Jacobs

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From mats@laplaza.org  Sun Oct  6 18:45:44 2002
From: mats@laplaza.org (Mats Wichmann)
Date: Sun, 06 Oct 2002 11:45:44 -0600
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <20021006160005.8968.48937.Mailman@mail.python.org>
Message-ID: <5.1.0.14.1.20021006114025.01dc2ca0@204.151.72.2>

 >[Anders Qvist]
 >> Several machines in the snake farm fail to build libpython2.3 because
 >> they don't have ar in their path. ar, ld and the like live in
 >> /usr/ccs/bin, which seems to be some kind of standard (for generous
 >> values of standard) as both SunOS 5.8, AIX 4.3 and HP/UX 11.00 seems
 >> ot have this dir.
 >>
 >> Should the configure script try with /usr/ccs/bin/ar and ld if none is
 >> explicitly supplied?
 >
 >This may be a good idea, but I don't know zit about this area.  I'm
 >cc'ing python-dev, maybeo someone there understands this.
 >
 >--Guido van Rossum (home page: http://www.python.org/~guido/)

/usr/ccs was a semi-standard location introduced in the
System V days, probably not before SVR4.  ccs = C Compilation
System, don't know whether to blame Sun or USL for this
innovation :-) which used to give me lots of support problems
because nobody ever had these things in their path.  These
days it can be fun when people try to run gcc without binutils
on Solaris, since again some key stuff may not be in the path.

Don't see that it would hurt to have the configure script
try there, perhaps limited by OS type? The only one of the
above I know about is Solaris, though.




From pedronis@bluewin.ch  Sun Oct  6 20:02:05 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 6 Oct 2002 21:02:05 +0200
Subject: [Python-Dev] C3 implementation
Message-ID: <031e01c26d6a$dcb4e9c0$6d94fea9@newmexico>

I have an apparently working implementation of C3 in C.

test_descr/test_descrtut pass apart some tests that involves hierarchies with
MRO order disagreements. In such cases my impl raise TypeError: MRO order
disagreement exceptions.

Some of them it's clear to me how to fix them myself, for others someone should
think about what was the original intention of the test.

I can add some tests about monotonicity etc, with examples from the paper and
the discussion we had here.

Should I submit this as  a patch on SF? at least is a start.

PS:
1. the implementation is worst-case quadratic in the total length of the
superclasses' mros, like the Python one I have presented here or the one used
in Goo. I don't think this has relevance in practice, it's a trade-off versus
even more complicated logic and data structure usage.



From guido@python.org  Sun Oct  6 20:40:48 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 15:40:48 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Sun, 06 Oct 2002 13:13:30 EDT."
 <Pine.LNX.4.44.0210061231560.28570-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210061231560.28570-100000@penguin.theopalgroup.com>
Message-ID: <200210061940.g96Jem719413@pcp02138704pcs.reston01.va.comcast.net>

You could have picked a better time than one day before the beta
releast to open this up.  I'm already asked to consider several
patches (the Psyco ones) at the very last minute.

In the SF entry I say that it's a clear semantic change and I don't
want to change it in 2.2.

I'll try to see if I can find time to read your message, but if I
don't, I won't feel guilty.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Oct  6 20:44:12 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 15:44:12 -0400
Subject: [Python-Dev] C3 implementation
In-Reply-To: Your message of "Sun, 06 Oct 2002 21:02:05 +0200."
 <031e01c26d6a$dcb4e9c0$6d94fea9@newmexico>
References: <031e01c26d6a$dcb4e9c0$6d94fea9@newmexico>
Message-ID: <200210061944.g96JiC019534@pcp02138704pcs.reston01.va.comcast.net>

> I have an apparently working implementation of C3 in C.

Woo hoo!

> test_descr/test_descrtut pass apart some tests that involves
> hierarchies with MRO order disagreements. In such cases my impl
> raise TypeError: MRO order disagreement exceptions.

I'd like that to be a (new?) warning instead, so the user can control
whether to turn it into an error or ignore it or fix it.

> Some of them it's clear to me how to fix them myself, for others
> someone should think about what was the original intention of the
> test.
> 
> I can add some tests about monotonicity etc, with examples from the
> paper and the discussion we had here.
> 
> Should I submit this as  a patch on SF? at least is a start.

Yes please!  Assign to me, I'll get on it after I push 2.2.2 beta 1
out of the door.

> PS:
> 1. the implementation is worst-case quadratic in the total length of
> the superclasses' mros, like the Python one I have presented here or
> the one used in Goo. I don't think this has relevance in practice,
> it's a trade-off versus even more complicated logic and data
> structure usage.

Sounds fine given the expected size of the average inheritance
lattice.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedronis@bluewin.ch  Sun Oct  6 20:41:11 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 6 Oct 2002 21:41:11 +0200
Subject: [Python-Dev] Fw: MRO 2.2 vs 2.3
Message-ID: <038401c26d70$52fd6c60$6d94fea9@newmexico>

----- Original Message ----- 
From: "Robin Becker" <robin@jessikat.fsnet.co.uk>
Newsgroups: comp.lang.python
Sent: Sunday, October 06, 2002 3:21 PM
Subject: MRO 2.2 vs 2.3


> Will the C3 method resolution algorithm be back ported into Python
> 2.2.x? It appears from the recently updated
> 
> http://www.python.org/2.2.1/descrintro.html#mro
> 
> that this will be used in 2.3, but the PBF version will then be out of
> step with the future.
> -- 
> Robin Becker


From martin@v.loewis.de  Sun Oct  6 20:53:38 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 21:53:38 +0200
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
References: <20021006115713.GB22400@lysator.liu.se>
 <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3n0pr1gzh.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Several machines in the snake farm fail to build libpython2.3 because
> > they don't have ar in their path. ar, ld and the like live in
> > /usr/ccs/bin, which seems to be some kind of standard (for generous
> > values of standard) as both SunOS 5.8, AIX 4.3 and HP/UX 11.00 seems
> > ot have this dir.
> > 
> > Should the configure script try with /usr/ccs/bin/ar and ld if none is
> > explicitly supplied?
> 
> This may be a good idea, but I don't know zit about this area.  I'm
> cc'ing python-dev, maybeo someone there understands this.

Trying ar from /usr/ccs/bin might be reasonable, but it *must* try
ar from PATH before that.

I fail to see what common scenario this would fail for, though: If you
don't have ar(1) in your path, how do you find make(1)?

In no case configure should try /usr/ccs/bin/ld: Invoking ld directly
is always incorrect (except for archaic systems).

Regards,
Martin


From guido@python.org  Sun Oct  6 21:30:21 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 16:30:21 -0400
Subject: [Python-Dev] Anders Qvist: Re: [snake-farm] Autodetect /usr/ccs/bin?
Message-ID: <200210062030.g96KULV28711@pcp02138704pcs.reston01.va.comcast.net>

I've no time to research this further right now, so I'm forwarding
Anders' mail here.  It seems that Martin disagrees with Pike's
solution though.

--Guido van Rossum (home page: http://www.python.org/~guido/)

------- Forwarded Message

Date:    Sun, 06 Oct 2002 22:03:57 +0200
From:    Anders Qvist <quest@lysator.liu.se>
To:      Guido van Rossum <guido@python.org>
Subject: Re: [snake-farm] Autodetect /usr/ccs/bin?

On Sun, Oct 06, 2002 at 09:29:24AM -0400, Guido van Rossum wrote:
> [Anders Qvist]
> > Several machines in the snake farm fail to build libpython2.3 because
> > they don't have ar in their path. ar, ld and the like live in
> > /usr/ccs/bin, which seems to be some kind of standard (for generous
> > values of standard) as both SunOS 5.8, AIX 4.3 and HP/UX 11.00 seems
> > ot have this dir.
> > 
> > Should the configure script try with /usr/ccs/bin/ar and ld if none is
> > explicitly supplied?
> 
> This may be a good idea, but I don't know zit about this area.  I'm
> cc'ing python-dev, maybeo someone there understands this.

Apparently, the pike people do this. They seem to think it's the right
thing to do. From their configure.in:

  case "$pike_cv_sys_os" in
    ...
    Solaris) LDSHARED="/usr/ccs/bin/ld -G";;
    ...

They also add it to their PATH during various circumstances. Have a
look at:

http://community.roxen.com/_internal/cvsview!0/8223/1.633/0/configure.in
- -- 
Anders "Quest" Qvist

"We've all heard that a million monkeys banging on a million typewriters
will eventually reproduce the entire works of Shakespeare. Now, thanks 
to the Internet, we know this is not true." -- Robert Wilensky 

------- End of Forwarded Message



From jacobs@penguin.theopalgroup.com  Sun Oct  6 21:31:40 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sun, 6 Oct 2002 16:31:40 -0400 (EDT)
Subject: [Python-Dev] Bug #537450 Improper object initialization for
 type.__new__
In-Reply-To: <200210061940.g96Jem719413@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>

On Sun, 6 Oct 2002, Guido van Rossum wrote:
> You could have picked a better time than one day before the beta
> releast to open this up.  I'm already asked to consider several
> patches (the Psyco ones) at the very last minute.

Sorry -- it was on the 2.2.2 wiki since the day it was created.  Plus, I've
been out of town much of the time since then.

> In the SF entry I say that it's a clear semantic change and I don't
> want to change it in 2.2.

> I'll try to see if I can find time to read your message, but if I
> don't, I won't feel guilty.

I would appreciate it.  This is something of a pet bug, since it is the
single thing in 2.2.2 that will break a fairly large application that we
have developed.  For somewhat complex reasons, there is no easy
work-around for it.

Thanks,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Sun Oct  6 21:57:36 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 16:57:36 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Sun, 06 Oct 2002 16:31:40 EDT."
 <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
Message-ID: <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>

> > You could have picked a better time than one day before the beta
> > releast to open this up.  I'm already asked to consider several
> > patches (the Psyco ones) at the very last minute.
> 
> Sorry -- it was on the 2.2.2 wiki since the day it was created.
> Plus, I've been out of town much of the time since then.

Sigh.  Wikis have one big problem: unless you check them regularly
it's easy to miss a discussion going on there.  I'm sorry about that
-- I've completely failed to check the Wiki.

BTW, the Wiki is here: http://manatee.mojam.com/py222wiki/

I tried to set up an email subscription but the Wiki isn't configured
to support that...

> > In the SF entry I say that it's a clear semantic change and I don't
> > want to change it in 2.2.
> 
> > I'll try to see if I can find time to read your message, but if I
> > don't, I won't feel guilty.
> 
> I would appreciate it.  This is something of a pet bug, since it is
> the single thing in 2.2.2 that will break a fairly large application
> that we have developed.  For somewhat complex reasons, there is no
> easy work-around for it.

I've now read your whole message.  I reluctantly agree that it's
reasonable to fix this and will try to get this in the beta.

These are the outstanding issues for the beta that I'm aware of now:

 - 537450 (this)
 - three patches from Armin Rigo to make life easier for Psyco
 - test_email fails on Windows
 - patch 619352: Updated .spec file for 2.2.2b1
 - whatever's in the Wiki

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Sun Oct  6 22:05:41 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sun, 6 Oct 2002 17:05:41 -0400 (EDT)
Subject: [Python-Dev] Anders Qvist: Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <200210062030.g96KULV28711@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0210061702080.29303-100000@penguin.theopalgroup.com>

On Sun, 6 Oct 2002, Guido van Rossum wrote:
> ------- Forwarded Message
> 
> Date:    Sun, 06 Oct 2002 22:03:57 +0200
> From:    Anders Qvist <quest@lysator.liu.se>
> To:      Guido van Rossum <guido@python.org>
> Subject: Re: [snake-farm] Autodetect /usr/ccs/bin?
> 
> Apparently, the pike people do this. They seem to think it's the right
> thing to do. From their configure.in:
> 
>   case "$pike_cv_sys_os" in
>     ...
>     Solaris) LDSHARED="/usr/ccs/bin/ld -G";;
>     ...
> 
> They also add it to their PATH during various circumstances. Have a
> look at:

I _know_ this is wrong for several non-native (non-SunPro) Solaris
compilers. My applications assume that $PATH has the correct ld/ar/cpp.  If
not, we augment the search to look in /usr/ccs, /opt/sunWPro, and other more
obscure locations. I believe this is in accord with Martin suggestion.

Regards,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From Raymond Hettinger" <python@rcn.com  Sun Oct  6 22:20:32 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Sun, 6 Oct 2002 17:20:32 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>  <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <003001c26d7e$3ce5df80$890ca044@oemcomputer>

From: "Guido van Rossum" <guido@python.org>
> These are the outstanding issues for the beta that I'm aware of now:
> 
>  - 537450 (this)
>  - three patches from Armin Rigo to make life easier for Psyco
>  - test_email fails on Windows
>  - patch 619352: Updated .spec file for 2.2.2b1
>  - whatever's in the Wiki

The last one includes a list of modules that still need
backport attention (by someone more knowledgable than me):

- Doc/ref/ref5.tex 
- Doc/ref/ref7.tex 
- Lib/socket.py 
- Lib/pydoc.py 
- Lib/pickle.py 
- Lib/posixpath.py

Also, I've made a large number of documentation
backports since Fred rebuilt the docs on Friday.  So,
one more doc build is in order before the beta goes out.


Raymond Hettinger


From martin@v.loewis.de  Sun Oct  6 22:23:28 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 23:23:28 +0200
Subject: [Python-Dev] Anders Qvist: Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <200210062030.g96KULV28711@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062030.g96KULV28711@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3zntrz2gf.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Apparently, the pike people do this. They seem to think it's the right
> thing to do. From their configure.in:
> 
>   case "$pike_cv_sys_os" in
>     ...
>     Solaris) LDSHARED="/usr/ccs/bin/ld -G";;
>     ...
> 
> They also add it to their PATH during various circumstances. Have a
> look at:

Using ld directly is wrong because it fails to add implicit linker
options, in particular crt*.o. For example, on SCO, giving -Kpthread
to the compiler implies giving -lpthread -l<some funny library that
you cannot guess> to the linker.

Likewise, using gcc on Solaris, linking with gcc implies linking
crtbegin.o and crtend.o, which is necessary to run constructors of
global C++ objects. The same holds for SunPR, which implicitly links
crti.o, crt1.o, values-xa.o, and crtn.o for an executable, and crti.o,
values-xa.o, and crtn.o for a shared object.

Regards,
Martin


From martin@v.loewis.de  Sun Oct  6 22:30:52 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 23:30:52 +0200
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
 <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3vg4fz243.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

>  - whatever's in the Wiki

I'd suggest to ignore the Wiki. If the PBF ever manages to act, there
will be a 2.2.3 release. Only consider things that just require a
decision; if you find that things require work beyond applying a
right-looking patch, have proponents do the work for 2.2.3.

Regards,
Martin



From martin@v.loewis.de  Sun Oct  6 22:34:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 23:34:04 +0200
Subject: [Python-Dev] Anders Qvist: Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <Pine.LNX.4.44.0210061702080.29303-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210061702080.29303-100000@penguin.theopalgroup.com>
Message-ID: <m3r8f3z1yr.fsf@mira.informatik.hu-berlin.de>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> I _know_ this is wrong for several non-native (non-SunPro) Solaris
> compilers. My applications assume that $PATH has the correct ld/ar/cpp.  If
> not, we augment the search to look in /usr/ccs, /opt/sunWPro, and other more
> obscure locations. I believe this is in accord with Martin suggestion.

Indeed it is - although I'd be surprised to find ar in /opt/SUNWpro
(that locations has no ar in my installation). The usual comment for
ld applies (don't ever use it); for cpp, cc -E should work just as
well.

Regards,
Martin



From martin@v.loewis.de  Sun Oct  6 22:36:12 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Oct 2002 23:36:12 +0200
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <003001c26d7e$3ce5df80$890ca044@oemcomputer>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
 <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
 <003001c26d7e$3ce5df80$890ca044@oemcomputer>
Message-ID: <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> The last one includes a list of modules that still need
> backport attention (by someone more knowledgable than me):

Do those need to happen for 2.2.2, or would it be acceptable if
whatever problems they fix get only fixed in 2.2.3?

I think it is of high importance that 2.2.2 is released soon; anything
that delays 2.2.2 must be of really high importance.

Regards,
Martin



From Raymond Hettinger" <python@rcn.com  Sun Oct  6 22:43:03 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Sun, 6 Oct 2002 17:43:03 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com><200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net><003001c26d7e$3ce5df80$890ca044@oemcomputer> <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de>
Message-ID: <007401c26d81$59f914e0$890ca044@oemcomputer>

> "Raymond Hettinger" <raymond.hettinger@verizon.net> writes:
> 
> > The last one includes a list of modules that still need
> > backport attention (by someone more knowledgable than me):
> 
> Do those need to happen for 2.2.2, or would it be acceptable if
> whatever problems they fix get only fixed in 2.2.3?
> 
> I think it is of high importance that 2.2.2 is released soon; anything
> that delays 2.2.2 must be of really high importance.

Yes, this is a list for 2.2.2 based on my review of the checkin logs.
If I felt comfortable applying these myself, they would already
be done.  There are about 8 or 9 total changes, each of them
are small (a few lines in a single file), and can be done in an
hour by someone who knows what they're doing.

If only one or two gets looked at, pickle.py and socket.py 
have the most important unapplied patches.


Raymond Hettinger





From guido@python.org  Sun Oct  6 23:13:14 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 18:13:14 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Sun, 06 Oct 2002 17:43:03 EDT."
 <007401c26d81$59f914e0$890ca044@oemcomputer>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com> <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net> <003001c26d7e$3ce5df80$890ca044@oemcomputer> <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de>
 <007401c26d81$59f914e0$890ca044@oemcomputer>
Message-ID: <200210062213.g96MDER30213@pcp02138704pcs.reston01.va.comcast.net>

> Yes, this is a list for 2.2.2 based on my review of the checkin logs.
> If I felt comfortable applying these myself, they would already
> be done.  There are about 8 or 9 total changes, each of them
> are small (a few lines in a single file), and can be done in an
> hour by someone who knows what they're doing.
> 
> If only one or two gets looked at, pickle.py and socket.py 
> have the most important unapplied patches.

Can you mail a list of which checkins specifically you are looking at
for those files that you want to be reviewed?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Sun Oct  6 23:15:07 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 07 Oct 2002 11:15:07 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <15773.1109.273430.632460@localhost.localdomain>
Message-ID: <200210062215.g96MF7F19321@oma.cosc.canterbury.ac.nz>

    Greg> Maybe if ":" were a sliceobject-creating operator usable anwyere,
    Greg> not just in [...], and you added arithmetic methods to
    Greg> sliceobjects so you could use them as rationals...

Skip> Does x[1:4] return a slice out of x or x[rational(1,4)]?

If rationals and slice objects were the same thing,
it wouldn't matter!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Sun Oct  6 23:18:55 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 06 Oct 2002 18:18:55 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
Message-ID: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>

The file pep_2777.py uses an encoding cookie that specifies UTF-8.
Unfortunately my toolchain doesn't know about this, and displays it as
Latin-1.  Since the only UTF-8 is in 8-bit string literals (not
Unicode literals), wouldn't it make more sense to drop the encoding
cookie and use \xXX escapes in those literals?  I'm not even sure this
use is legal in phase 2 of PEP 263.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Raymond Hettinger" <python@rcn.com  Sun Oct  6 23:29:20 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Sun, 6 Oct 2002 18:29:20 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com> <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net> <003001c26d7e$3ce5df80$890ca044@oemcomputer> <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de>              <007401c26d81$59f914e0$890ca044@oemcomputer>  <200210062213.g96MDER30213@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <00e701c26d87$d0f317c0$890ca044@oemcomputer>

[GvR]
> Can you mail a list of which checkins specifically you are looking at
> for those files that you want to be reviewed?


== Socket.py ==

Revision 1.35 :On Windows, make sure SocketType is the same as socket.  (SF bug
598097)

Revision 1.34: Delete junk attributes left behind by _socketobject class construction.

Revision 1.33:  The _socketobject class has no need for a __del__ method: all it did was
to delete the reference to self._sock, and the regular destructor will
do that just fine.  This made some hacks in close() unnecessary.
The _fileobject class still has a __del__ method, because it must flush.

Revision 1.32: OK, one more hack: speed up the case of readline() in unbuffered mode.
This is important IMO because httplib reads the headers this way.


== Pickle.py ==

Revision 1.70:  whichmodule() should skip dummy package entries in sys.modules.
This fixes the charming, but unhelpful error message for
>>> pickle.dumps(type.__new__)
Can't pickle <built-in method __new__ of type object at 0x812a440>: it's not the same object as datetime.math.__new__
Bugfix candidate.

Revision 1.69:  Patch #505705: Remove eval in pickle and cPickle.


== pydoc.py ==

Revision 1.63 :In both spilldata() functions, pretend that the docstring for
non-callable objects is always None.  This makes for less confusing
output and fixes the problem reported in SF patch #550290.

Revision 1.68: Extend stripid() to handle strings ending in more than one '>'.
Add resolve() to handle looking up objects and names (fix SF bug 586931).
Add a nicer error message when given a filename that doesn't exist.


== ref5.tex ==

Revision 1.63: Simplify the production for argument list, making sure that it
actually allows all the legal syntax, and nothing else.  Previously,
it did not allow a call like func(arg, **dictionary).
This closes (again!) SF bug #493243.

Revision 1.62: Played contortionist games with the argument_list production so it
might be easier to understand.
This relates to SF bug #493243, which will be closed.







From mhammond@skippinet.com.au  Mon Oct  7 00:39:17 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Mon, 7 Oct 2002 09:39:17 +1000
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
Message-ID: <LCEPIIGDJPKCOIHOBJEPKEBDGMAA.mhammond@skippinet.com.au>

I'm a little confused by the new world order for working with integers in
extension modules.

At the end of the day, my question is this:

Assume my extension module has an unsigned integer it wishes to return to
Python.  Further, assume that this unsigned integer is not really an integer
as such, but more a set of bits, or some other value "encoded" in 32 bits,
such as an enum. (To put it another way, the "signedness" of this value
seems more random than chosen)

How should I create the object to return to Python?

For a concrete example.  Let's say I want to return the value of the Win32
function GetVersion().

The documentation for this function declares it is an unsigned 32 bit value.
The documentation then explains that to decode this value, specific bits in
the value should be examined.  It then expounds on this with C sample code
that relies on this unsigned behaviour by using a simple "> 0x80000000"
comparison to check the high bit!

I see 2 choices for returning this value:

* Use PyInt_FromLong() - this will give me a *signed* Python integer, but
with an identical bit pattern.

* Use PyLong_FromUnsignedLong() - this function will correctly be signed,
but may no longer fit in 32 bits.

Now, I think I am trying to stay too close to the hardware for a language
like Python, but something just seems wrong with promoting my nice 32 bit
value to a Python long, simply for the sake of retaining the sign for a
value that the whole concept of "signed" doesn't make much sense (as it
doesn't in this case, or in the case of enums etc).

Any suggestions or general advice?  While this case seems quite trivial, I
am starting to face this issue more and more, especially as I am seeing
these lovely "FutureWarnings" from all my lovely 32 bit hexadecimal
constants <wink/frown>

Thanks,

Mark.



From thomas@xs4all.net  Mon Oct  7 02:01:02 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 7 Oct 2002 03:01:02 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPKEBDGMAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPKEBDGMAA.mhammond@skippinet.com.au>
Message-ID: <20021007010102.GC5116@xs4all.nl>

On Mon, Oct 07, 2002 at 09:39:17AM +1000, Mark Hammond wrote:

> Assume my extension module has an unsigned integer it wishes to return to
> Python.  Further, assume that this unsigned integer is not really an integer
> as such, but more a set of bits, or some other value "encoded" in 32 bits,
> such as an enum. (To put it another way, the "signedness" of this value
> seems more random than chosen)

> I see 2 choices for returning this value:

> * Use PyInt_FromLong() - this will give me a *signed* Python integer, but
> with an identical bit pattern.

If you intend to hand out the bitvector for informational purposes, and
aren't too worried (because of your excellent documentational powers) about
users messing up the bitvector in incorrect calculations before passing it
back, or about them being dazzled by the concept of bitvectors as a whole,
go for it. I'm personally still in that phase where I really like ANDing and
ORing, even in Python :)

> * Use PyLong_FromUnsignedLong() - this function will correctly be signed,
> but may no longer fit in 32 bits.

You mean on the way back, I presume ?

> Now, I think I am trying to stay too close to the hardware for a language
> like Python,

Yes, I think you are, actually. Converting those cute little 32 bits into a
real Python object representing it more verbosely, so to speak, might seem a
waste, but makes it a lot more user-friendly. It doesn't have to be a type
on its own, it can be a tuple of strings or ints, depending on what kind of
info you want to return.

If it's really a plain bitvector, you can also consider returning a string.
A hobby project of mine, a type of MUD, converts internal bitvectors to
strings where each letter from 'a' to 'F' represents a bit in the bitvector.
So 0x8a would come out as "bdh". This is very suited to the project itself,
since the internal language (MUSHcode, an ass-backwards retarded
vaguely-lisplike death-trap of a recursively-parsed scripting language) has
no bitwise operators, and the typical MUSHcoder has trouble grasping bitwise
arithmatic anyway. Not that MUSHcode has an efficient way to say
'"a" in "abc"', mind you, but they manage to do it in less than 5
functioncalls. (And yes, I am trying to replace MUSHcode with Python.)

I should point out that neither the bitvector-to-string code (and its
reverse counterpart) nor the braindead internal language are mine :) I just
inherited the project. But the bitvector routines are simple, I'm sure
you'll have no trouble figuring them out ;) When using them in Python code
you might want to add the flag-characters as constants in your module, so
the test becomes:

  if ce.hasdisk in ce.systemflags():

Insomaniac'ly y'rs,

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From ark@research.att.com  Mon Oct  7 02:32:52 2002
From: ark@research.att.com (Andrew Koenig)
Date: 06 Oct 2002 21:32:52 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210062215.g96MF7F19321@oma.cosc.canterbury.ac.nz>
References: <200210062215.g96MF7F19321@oma.cosc.canterbury.ac.nz>
Message-ID: <yu99fzvjghiz.fsf@europa.research.att.com>

Skip> Does x[1:4] return a slice out of x or x[rational(1,4)]?

Greg> If rationals and slice objects were the same thing,
Greg> it wouldn't matter!

But then how would you distinguish between x[1:4] and x[2:8]?

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From greg@cosc.canterbury.ac.nz  Mon Oct  7 02:39:25 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 07 Oct 2002 14:39:25 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <yu99fzvjghiz.fsf@europa.research.att.com>
Message-ID: <200210070139.g971dPs19874@oma.cosc.canterbury.ac.nz>

Greg> If rationals and slice objects were the same thing,
Greg> it wouldn't matter!

Andrew> But then how would you distinguish between x[1:4] and x[2:8]?

Well, if you reduced the rational to lowest terms you
would have to keep the original values around for when
you were using it as a slice object...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Mon Oct  7 02:43:21 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 06 Oct 2002 21:43:21 -0400
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPKEBDGMAA.mhammond@skippinet.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEJIBJAB.tim.one@comcast.net>

[Mark Hammond]
> I'm a little confused by the new world order for working with integers
> in extension modules.
>
> At the end of the day, my question is this:
>
> Assume my extension module has an unsigned integer it wishes to return
> to Python.  Further, assume that this unsigned integer is not really
> an integer as such, but more a set of bits, or some other value
> "encoded" in 32 bits, such as an enum. (To put it another way, the
> "signedness" of this value seems more random than chosen)

Well, it matters:  a bitset is most naturally thought of as unsigned, so
that shifting doesn't introduce "mystery bits".  OTOH, a C enum is, by
definition, a signed integer.

> How should I create the object to return to Python?

I'd create a Python long.

> For a concrete example.  Let's say I want to return the value of the
> Win32 function GetVersion().
>
> The documentation for this function declares it is an unsigned 32
> bit value.  The documentation then explains that to decode this value,
> specific bits in the value should be examined.  It then expounds on
> this with C sample code that relies on this unsigned behaviour by
> using a simple "> 0x80000000" comparison to check the high bit!

Yup.  The docs also say <wink>:

   This function has been superseded by GetVersionEx, which is the
   preferred method for obtaining system version number information.
   New applications should use GetVersionEx.  The GetVersionEx function
   was developed because many existing applications err when examining
   the DWORD return value of a GetVersion function call, transposing
   the major and minor version numbers packed into that DWORD.

IOW, the bag-of-bits mixed-with bag-of-bytes model was too confusing to work
with.

> I see 2 choices for returning this value:
>
> * Use PyInt_FromLong() - this will give me a *signed* Python integer,
>   but with an identical bit pattern.
>
> * Use PyLong_FromUnsignedLong() - this function will correctly be
>   signed, but may no longer fit in 32 bits.

Python ints don't fit in 32 bits either:  they've got object headers like
all objects have.  The space difference here is trivial.

A third choice is to pick the values apart in C, delivering a

   (NT_or_later_bool, major_version_int, minor_version_int, build_int)

tuple back to the Python user.  Making people pick apart the bits in Python
code seems too low-level here:

dwVersion = GetVersion();

// Get major and minor version numbers of Windows
dwWindowsMajorVersion =  (DWORD)(LOBYTE(LOWORD(dwVersion)));
dwWindowsMinorVersion =  (DWORD)(HIBYTE(LOWORD(dwVersion)));
// Get build numbers for Windows NT or Win32s
if (dwVersion < 0x80000000)                // Windows NT
    dwBuild = (DWORD)(HIWORD(dwVersion));
else if (dwWindowsMajorVersion < 4)        // Win32s
    dwBuild = (DWORD)(HIWORD(dwVersion) & ~0x8000);
else         // Windows 95 -- No build numbers provided
    dwBuild =  0;

> Now, I think I am trying to stay too close to the hardware for a
> language like Python, but something just seems wrong with promoting
> my nice 32 bit value to a Python long, simply for the sake of
> retaining the sign for a value that the whole concept of "signed"
> doesn't make much sense (as it doesn't in this case, or in the case
> of enums etc).

Except Python doesn't *have* unsigned ints, so the only faithful way to
return one is to make a Python long.  In this specific case, though, I think
it would be better to pick the bits apart *for* the user -- there's really
no use for the raw int, signed or unsigned, except after picking it apart.

> Any suggestions or general advice?  While this case seems quite
> trivial, I am starting to face this issue more and more, especially
> as I am seeing these lovely "FutureWarnings" from all my lovely 32
> bit hexadecimal constants <wink/frown>

Sticking "L" at the end is usually all it takes.



From martin@v.loewis.de  Mon Oct  7 03:32:42 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 04:32:42 +0200
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <00e701c26d87$d0f317c0$890ca044@oemcomputer>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
 <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
 <003001c26d7e$3ce5df80$890ca044@oemcomputer>
 <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de>
 <007401c26d81$59f914e0$890ca044@oemcomputer>
 <200210062213.g96MDER30213@pcp02138704pcs.reston01.va.comcast.net>
 <00e701c26d87$d0f317c0$890ca044@oemcomputer>
Message-ID: <m3vg4fvv05.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> Revision 1.69:  Patch #505705: Remove eval in pickle and cPickle.

It is not realistic to backport this patch for 2.2.2. I would suggest
to ignore it completely for 2.2.

Regards,
Martin


From Raymond Hettinger" <python@rcn.com  Mon Oct  7 03:40:23 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Sun, 6 Oct 2002 22:40:23 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com><200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net><003001c26d7e$3ce5df80$890ca044@oemcomputer><m3n0prz1v7.fsf@mira.informatik.hu-berlin.de><007401c26d81$59f914e0$890ca044@oemcomputer><200210062213.g96MDER30213@pcp02138704pcs.reston01.va.comcast.net><00e701c26d87$d0f317c0$890ca044@oemcomputer> <m3vg4fvv05.fsf@mira.informatik.hu-berlin.de>
Message-ID: <00ab01c26daa$e30f64e0$890ca044@oemcomputer>

Martin, thanks for looking at it.

BTW, I'm not a proponent of any of those changes.
They are just ones that looked like viable candidates
but  I didn't think I should handle myself.


Raymond Hettinger



----- Original Message ----- 
From: "Martin v. Loewis" <martin@v.loewis.de>
To: "Raymond Hettinger" <python@rcn.com>
Cc: "Guido van Rossum" <guido@python.org>; <python-dev@python.org>
Sent: Sunday, October 06, 2002 10:32 PM
Subject: Re: [Python-Dev] Bug #537450 Improper object initialization for type.__new__


> "Raymond Hettinger" <raymond.hettinger@verizon.net> writes:
> 
> > Revision 1.69:  Patch #505705: Remove eval in pickle and cPickle.
> 
> It is not realistic to backport this patch for 2.2.2. I would suggest
> to ignore it completely for 2.2.
> 
> Regards,
> Martin


From martin@v.loewis.de  Mon Oct  7 03:41:09 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 04:41:09 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> The file pep_2777.py uses an encoding cookie that specifies UTF-8.
> Unfortunately my toolchain doesn't know about this, and displays it as
> Latin-1.  

What do you mean by "toolchain"? At the end of the chain is python, it
should know about this well enough.

Did you try to open the file in IDLE? What other tools are you using?

> Since the only UTF-8 is in 8-bit string literals (not Unicode
> literals), wouldn't it make more sense to drop the encoding cookie
> and use \xXX escapes in those literals?

Having the original byte strings allows to verify correctness of the
test visually: the files created should look the same in the operating
system (using ls, or a file explorer) as they do in the source code.

> I'm not even sure this use is legal in phase 2 of PEP 263.

It is: the file has a decoding declared, and decodes properly under
that decoding. String literals are preserved in their original
encoding, so they will come out at run-time the same as they are on
disk.

Regards,
Martin


From jafo@tummy.com  Mon Oct  7 05:17:06 2002
From: jafo@tummy.com (Sean Reifschneider)
Date: Sun, 6 Oct 2002 22:17:06 -0600
Subject: [Python-Dev] Patch to fix Distutils bdist_rpm on Red Hat 8.0
Message-ID: <20021007041706.GA11457@tummy.com>

At the 8.0 release, Red Hat has split the "rpm" command such that the
build options reside only in the "rpmbuild" command.  Actually, this
happened a while ago, as "rpmbuild" was available on 7.3.  However, at
8.0 the "rpm" command would no longer do building and it fails out.

I've submitted patch #619493 to correct this.  If /usr/bin/rpmbuild or
/bin/rpmbuild exists, the rpm command is changed from "rpm" to
"rpmbuild".  This probably should go into the 2.2.2 release.

I'm submitting a bug to Red Hat as well, as they probably should release
a fixed python package to address this as well.

Sean
-- 
 "The phrase ``ship it!'' is one with long and deep resonances in my
 benighted and antiquated big-software-dept career." -- John Shipman, 1998
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com, ltd. - Linux Consulting since 1995.  Qmail, Python, SysAdmin


From tim@multitalents.net  Mon Oct  7 05:54:44 2002
From: tim@multitalents.net (Tim Rice)
Date: Sun, 6 Oct 2002 21:54:44 -0700 (PDT)
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.UW2.4.44.0210062151210.13986-100000@ou8.int.multitalents.net>

On Sun, 6 Oct 2002, Guido van Rossum wrote:

> [Anders Qvist]
> > Several machines in the snake farm fail to build libpython2.3 because
> > they don't have ar in their path. ar, ld and the like live in
> > /usr/ccs/bin, which seems to be some kind of standard (for generous
> > values of standard) as both SunOS 5.8, AIX 4.3 and HP/UX 11.00 seems
> > ot have this dir.
> >
> > Should the configure script try with /usr/ccs/bin/ar and ld if none is
> > explicitly supplied?
>

How about?
AC_PATH_PROG(AR, ar,[AC_MSG_ERROR(Can't find ar in your path)])


> This may be a good idea, but I don't know zit about this area.  I'm
> cc'ing python-dev, maybeo someone there understands this.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net




From mal@lemburg.com  Mon Oct  7 08:56:12 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 09:56:12 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DA13E1C.5080505@lemburg.com>

Guido van Rossum wrote:
> The file pep_2777.py uses an encoding cookie that specifies UTF-8.
> Unfortunately my toolchain doesn't know about this, and displays it as
> Latin-1.  Since the only UTF-8 is in 8-bit string literals (not
> Unicode literals), wouldn't it make more sense to drop the encoding
> cookie and use \xXX escapes in those literals? 

You could, but the whole point of PEP 263 is that the source code
encoding is made explicit, so removing the cookie would defeat the
purpose. Note that the cookie also implicitly tests the PEP 263
implementation.

> I'm not even sure this
> use is legal in phase 2 of PEP 263.

It is: if you use non-UTF-8 string literals in that source file,
the parser will raise an exception during compilation.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From vinay_sajip@red-dove.com  Mon Oct  7 09:16:29 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Mon, 7 Oct 2002 09:16:29 +0100
Subject: [Python-Dev] PEP 282 Implementation
References: <00e001c2261d$19bfc320$652b6992@alpha> <200208092040.g79Ke3S31416@pcp02138704pcs.reston01.va.comcast.net> <006601c24e7f$fcff5440$652b6992@alpha> <200209231741.g8NHfEl11092@pcp02138704pcs.reston01.va.comcast.net> <01aa01c26379$8d6def60$4901000a@dorothy>              <003901c2639f$e7e21ae0$652b6992@alpha>  <200209241537.g8OFbnc13228@odiug.zope.com>
Message-ID: <002201c26dd9$d7c46b60$652b6992@alpha>

Guido van Rossum wrote:
>> Chris McDonough wrote:
>>> It would be helpful for the FileHandler class to define a method
>>> which just closes and reopens the current logfile (instead of
>>> actually rotating a set like-named logfiles).  This would allow
>>> logfile rotation to be performed by a separate process (e.g.
>>> RedHat's logrotate).  Sometimes it's better (and even necessary) to
>>> be able to use system-provided log rotation facilities instead of
>>> relying on the native rotation facilities.
> Having looked at it again, I think that this is definitely better than
> doing log rotation in the FileHandler.  The rotation code in the log
> handler currently calls tell() after each record is emitted.  This is
> expensive, and not needed if you use an external process to watch over
> the log files and rotate them.

I think file rotation can definitely come out of the core, since there are
many ways to do it and no one "right" way. I'll move the rotation stuff into
a subclass of FileHandler called RotatingFileHandler (anyone think of a
better name?). The FileHandler will remain in the core module; the subclass
will be in the "logging.handlers" module.

Vinay



From mwh@python.net  Mon Oct  7 10:30:49 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Oct 2002 10:30:49 +0100
Subject: [Python-Dev] Python 2.2.2 beta release on Monday!
In-Reply-To: Guido van Rossum's message of "Sat, 05 Oct 2002 03:18:47 -0400"
References: <200210050718.g957IlF01399@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <2m65wesiie.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> I plan to do a Python 2.2.2 beta release this Monday (if Fred's and
> Tim's time is available to do their part).
> 
> Please finish your changes to 2.2.2 before the weekend is over!

Timezones to the rescue... there are definitely a few things I want to
do before a beta release.

there-goes-the-coffee-break-ly y'rs
M.

-- 
  In case you're not a computer person, I should probably point out
  that "Real Soon Now" is a technical term meaning "sometime before
  the heat-death of the universe, maybe".
                                     -- Scott Fahlman <sef@cs.cmu.edu>


From thomas.heller@ion-tof.com  Mon Oct  7 11:07:11 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 7 Oct 2002 12:07:11 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
References: <LNBBLJKPBEHFEDALKOLCGEJIBJAB.tim.one@comcast.net>
Message-ID: <00ca01c26de9$4dbcedb0$e000a8c0@thomasnotebook>

> Except Python doesn't *have* unsigned ints, so the only faithful way to
> return one is to make a Python long.  In this specific case, though, I think
> it would be better to pick the bits apart *for* the user -- there's really
> no use for the raw int, signed or unsigned, except after picking it apart.
> 
> > Any suggestions or general advice?  While this case seems quite
> > trivial, I am starting to face this issue more and more, especially
> > as I am seeing these lovely "FutureWarnings" from all my lovely 32
> > bit hexadecimal constants <wink/frown>
> 
[Tim]
> Sticking "L" at the end is usually all it takes.

That removes the warnings for 'x = 0x80000000L'.
Is there a way (other than the -w command-line arg)
to suppress the warnings when doing 'hex(-1)'?
Shouldn't there be a __future__ option?

Thomas


From guido@python.org  Mon Oct  7 13:20:27 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 08:20:27 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: Your message of "Mon, 07 Oct 2002 04:41:09 +0200."
 <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net>

> > The file pep_2777.py uses an encoding cookie that specifies UTF-8.
> > Unfortunately my toolchain doesn't know about this, and displays it as
> > Latin-1.  
> 
> What do you mean by "toolchain"? At the end of the chain is python, it
> should know about this well enough.
> 
> Did you try to open the file in IDLE? What other tools are you using?

XEmacs.  The particular version or configuration I'm using apparently
doesn't recogize the coding cookie.

> > Since the only UTF-8 is in 8-bit string literals (not Unicode
> > literals), wouldn't it make more sense to drop the encoding cookie
> > and use \xXX escapes in those literals?
> 
> Having the original byte strings allows to verify correctness of the
> test visually: the files created should look the same in the operating
> system (using ls, or a file explorer) as they do in the source code.

You could do this with a comment.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct  7 13:28:24 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 08:28:24 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: Your message of "Mon, 07 Oct 2002 09:56:12 +0200."
 <3DA13E1C.5080505@lemburg.com>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
Message-ID: <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>

> You could, but the whole point of PEP 263 is that the source code
> encoding is made explicit, so removing the cookie would defeat the
> purpose. Note that the cookie also implicitly tests the PEP 263
> implementation.

I'm still not 100% comfortable with using arbitrary coding cookies in
the Python distribution.  I like having the feature, but I would
prefer if the Python source distribution could be viewed with tools
that aren't encoding-aware.  On my Red Hat 7.2 Linux box, I have found
no programs that understand the UTF8 cookie.  They all seem to default
to Latin-1.  Since I also don't really want to assume Latin-1, I'd
prefer if we could stick to ASCII, with the exception of tests
explicitly needed for PEP 263.  I don't want to make test_pep277.py an
implicit PEP 263 test.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Oct  7 14:02:10 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 15:02:10 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>              <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DA185D2.3010100@lemburg.com>

Guido van Rossum wrote:
>>You could, but the whole point of PEP 263 is that the source code
>>encoding is made explicit, so removing the cookie would defeat the
>>purpose. Note that the cookie also implicitly tests the PEP 263
>>implementation.
> 
> 
> I'm still not 100% comfortable with using arbitrary coding cookies in
> the Python distribution.  I like having the feature, but I would
> prefer if the Python source distribution could be viewed with tools
> that aren't encoding-aware.  On my Red Hat 7.2 Linux box, I have found
> no programs that understand the UTF8 cookie. 

I just checked my SuSE 8.0 XEmacs and it seems that it doesn't
come with the UTF-8 coding system installed either.

> They all seem to default
> to Latin-1.  Since I also don't really want to assume Latin-1, I'd
> prefer if we could stick to ASCII, with the exception of tests
> explicitly needed for PEP 263.  I don't want to make test_pep277.py an
> implicit PEP 263 test.

Fine with me. I guess we ought to document this somewhere in the
Python core programming style guide (if such a document exists ;-).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Mon Oct  7 14:23:12 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 09:23:12 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Sun, 06 Oct 2002 18:29:20 EDT."
 <00e701c26d87$d0f317c0$890ca044@oemcomputer>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com> <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net> <003001c26d7e$3ce5df80$890ca044@oemcomputer> <m3n0prz1v7.fsf@mira.informatik.hu-berlin.de> <007401c26d81$59f914e0$890ca044@oemcomputer> <200210062213.g96MDER30213@pcp02138704pcs.reston01.va.comcast.net>
 <00e701c26d87$d0f317c0$890ca044@oemcomputer>
Message-ID: <200210071323.g97DNCH08002@pcp02138704pcs.reston01.va.comcast.net>

> == Socket.py ==
> 
> Revision 1.35 :On Windows, make sure SocketType is the same as socket.  (SF bug
> 598097)
> 
> Revision 1.34: Delete junk attributes left behind by _socketobject class construction.
> 
> Revision 1.33:  The _socketobject class has no need for a __del__ method: all it did was
> to delete the reference to self._sock, and the regular destructor will
> do that just fine.  This made some hacks in close() unnecessary.
> The _fileobject class still has a __del__ method, because it must flush.
> 
> Revision 1.32: OK, one more hack: speed up the case of readline() in unbuffered mode.
> This is important IMO because httplib reads the headers this way.

Most of these build upon major restructuring of socket.py that I did
for 2.3.  Backporting would be too much of a hassle, so I'll skip them
all.


> == Pickle.py ==
> 
> Revision 1.70:  whichmodule() should skip dummy package entries in sys.modules.
> This fixes the charming, but unhelpful error message for
> >>> pickle.dumps(type.__new__)
> Can't pickle <built-in method __new__ of type object at 0x812a440>: it's not the same object as datetime.math.__new__
> Bugfix candidate.

Fixed (together with 1.71 which was a cleanup of 1.70).

> Revision 1.69:  Patch #505705: Remove eval in pickle and cPickle.

MvL already said this is a no-go; it depends on a new encoding that
knows about string literal escape sequences.


> == pydoc.py ==
> 
> Revision 1.63 :In both spilldata() functions, pretend that the docstring for
> non-callable objects is always None.  This makes for less confusing
> output and fixes the problem reported in SF patch #550290.

This applies cleanly so I'll check it in.

> Revision 1.68: Extend stripid() to handle strings ending in more than one '>'.
> Add resolve() to handle looking up objects and names (fix SF bug 586931).
> Add a nicer error message when given a filename that doesn't exist.

Looks messy, I'm not an expert on pydoc.  Ping, should this be
backported?



> == ref5.tex ==
> 
> Revision 1.63: Simplify the production for argument list, making sure that it
> actually allows all the legal syntax, and nothing else.  Previously,
> it did not allow a call like func(arg, **dictionary).
> This closes (again!) SF bug #493243.
> 
> Revision 1.62: Played contortionist games with the argument_list production so it
> might be easier to understand.
> This relates to SF bug #493243, which will be closed.

I've forwarded these two to Fred Drake.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Mon Oct  7 14:25:42 2002
From: aahz@pythoncraft.com (Aahz)
Date: Mon, 7 Oct 2002 09:25:42 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <3DA185D2.3010100@lemburg.com>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com>
Message-ID: <20021007132542.GA25844@panix.com>

On Mon, Oct 07, 2002, M.-A. Lemburg wrote:
> Guido van Rossum wrote:
>>
>>They all seem to default
>>to Latin-1.  Since I also don't really want to assume Latin-1, I'd
>>prefer if we could stick to ASCII, with the exception of tests
>>explicitly needed for PEP 263.  I don't want to make test_pep277.py an
>>implicit PEP 263 test.
> 
> Fine with me. I guess we ought to document this somewhere in the
> Python core programming style guide (if such a document exists ;-).

PEP 8
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/


From list-python@ccraig.org  Mon Oct  7 14:28:28 2002
From: list-python@ccraig.org (Christopher A. Craig)
Date: 07 Oct 2002 09:28:28 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <000001c26d53$64253b70$6701a8c0@NICKLEBY>
References: <000001c26d53$64253b70$6701a8c0@NICKLEBY>
Message-ID: <whlm5ajs3n.fsf@olorin.ts.gatech.edu>

"Paul F Dubois" <paul@pfdubois.com> writes:

> I haven't followed the rational discussion but obviously there is no
> other long-standing and highly used notation for rationals except
> 1.0/2.0, is there?  

Kirby Urner suggested 1r2 as a way of expressing rationals, which I
rather liked, but no I don't know of a traditional notation for
rationals.

-- 
Christopher A. Craig <list-python@ccraig.org>
"Bah!  Puny, weak newsreaders!"- GNUS info, in reference to 'global' kills


From guido@python.org  Mon Oct  7 14:41:02 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 09:41:02 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: Your message of "Mon, 07 Oct 2002 15:02:10 +0200."
 <3DA185D2.3010100@lemburg.com>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
Message-ID: <200210071341.g97Df2R08168@pcp02138704pcs.reston01.va.comcast.net>

> > I'm still not 100% comfortable with using arbitrary coding cookies in
> > the Python distribution.  I like having the feature, but I would
> > prefer if the Python source distribution could be viewed with tools
> > that aren't encoding-aware.  On my Red Hat 7.2 Linux box, I have found
> > no programs that understand the UTF8 cookie. 
> 
> I just checked my SuSE 8.0 XEmacs and it seems that it doesn't
> come with the UTF-8 coding system installed either.
> 
> > They all seem to default
> > to Latin-1.  Since I also don't really want to assume Latin-1, I'd
> > prefer if we could stick to ASCII, with the exception of tests
> > explicitly needed for PEP 263.  I don't want to make test_pep277.py an
> > implicit PEP 263 test.
> 
> Fine with me. I guess we ought to document this somewhere in the
> Python core programming style guide (if such a document exists ;-).

I've added this to PEP 8.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ark@research.att.com  Mon Oct  7 14:47:45 2002
From: ark@research.att.com (Andrew Koenig)
Date: 07 Oct 2002 09:47:45 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210070139.g971dPs19874@oma.cosc.canterbury.ac.nz>
References: <200210070139.g971dPs19874@oma.cosc.canterbury.ac.nz>
Message-ID: <yu99d6qmxsvy.fsf@europa.research.att.com>

Greg> If rationals and slice objects were the same thing,
Greg> it wouldn't matter!

Andrew> But then how would you distinguish between x[1:4] and x[2:8]?

Greg> Well, if you reduced the rational to lowest terms you
Greg> would have to keep the original values around for when
Greg> you were using it as a slice object...

In that case, wouldn't you have a situation where a==b, a and b are
the same type, but x[a]!=x[b]?

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From barry@python.org  Mon Oct  7 14:59:15 2002
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 7 Oct 2002 09:59:15 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
Message-ID: <15777.37683.501642.428503@gargle.gargle.HOWL>

>>>>> "MAL" == M  <mal@lemburg.com> writes:

    MAL> I just checked my SuSE 8.0 XEmacs and it seems that it
    MAL> doesn't come with the UTF-8 coding system installed either.

Was that XEmacs built with MULE, and does that matter?  (I'm
struggling with utf-8 issues on the mailman-i18n lists, trying to
decode and display an RFC 2047 header encoded with utf-8.  Doesn't
seem possible).  I just visited the file in my MULE XEmacs 21.4.9 and
it doesn't seem to grok the utf-8 encoding either.

    MAL> Fine with me. I guess we ought to document this somewhere in
    MAL> the Python core programming style guide (if such a document
    MAL> exists ;-).

PEP 8

-Barry


From sjoerd@acm.org  Mon Oct  7 15:20:45 2002
From: sjoerd@acm.org (Sjoerd Mullender)
Date: Mon, 07 Oct 2002 16:20:45 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <15777.37683.501642.428503@gargle.gargle.HOWL>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
Message-ID: <200210071420.g97EKjw19204@indus.ins.cwi.nl>

Have you seen
<http://www.xemacs.org/Documentation/packages/html/mule-ucs_2.html>?
I don't know whether it helps in your case, but it does tell you how
to get UTF-8 support in XEmacs.

On Mon, Oct 7 2002 Barry A. Warsaw wrote:

> 
> >>>>> "MAL" == M  <mal@lemburg.com> writes:
> 
>     MAL> I just checked my SuSE 8.0 XEmacs and it seems that it
>     MAL> doesn't come with the UTF-8 coding system installed either.
> 
> Was that XEmacs built with MULE, and does that matter?  (I'm
> struggling with utf-8 issues on the mailman-i18n lists, trying to
> decode and display an RFC 2047 header encoded with utf-8.  Doesn't
> seem possible).  I just visited the file in my MULE XEmacs 21.4.9 and
> it doesn't seem to grok the utf-8 encoding either.
> 
>     MAL> Fine with me. I guess we ought to document this somewhere in
>     MAL> the Python core programming style guide (if such a document
>     MAL> exists ;-).
> 
> PEP 8
> 
> -Barry
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 

-- Sjoerd Mullender <sjoerd@acm.org>


From mwh@python.net  Mon Oct  7 15:25:01 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Oct 2002 15:25:01 +0100
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL>
Message-ID: <2m8z1az5qa.fsf@starship.python.net>

barry@python.org (Barry A. Warsaw) writes:

> Was that XEmacs built with MULE, and does that matter?  (I'm
> struggling with utf-8 issues on the mailman-i18n lists, trying to
> decode and display an RFC 2047 header encoded with utf-8.  Doesn't
> seem possible).  I just visited the file in my MULE XEmacs 21.4.9 and
> it doesn't seem to grok the utf-8 encoding either.

GNU Emacs 21 handles it fine (once you find a font that has all the
characters).  But since the coding: header idea was filched from a GNU
Emacs manual, I guess that's hardly surprising...

Cheers,
M.

-- 
 "Well, the old ones go Mmmmmbbbbzzzzttteeeeeep as they start up and
  the new ones go whupwhupwhupwhooopwhooooopwhooooooommmmmmmmmm."
                         -- Graham Reed explains subway engines on asr


From tim.one@comcast.net  Mon Oct  7 15:45:53 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 10:45:53 -0400
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <00ca01c26de9$4dbcedb0$e000a8c0@thomasnotebook>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>

[Mark Hammond]
> While this case seems quite trivial, I am starting to face this issue
> more and more, especially as I am seeing these lovely "FutureWarnings"
> from all my lovely 32 bit hexadecimal constants <wink/frown>

[Tim]
> Sticking "L" at the end is usually all it takes.

[Thomas Heller]
> That removes the warnings for 'x = 0x80000000L'.

What else do you think Mark may have meant by "32 bit hexadecimal
constants"?

> Is there a way (other than the -w command-line arg)
> to suppress the warnings when doing 'hex(-1)'?

The result of that is also currently platform-dependent, so you need to say
what you mean in a platform-independent way.  It's likely that you meant

    hex(-1 & 0xffffffffL)

or, more directly,

    hex(0xffffffffL)

but that someone on a 64-bit box meant something other than that.

> Shouldn't there be a __future__ option?

You seem to want most of all to avoid warning msgs, but __future__ options
don't accomplish that:  it's the *point* of __future__ thingies to spit out
warnings about semantics that are going to change.

I don't know that it's a useful point, though.  That is, the only comments
I've seen about __future__ from users in practice is from those who are
annoyed by needing to say "__future__" all over the place to get at the new
features they want right away.  Alas, I haven't seen evidence that it eases
migration.  That could be because it works so well people don't feel a need
to comment about it, but somehow that seems a tad unlikely <wink>.



From martin@v.loewis.de  Mon Oct  7 15:58:42 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 16:58:42 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m34rbyb8il.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> I'm still not 100% comfortable with using arbitrary coding cookies in
> the Python distribution.  I like having the feature, but I would
> prefer if the Python source distribution could be viewed with tools
> that aren't encoding-aware.  

Does that mean you also want to remove test_pep263.py?

> On my Red Hat 7.2 Linux box, I have found no programs that
> understand the UTF8 cookie.

Try IDLE :-)

> They all seem to default to Latin-1.  Since I also don't really want
> to assume Latin-1, I'd prefer if we could stick to ASCII, with the
> exception of tests explicitly needed for PEP 263.

There are a number of files that have Latin-1 in them, mostly in
comments. Should we change those as well?

Regards,
Martin



From martin@v.loewis.de  Mon Oct  7 16:02:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 17:02:46 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
Message-ID: <m3y99a9trd.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> The result of that is also currently platform-dependent, so you need to say
> what you mean in a platform-independent way.  It's likely that you meant
> 
>     hex(-1 & 0xffffffffL)
> 
> or, more directly,
> 
>     hex(0xffffffffL)
> 
> but that someone on a 64-bit box meant something other than that.

Alternatively, if he really means what hex(-1) will mean in the
future, he should really write hex(-1L), or, if -1 is a variable,
hex(long(-1)).

Regards,
Martin


From mal@lemburg.com  Mon Oct  7 16:06:56 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 17:06:56 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>	<3DA13E1C.5080505@lemburg.com>	<200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>	<3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL>
Message-ID: <3DA1A310.9030606@lemburg.com>

Barry A. Warsaw wrote:
>>>>>>"MAL" == M  <mal@lemburg.com> writes:
> 
> 
>     MAL> I just checked my SuSE 8.0 XEmacs and it seems that it
>     MAL> doesn't come with the UTF-8 coding system installed either.
> 
> Was that XEmacs built with MULE, and does that matter?  (I'm
> struggling with utf-8 issues on the mailman-i18n lists, trying to
> decode and display an RFC 2047 header encoded with utf-8.  Doesn't
> seem possible).  I just visited the file in my MULE XEmacs 21.4.9 and
> it doesn't seem to grok the utf-8 encoding either.

XEmacs says:

XEmacs 21.4 (patch 4) "Artificial Intelligence" (linux, Mule)
of Mon Mar 25 2002 on amdsim3

So I guess, everything is in place. However the instruction
posted by Sjoerd don't work for some reason.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From barry@python.org  Mon Oct  7 16:20:08 2002
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 7 Oct 2002 11:20:08 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
Message-ID: <15777.42536.817787.472626@gargle.gargle.HOWL>

>>>>> "MAL" == M  <mal@lemburg.com> writes:

    MAL> So I guess, everything is in place. However the instruction
    MAL> posted by Sjoerd don't work for some reason.

I couldn't get them to work either.
-Barry


From sjoerd@acm.org  Mon Oct  7 16:40:08 2002
From: sjoerd@acm.org (Sjoerd Mullender)
Date: Mon, 07 Oct 2002 17:40:08 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <15777.42536.817787.472626@gargle.gargle.HOWL>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL> <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
Message-ID: <200210071540.g97Fe9822116@indus.ins.cwi.nl>

When I execute the three lisp form that were mentioned in the page I
pointed you to*, my XEmacs does recognize utf-8 encoding automatically.
I just find-file a file which contains UTF-8 encoded data and I see
all sorts of accented characters (the ones I expect to see even!).
My XEmacs version is

XEmacs 21.4 (patch 6) "Common Lisp" [Lucid] (i386-redhat-linux, Mule)
of Fri Feb 22 2002 on daffy.perf.redhat.com

*
(require 'un-define)
(set-coding-priority-list '(utf-8))
(set-coding-category-system 'utf-8 'utf-8)

On Mon, Oct 7 2002 Barry A. Warsaw wrote:

> 
> >>>>> "MAL" == M  <mal@lemburg.com> writes:
> 
>     MAL> So I guess, everything is in place. However the instruction
>     MAL> posted by Sjoerd don't work for some reason.
> 
I couldn't get them to work either.
> -Barry
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 

-- Sjoerd Mullender <sjoerd@acm.org>


From martin@v.loewis.de  Mon Oct  7 17:03:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 18:03:41 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3hefy9qxu.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> On my Red Hat 7.2 Linux box, I have found no programs that
> understand the UTF8 cookie.

It appears that atleast kate (from KDE) can edit UTF-8 files. It won't
recognize cookies, but you can specify the encoding when opening the
file.

Regards,
Martin


From martin@v.loewis.de  Mon Oct  7 17:09:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 18:09:40 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071540.g97Fe9822116@indus.ins.cwi.nl>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
 <200210071540.g97Fe9822116@indus.ins.cwi.nl>
Message-ID: <m3d6qm9qnv.fsf@mira.informatik.hu-berlin.de>

Sjoerd Mullender <sjoerd@acm.org> writes:

> When I execute the three lisp form that were mentioned in the page I
> pointed you to*, my XEmacs does recognize utf-8 encoding automatically.
> I just find-file a file which contains UTF-8 encoded data and I see
> all sorts of accented characters (the ones I expect to see even!).
[...]
> (set-coding-priority-list '(utf-8))
> (set-coding-category-system 'utf-8 'utf-8)

That's a slightly different issue: the question is whether it will
recognize the coding of test_pep277.py even without specifying a
coding priority list.

Setting the priority list to utf-8 is a good thing as long is it does
not exist to save all new files in UTF-8, then.

Regards,
Martin



From barry@python.org  Mon Oct  7 17:39:49 2002
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 7 Oct 2002 12:39:49 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
 <200210071540.g97Fe9822116@indus.ins.cwi.nl>
Message-ID: <15777.47317.362113.638027@gargle.gargle.HOWL>

>>>>> "SM" == Sjoerd Mullender <sjoerd@acm.org> writes:

    SM> When I execute the three lisp form that were mentioned in the
    SM> page I pointed you to*, my XEmacs does recognize utf-8
    SM> encoding automatically.

Ah, I had to restart XEmacs.  I added those lines to my .emacs file,
restart, and visited test_pep277.py, and sure enough I see (the right)
funny characters too. :)

Thanks.  Now I wonder if I can get utf-8 encoded RFC 2047 headers to
work in email.

-Barry


From guido@python.org  Mon Oct  7 17:43:56 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 12:43:56 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: Your message of "Mon, 07 Oct 2002 16:58:42 +0200."
 <m34rbyb8il.fsf@mira.informatik.hu-berlin.de>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <m34rbyb8il.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210071643.g97Ghuo16131@pcp02138704pcs.reston01.va.comcast.net>

> > I'm still not 100% comfortable with using arbitrary coding cookies in
> > the Python distribution.  I like having the feature, but I would
> > prefer if the Python source distribution could be viewed with tools
> > that aren't encoding-aware.  
> 
> Does that mean you also want to remove test_pep263.py?

No, see what I added to PEP 8. :-)

> > On my Red Hat 7.2 Linux box, I have found no programs that
> > understand the UTF8 cookie.
> 
> Try IDLE :-)
> 
> > They all seem to default to Latin-1.  Since I also don't really want
> > to assume Latin-1, I'd prefer if we could stick to ASCII, with the
> > exception of tests explicitly needed for PEP 263.
> 
> There are a number of files that have Latin-1 in them, mostly in
> comments. Should we change those as well?

Mostly no, see what I added to PEP 8. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Oct  7 17:49:08 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 18:49:08 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL> <3DA1A310.9030606@lemburg.com>             <15777.42536.817787.472626@gargle.gargle.HOWL> <200210071540.g97Fe9822116@indus.ins.cwi.nl>
Message-ID: <3DA1BB04.6070905@lemburg.com>

Sjoerd Mullender wrote:
> When I execute the three lisp form that were mentioned in the page I
> pointed you to*, my XEmacs does recognize utf-8 encoding automatically.
> I just find-file a file which contains UTF-8 encoded data and I see
> all sorts of accented characters (the ones I expect to see even!).
> My XEmacs version is
> 
> XEmacs 21.4 (patch 6) "Common Lisp" [Lucid] (i386-redhat-linux, Mule)
> of Fri Feb 22 2002 on daffy.perf.redhat.com
> 
> *
> (require 'un-define)
> (set-coding-priority-list '(utf-8))
> (set-coding-category-system 'utf-8 'utf-8)

Hmm, I get a "file not found" error. Looks like SuSE doesn't ship
with the 'un-define' package.

Anyway, thanks for the hint,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From barry@python.org  Mon Oct  7 17:50:29 2002
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 7 Oct 2002 12:50:29 -0400
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
 <200210071540.g97Fe9822116@indus.ins.cwi.nl>
 <3DA1BB04.6070905@lemburg.com>
Message-ID: <15777.47957.552536.42555@gargle.gargle.HOWL>

>>>>> "MAL" == M  <mal@lemburg.com> writes:

    MAL> Hmm, I get a "file not found" error. Looks like SuSE doesn't
    MAL> ship with the 'un-define' package.

FWIW, I always build XEmacs from source --with-mule, and install all
the packages from the package manager.  I'm also using 21.4.9.

-Barry



From Gerson.Kurz@t-online.de  Mon Oct  7 18:02:10 2002
From: Gerson.Kurz@t-online.de (Gerson Kurz)
Date: Mon, 7 Oct 2002 19:02:10 +0200
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <20021007133503.25994.5310.Mailman@mail.python.org>
Message-ID: <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>

Hi folks,

yesterday I posted a problem with importing a very large sourcefile in
python 2.2.1 to clp. For the original report, see
<news:3da070ec.18868296@news.t-online.de>. Andrew MacIntyre suggested that
2.3 would handle this situation much better, so I got the current version
from CVS, built it with cygwin and tested it. Now, instead of just a
MemoryError, I get this:

$ python
Python 2.3a0 (#1, Oct  7 2002, 18:49:14)
[GCC 2.95.3-5 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more informatio
>>> import test
Traceback (most recent call last):
Segmentation fault (core dumped)

The stackdump reads

Exception: STATUS_ACCESS_VIOLATION at eip=61045659
eax=1A013E68 ebx=610C02D0 ecx=610C02D0 edx=0022E6F8 esi=0022E6F8
edi=1A013E68
ebp=0022E6B0 esp=0022E698 program=C:\cygwin\usr\local\bin\python2.3.exe
cs=001B ds=0023 es=0023 fs=0038 gs=0000 ss=0023
Stack trace:
Frame     Function  Args
0022E6B0  61045659  (0022E6F8, 0022EE78, 0022F1A4, 0022F1A8)
0022E820  61048989  (1A010000, 0022ECE8, 0022EE78, 0022F1A4)
0022EE10  610462BA  (00000000, 00010000, 0022EE40, 610BD2CB)
0022EE40  610096F7  (616800F4, 00000003, 0A032B14, 00000000)
0022F1F0  61071CAF  (0A032B14, 00010000, 000001B6, 610AA142)
0022F210  610ACFC6  (610BF030, 0A032B14, 00010000, 000001B6)
0022F240  6109EEA8  (610BF030, 0A032B14, 6B2C525A, 6B241813)
0022F260  6109EF51  (0A032B14, 6B2C525A, 0A0113E0, 61070534)
0022FB90  6B2C52D1  (0A01D090, 0A032B14, 00000001, 0A043994)
0022FBE0  6B2C5795  (0A0358EC, 0A01D090, 0022FC40, 6B2C1861)
0022FC40  6B2C1897  (0A015DDC, 0A0229EC, 0A0358EC, 6B278D43)
0022FC70  6B2C4C8E  (00000000, 0A027AA4, 0022FCC0, 6B2BEA4C)
0022FCB0  6B26BC4D  (0A01BA2C, 0A027AA4, 00000000, 6B2BF4F5)
0022FCE0  6B248486  (0A01BA2C, 0A027AA4, 00000000, 6B2C4AE7)
0022FD10  6B29F082  (0A01BA2C, 0A027AA4, 00000000, 6B2C15D2)
0022FD70  6B2C15E1  (00000001, 6B2C9115, 0022FDE0, 6B2C0D25)
End of stack trace (more stack frames may be present)

So I guess its back in your (aka Developers) corner. You can download the
source that causes this at

http://p-nand-q.com/python/toobigfor2.3.zip

but beware, the zipped source is 613k large (unzipped around 4.2 mb).

Bye, Gerson Kurz
http://p-nand-q.com



From guido@python.org  Mon Oct  7 18:02:44 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 13:02:44 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 19:02:10 +0200."
 <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>
References: <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>
Message-ID: <200210071702.g97H2iw16346@pcp02138704pcs.reston01.va.comcast.net>

Can you please submit a SF bug report for this?  Mail to python-dev
has a tendency to get lost in the pile.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Mon Oct  7 18:04:48 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 19:04:48 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>	<3DA13E1C.5080505@lemburg.com>	<200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>	<3DA185D2.3010100@lemburg.com>	<15777.37683.501642.428503@gargle.gargle.HOWL>	<3DA1A310.9030606@lemburg.com>	<15777.42536.817787.472626@gargle.gargle.HOWL>	<200210071540.g97Fe9822116@indus.ins.cwi.nl>	<3DA1BB04.6070905@lemburg.com> <15777.47957.552536.42555@gargle.gargle.HOWL>
Message-ID: <3DA1BEB0.4030306@lemburg.com>

Barry A. Warsaw wrote:
>>>>>>"MAL" == M  <mal@lemburg.com> writes:
> 
> 
>     MAL> Hmm, I get a "file not found" error. Looks like SuSE doesn't
>     MAL> ship with the 'un-define' package.
> 
> FWIW, I always build XEmacs from source --with-mule, and install all
> the packages from the package manager.  I'm also using 21.4.9.

Turns out that SuSE chose to compile XEmacs with Mule support
but placed the Mule files into a separate package called
'mule-ucs-xemacs'. After I installed that package, XEmacs now
correctly interprets the coding marker and works just fine with
UTF-8 files. Cool :-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Mon Oct  7 18:24:28 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 19:24:28 +0200
Subject: [Python-Dev] PEP 263 in the works (Non-ASCII characters in test_pep277.py
 in 2.3)
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>	<3DA13E1C.5080505@lemburg.com>	<200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>	<3DA185D2.3010100@lemburg.com>	<15777.37683.501642.428503@gargle.gargle.HOWL>	<3DA1A310.9030606@lemburg.com>	<15777.42536.817787.472626@gargle.gargle.HOWL>	<200210071540.g97Fe9822116@indus.ins.cwi.nl>	<3DA1BB04.6070905@lemburg.com> <15777.47957.552536.42555@gargle.gargle.HOWL> <3DA1BEB0.4030306@lemburg.com>
Message-ID: <3DA1C34C.5090506@lemburg.com>

Now that I can edit UTF-8 directly, I find a "feature" made
possible by the PEP 263 support of Python 2.3 rather
puzzling:

Let's say I edit a file testencoding.py in XEmacs with UTF-8
support:

# -*- coding: utf-8; -*-
# comment =E4=F6=FC
s =3D "=E4=F6=FC"
u =3D u"=E4=F6=FC"
print s
print u.encode('latin-1')
print 'works !'

With Python 2.3 this prints:

=C3=A4=C3=B6=C3=BC
=E4=F6=FC
works !

I would have expected that s turns out as "=E4=F6=FC" using print,
since that's how I wrote it in the source file.

This suggests to me that mixing string and Unicode literals
using non-ASCII characters in a single file should probably
be avoided.

--=20
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From martin@v.loewis.de  Mon Oct  7 18:28:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 19:28:11 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071643.g97Ghuo16131@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <m34rbyb8il.fsf@mira.informatik.hu-berlin.de>
 <200210071643.g97Ghuo16131@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m31y726tw4.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Mostly no, see what I added to PEP 8. :-)

Ok, I've changed PEP 277 to use Unicode literals with escapes.

Regards,
Martin


From guido@python.org  Mon Oct  7 18:31:03 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 13:31:03 -0400
Subject: [Python-Dev] PEP 263 in the works (Non-ASCII characters in test_pep277.py in 2.3)
In-Reply-To: Your message of "Mon, 07 Oct 2002 19:24:28 +0200."
 <3DA1C34C.5090506@lemburg.com>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL> <3DA1A310.9030606@lemburg.com> <15777.42536.817787.472626@gargle.gargle.HOWL> <200210071540.g97Fe9822116@indus.ins.cwi.nl> <3DA1BB04.6070905@lemburg.com> <15777.47957.552536.42555@gargle.gargle.HOWL> <3DA1BEB0.4030306@lemburg.com>
 <3DA1C34C.5090506@lemburg.com>
Message-ID: <200210071731.g97HV3v24668@pcp02138704pcs.reston01.va.comcast.net>

> Now that I can edit UTF-8 directly, I find a "feature" made
> possible by the PEP 263 support of Python 2.3 rather
> puzzling:
> 
> Let's say I edit a file testencoding.py in XEmacs with UTF-8
> support:

(Note that I'm viewing this as Latin-1.  The comment, s and u in the
source are all three the same: a-umlaut, o-umlaut, u-umlaut.)

> # -*- coding: utf-8; -*-
> # comment äöü
> s = "äöü"
> u = u"äöü"
> print s
> print u.encode('latin-1')
> print 'works !'
> 
> With Python 2.3 this prints:
> 
> äöü
> äöü
> works !
> 
> I would have expected that s turns out as "äöü" using print,
> since that's how I wrote it in the source file.

No, because stdout isn't assumed to be UTF-8.  The string s is your
string encoded in UTF-8, and those are the bytes written by print.

> This suggests to me that mixing string and Unicode literals
> using non-ASCII characters in a single file should probably
> be avoided.

Or it suggests that we need a way to deal with encodings on stdout
more gently.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From zack@codesourcery.com  Mon Oct  7 18:38:34 2002
From: zack@codesourcery.com (Zack Weinberg)
Date: Mon, 7 Oct 2002 10:38:34 -0700
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>
References: <20021007133503.25994.5310.Mailman@mail.python.org> <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>
Message-ID: <20021007173834.GA10088@codesourcery.com>

On Mon, Oct 07, 2002 at 07:02:10PM +0200, Gerson Kurz wrote:
> Hi folks,
> 
> yesterday I posted a problem with importing a very large sourcefile in
> python 2.2.1 to clp. For the original report, see
> <news:3da070ec.18868296@news.t-online.de>. Andrew MacIntyre suggested that
> 2.3 would handle this situation much better, so I got the current version
> from CVS, built it with cygwin and tested it. Now, instead of just a
> MemoryError, I get this:

For what it's worth, this may be system-specific.  I was able to
import your test case under both 2.2.1 and 2.3a0 with no problem, on
linux.  The 2.3 compiler is a lot faster, which is nice.

However, running python 2.3 under valgrind produces floods of 'invalid
read' and 'conditional depends on uninitialized value' errors before
we ever get to the prompt, so this may well be a system-independent
bug that only happens to expose itself on cygwin.

zw

Python 2.2.1 (#2, Sep 13 2002, 23:25:07) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2

# from .py
>>> profile.run('import test')
         3 function calls in 49.840 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   48.980   48.980   49.830   49.830 <string>:1(?)
        1    0.010    0.010   49.840   49.840 profile:0(import test)
        0    0.000             0.000          profile:0(profiler)
        1    0.850    0.850    0.850    0.850 test.py:2(?)

# from .pyc
>>> profile.run('import test') 
         3 function calls in 1.140 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.420    0.420    1.130    1.130 <string>:1(?)
        1    0.010    0.010    1.140    1.140 profile:0(import test)
        0    0.000             0.000          profile:0(profiler)
        1    0.710    0.710    0.710    0.710 test.py:2(?)

Python 2.3a0 (#30, Oct  7 2002, 10:25:24) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2

# from .py
>>> profile.run('import test')
         3 function calls in 19.030 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   18.370   18.370   19.000   19.000 <string>:1(?)
        1    0.030    0.030   19.030   19.030 profile:0(import test)
        0    0.000             0.000          profile:0(profiler)
        1    0.630    0.630    0.630    0.630 test.py:2(?)

# from .pyc
>>> profile.run('import test')
         3 function calls in 0.920 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.420    0.420    0.910    0.910 <string>:1(?)
        1    0.010    0.010    0.920    0.920 profile:0(import test)
        0    0.000             0.000          profile:0(profiler)
        1    0.490    0.490    0.490    0.490 test.py:2(?)


From martin@v.loewis.de  Mon Oct  7 18:53:03 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 19:53:03 +0200
Subject: [Python-Dev] PEP 263 in the works (Non-ASCII characters in test_pep277.py in 2.3)
In-Reply-To: <3DA1C34C.5090506@lemburg.com>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
 <200210071540.g97Fe9822116@indus.ins.cwi.nl>
 <3DA1BB04.6070905@lemburg.com>
 <15777.47957.552536.42555@gargle.gargle.HOWL>
 <3DA1BEB0.4030306@lemburg.com> <3DA1C34C.5090506@lemburg.com>
Message-ID: <m3vg4e5e68.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> With Python 2.3 this prints:
>=20
> =C3=83=C2=A4=C3=83=C2=B6=C3=83=C2=BC

As Guido explains: No, it doesn't. Your terminal display this
incorrectly.

Please understand that Python cannot possibly know, at run-time, what
encoding a byte string has. This is only possible for Unicode strings.

Regards,
Martin


From martin@v.loewis.de  Mon Oct  7 18:57:31 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 19:57:31 +0200
Subject: [Python-Dev] PEP 263 in the works (Non-ASCII characters in test_pep277.py in 2.3)
In-Reply-To: <200210071731.g97HV3v24668@pcp02138704pcs.reston01.va.comcast.net>
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <3DA13E1C.5080505@lemburg.com>
 <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net>
 <3DA185D2.3010100@lemburg.com>
 <15777.37683.501642.428503@gargle.gargle.HOWL>
 <3DA1A310.9030606@lemburg.com>
 <15777.42536.817787.472626@gargle.gargle.HOWL>
 <200210071540.g97Fe9822116@indus.ins.cwi.nl>
 <3DA1BB04.6070905@lemburg.com>
 <15777.47957.552536.42555@gargle.gargle.HOWL>
 <3DA1BEB0.4030306@lemburg.com> <3DA1C34C.5090506@lemburg.com>
 <200210071731.g97HV3v24668@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3r8f25dys.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Or it suggests that we need a way to deal with encodings on stdout
> more gently.

See python.org/sf/612627. With that patch, MAL should be able to do

print u

and get it out correctly no matter what terminal encoding the user
uses (assuming the user follows system conventions).

It would also allow to do

print s.decode("utf-8").encode(sys.stdout.encoding)

Either would work as long as sys.stdout is a terminal, which should
include IDLE interactive sys.stdout. It is debatable whether this
should work for interactive mode only, or for all scripts.

Regards,
Martin


From Gerson.Kurz@t-online.de  Mon Oct  7 19:03:56 2002
From: Gerson.Kurz@t-online.de (Gerson Kurz)
Date: Mon, 7 Oct 2002 20:03:56 +0200
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <20021007173834.GA10088@codesourcery.com>
Message-ID: <KGEPLGONBPHFGIHOBLCKIEAKCKAA.gerson.kurz@t-online.de>

Zack Weinberg wrote:

> For what it's worth, this may be system-specific.  I was able to
> import your test case under both 2.2.1 and 2.3a0 with no problem, on
> linux.  The 2.3 compiler is a lot faster, which is nice.

It seems like that. I compiled with DevStudio 6, and the import works (at
least in the debug build, but thats the only one I checked). I recompiled
python unter cygwin using "--with-pydebug", but to the same effect: cygwins'
python.exe raises a segmentation fault. So maybe, just maybe, its a fault of
cygwin ?

$ uname -a
CYGWIN_NT-5.0 DREIZEHN 1.3.12(0.54/3/2) 2002-07-06 02:16 i686 unknown






From guido@python.org  Mon Oct  7 19:03:10 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 14:03:10 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 10:38:34 PDT."
 <20021007173834.GA10088@codesourcery.com>
References: <20021007133503.25994.5310.Mailman@mail.python.org> <KGEPLGONBPHFGIHOBLCKEEAJCKAA.gerson.kurz@t-online.de>
 <20021007173834.GA10088@codesourcery.com>
Message-ID: <200210071803.g97I3AE08189@pcp02138704pcs.reston01.va.comcast.net>

> However, running python 2.3 under valgrind produces floods of
> 'invalid read' and 'conditional depends on uninitialized value'
> errors before we ever get to the prompt, so this may well be a
> system-independent bug that only happens to expose itself on cygwin.

Are you sure these aren't false alarms?  I believe I tried valgrind
once on Python and every lead it gave me was a false alarm or
something in libc. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Oct  7 19:09:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 14:09:38 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <KGEPLGONBPHFGIHOBLCKIEAKCKAA.gerson.kurz@t-online.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENKBJAB.tim.one@comcast.net>

[Gerson Kurz]
> ...
> I compiled with DevStudio 6, and the import works (at least in the
> debug build, but thats the only one I checked). I compiled
> python unter cygwin using "--with-pydebug", but to the same
> effect: cygwins' python.exe raises a segmentation fault. So maybe,
> just maybe, its a fault of cygwin ?

Dunno.  If cygwin emulates the Linux malloc overcommitment policies, memory
faults of this nature are unavoidable, and there's nothing we can do about
it.  There's something you can do about it, though:  don't create 4+MB
source files to begin with.  I don't take problems from such tomfoolery
seriously, although the younger folks here might <wink>.



From pedronis@bluewin.ch  Mon Oct  7 19:09:06 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Mon, 7 Oct 2002 20:09:06 +0200
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
References: <KGEPLGONBPHFGIHOBLCKIEAKCKAA.gerson.kurz@t-online.de>
Message-ID: <07c401c26e2c$a0ed0720$6d94fea9@newmexico>

From: "Gerson Kurz" <Gerson.Kurz@t-online.de>
> $ uname -a
> CYGWIN_NT-5.0 DREIZEHN 1.3.12(0.54/3/2) 2002-07-06 02:16 i686 unknown

yesterday I built the Python 2.3a0 under cygwin and could rarely manage to get
make test complete without some segfault.
But I have not tracked the problems. I continued my hacking under Linux where I
didn't get any, (apart one, but that was me :)).

regards.



From fdrake@acm.org  Mon Oct  7 19:12:48 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 7 Oct 2002 14:12:48 -0400
Subject: [Python-Dev] Python 2.2.2 beta 1 docs finalized
Message-ID: <15777.52896.164933.656370@grendel.zope.com>

I've posted the Python 2.2.2b1 docs online; please report any (new)
problems via SourceForge with a priority of at least 6.

I'd like to thank Raymond for putting so much effort into the
documentation for 2.2.2; he's really doing a great job!

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From mal@lemburg.com  Mon Oct  7 19:19:26 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 07 Oct 2002 20:19:26 +0200
Subject: [Python-Dev] PEP 263 in the works (Non-ASCII characters in test_pep277.py
 in 2.3)
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <3DA13E1C.5080505@lemburg.com> <200210071228.g97CSOL06547@pcp02138704pcs.reston01.va.comcast.net> <3DA185D2.3010100@lemburg.com> <15777.37683.501642.428503@gargle.gargle.HOWL> <3DA1A310.9030606@lemburg.com> <15777.42536.817787.472626@gargle.gargle.HOWL> <200210071540.g97Fe9822116@indus.ins.cwi.nl> <3DA1BB04.6070905@lemburg.com> <15777.47957.552536.42555@gargle.gargle.HOWL> <3DA1BEB0.4030306@lemburg.com>              <3DA1C34C.5090506@lemburg.com> <200210071731.g97HV3v24668@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DA1D02E.4080304@lemburg.com>

Guido van Rossum wrote:
>>Now that I can edit UTF-8 directly, I find a "feature" made
>>possible by the PEP 263 support of Python 2.3 rather
>>puzzling:
>>
>>Let's say I edit a file testencoding.py in XEmacs with UTF-8
>>support:
>=20
>=20
> (Note that I'm viewing this as Latin-1.  The comment, s and u in the
> source are all three the same: a-umlaut, o-umlaut, u-umlaut.)

Oh, yes, forgot to mention that.

>># -*- coding: utf-8; -*-
>># comment =E4=F6=FC
>>s =3D "=E4=F6=FC"
>>u =3D u"=E4=F6=FC"
>>print s
>>print u.encode('latin-1')
>>print 'works !'
>>
>>With Python 2.3 this prints:
>>
>>=C3=A4=C3=B6=C3=BC
>>=E4=F6=FC
>>works !
>>
>>I would have expected that s turns out as "=E4=F6=FC" using print,
>>since that's how I wrote it in the source file.
>=20
>=20
> No, because stdout isn't assumed to be UTF-8.  The string s is your
> string encoded in UTF-8, and those are the bytes written by print.

Uhm, I would have *expected* that behaviour -- using the left side
of my head. The right side knows why this technically doesn't work
out of the box :-)

>>This suggests to me that mixing string and Unicode literals
>>using non-ASCII characters in a single file should probably
>>be avoided.
>=20
> Or it suggests that we need a way to deal with encodings on stdout
> more gently.

--=20
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From tim.one@comcast.net  Mon Oct  7 19:26:31 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 14:26:31 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <200210071803.g97I3AE08189@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENMBJAB.tim.one@comcast.net>

Note that pymalloc causes valgrind to raise tons of false alarms.  This was
discussed previously here, but no time to track down a reference; you need a
valgrind suppression file to avoid this.



From skip@pobox.com  Mon Oct  7 19:37:39 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 7 Oct 2002 13:37:39 -0500
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: <20021004135948.GA30482@epoch.metaslash.com>
References: <20021004135948.GA30482@epoch.metaslash.com>
Message-ID: <15777.54387.246026.24804@montanaro.dyndns.org>

(a bit behind - having mail problems on my new Powerbook...)

    Neal> Is anyone else having problems with 2.2.2 tests?
    Neal> minidom & sax fail for me on Linux:
    ...

On MacOS 10.2 I'm not getting that far.  In test_builtin it gets into the
list test then gobbles up all of memory.  It appears the
list(xrange(sys.maxint/4)) test is the culprit.  This is on 2.2.2 and 2.3.

Skip



From Gerson.Kurz@t-online.de  Mon Oct  7 19:45:14 2002
From: Gerson.Kurz@t-online.de (Gerson Kurz)
Date: Mon, 7 Oct 2002 20:45:14 +0200
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCENKBJAB.tim.one@comcast.net>
Message-ID: <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>

Tim Peters wrote
> There's something you can do about it, though:  don't create 4+MB
> source files to begin with.  I don't take problems from such tomfoolery
> seriously, although the younger folks here might <wink>.

Ok, here is a 55 bytes sourcecode:

b = {}
for i in range(1000000):
    b[i] = [0] * 60

--> segfault with cygwin.



From Gerson.Kurz@t-online.de  Mon Oct  7 19:54:03 2002
From: Gerson.Kurz@t-online.de (Gerson Kurz)
Date: Mon, 7 Oct 2002 20:54:03 +0200
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>
Message-ID: <KGEPLGONBPHFGIHOBLCKOEALCKAA.gerson.kurz@t-online.de>

Gerson Kurz wrote

> Ok, here is a 55 bytes sourcecode:
>
> b = {}
> for i in range(1000000):
>     b[i] = [0] * 60
>
> --> segfault with cygwin.
>

as a follow up, here is an interesting behaviour when used with 2.3:

Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> b = {}
[3319 refs]
>>> for i in range(1000000):
...     b[i]=[i]*60
...
[62003414 refs]
>>> for k in range(1000000):
...     del b[k]
...
[1003419 refs]
>>> print b[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
KeyError: 0
[1003458 refs]
>>> print len(b.keys())
0
[1003621 refs]

The funny thing is, the memory allocated by python_d.exe *INCREASES* when I
do the del b[k]-bit thingy. OK, so I guess its time to do a gc:

>>> import gc
[1003753 refs]
>>> gc.collect()
0
[1003758 refs]
>>> gc.isenabled()
1
[1003758 refs]
>>>

Which frees up, like, nothing. Hm. When I do that with

PythonWin 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32.
...
>>> b = {}
>>> for i in range(100000):
... 	b[i] = [0] * 60
...
>>> for k in range(100000):
... 	del b[k]
...
>>>

(higher numbers throw MemoryException) I get a much more expected behaviour,
del b[k] frees up about 50% of the memory used (50 meg total), although
gc.collect() likewise has no measurable effect.






From guido@python.org  Mon Oct  7 19:49:47 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 14:49:47 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: Your message of "Mon, 07 Oct 2002 13:37:39 CDT."
 <15777.54387.246026.24804@montanaro.dyndns.org>
References: <20021004135948.GA30482@epoch.metaslash.com>
 <15777.54387.246026.24804@montanaro.dyndns.org>
Message-ID: <200210071849.g97Inlt08581@pcp02138704pcs.reston01.va.comcast.net>

> On MacOS 10.2 I'm not getting that far.  In test_builtin it gets into the
> list test then gobbles up all of memory.  It appears the
> list(xrange(sys.maxint/4)) test is the culprit.  This is on 2.2.2 and 2.3.

That's strange -- that test is specifically designed to fail before it
ever gets to allocating memory.  Can you track this down with a
debugger?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct  7 19:50:35 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 14:50:35 -0400
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 20:45:14 +0200."
 <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>
References: <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>
Message-ID: <200210071850.g97IoZa08599@pcp02138704pcs.reston01.va.comcast.net>

> Ok, here is a 55 bytes sourcecode:
> 
> b = {}
> for i in range(1000000):
>     b[i] = [0] * 60
> 
> --> segfault with cygwin.

Looks like a bug in CYGWIN malloc then.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From sholden@holdenweb.com  Mon Oct  7 19:48:33 2002
From: sholden@holdenweb.com (Steve Holden)
Date: Mon, 7 Oct 2002 14:48:33 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
References: <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>
Message-ID: <060b01c26e32$25a166f0$6300000a@holdenweb.com>

Gerson Kurz wrote ...

> Tim Peters wrote
> > There's something you can do about it, though:  don't create 4+MB
> > source files to begin with.  I don't take problems from such tomfoolery
> > seriously, although the younger folks here might <wink>.
>
> Ok, here is a 55 bytes sourcecode:
>
> b = {}
> for i in range(1000000):
>     b[i] = [0] * 60
>
> --> segfault with cygwin.
>

This is a fixed bug, due to a problem with malloc on the Cygwin platform, I
believe. From what I can remember it's been fixed, but it wasn't directly a
Python bug to start with.

regards
-----------------------------------------------------------------------
Steve Holden                                  http://www.holdenweb.com/
Python Web Programming                 http://pydish.holdenweb.com/pwp/
Previous .sig file retired to                    www.homeforoldsigs.com
-----------------------------------------------------------------------




From guido@python.org  Mon Oct  7 19:53:22 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 14:53:22 -0400
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 20:54:03 +0200."
 <KGEPLGONBPHFGIHOBLCKOEALCKAA.gerson.kurz@t-online.de>
References: <KGEPLGONBPHFGIHOBLCKOEALCKAA.gerson.kurz@t-online.de>
Message-ID: <200210071853.g97IrMO08662@pcp02138704pcs.reston01.va.comcast.net>

> as a follow up, here is an interesting behaviour when used with 2.3:
> 
> Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> b = {}
> [3319 refs]
> >>> for i in range(1000000):
> ...     b[i]=[i]*60
> ...
> [62003414 refs]
> >>> for k in range(1000000):
> ...     del b[k]
> ...
> [1003419 refs]
> >>> print b[0]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> KeyError: 0
> [1003458 refs]
> >>> print len(b.keys())
> 0
> [1003621 refs]
> 
> The funny thing is, the memory allocated by python_d.exe *INCREASES* when I
> do the del b[k]-bit thingy. OK, so I guess its time to do a gc:

No, there are no cycles in the data structures you create here.

I suspect another CYGWIN malloc bug.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Oct  7 19:53:42 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 14:53:42 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <KGEPLGONBPHFGIHOBLCKGEALCKAA.gerson.kurz@t-online.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEOABJAB.tim.one@comcast.net>

[Gerson Kurz]
> Ok, here is a 55 bytes sourcecode:
>
> b = {}
> for i in range(1000000):
>     b[i] = [0] * 60
>
> --> segfault with cygwin.

As Guido said, bug reports belong on SourceForge.  It doesn't segfault for
me (MSVC6 + Win98SE), although it takes a long time to complete and thrashes
to disk horribly.  But distinctions like that are lost forever unless
they're attached to a bug report.

BTW, if you need to do things "like this" frequently, buy a 64-bit box, a
terabyte of RAM, and get a real operating system <wink>.



From Gerson.Kurz@t-online.de  Mon Oct  7 20:00:35 2002
From: Gerson.Kurz@t-online.de (Gerson Kurz)
Date: Mon, 7 Oct 2002 21:00:35 +0200
Subject: AW: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <200210071853.g97IrMO08662@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <KGEPLGONBPHFGIHOBLCKKEAMCKAA.gerson.kurz@t-online.de>

> No, there are no cycles in the data structures you create here.
> 
> I suspect another CYGWIN malloc bug.
> 
No, that was done with the DevStudio build. Note also this:

Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

python_d mem usage : 2.340 K

>>> b = {}
[3319 refs]
>>> for i in range(1000000):
...     b[i] = [i] * 60
...
[62003414 refs]

python_d mem usage : 376.320 K

>>> del b
[3413 refs]
>>>

python_d mem usage : 351.716 K

Again, with the devstudio build (the cygwin build will crash in the loop)


From zack@codesourcery.com  Mon Oct  7 20:05:47 2002
From: zack@codesourcery.com (Zack Weinberg)
Date: Mon, 7 Oct 2002 12:05:47 -0700
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIENMBJAB.tim.one@comcast.net>
References: <200210071803.g97I3AE08189@pcp02138704pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCIENMBJAB.tim.one@comcast.net>
Message-ID: <20021007190547.GA18632@codesourcery.com>

On Mon, Oct 07, 2002 at 02:26:31PM -0400, Tim Peters wrote:
> Note that pymalloc causes valgrind to raise tons of false alarms.  This was
> discussed previously here, but no time to track down a reference; you need a
> valgrind suppression file to avoid this.

Yeah, many of them appeared to be pymalloc-related.  I didn't take the
time to dig through the list to see if any weren't; I wanted to raise
a potential concern rather than an actual one.

zw


From thomas@xs4all.net  Mon Oct  7 20:06:31 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 7 Oct 2002 21:06:31 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
References: <00ca01c26de9$4dbcedb0$e000a8c0@thomasnotebook> <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
Message-ID: <20021007190631.GD5116@xs4all.nl>

On Mon, Oct 07, 2002 at 10:45:53AM -0400, Tim Peters wrote:

> You seem to want most of all to avoid warning msgs, but __future__ options
> don't accomplish that:  it's the *point* of __future__ thingies to spit out
> warnings about semantics that are going to change.

> I don't know that it's a useful point, though.  That is, the only comments
> I've seen about __future__ from users in practice is from those who are
> annoyed by needing to say "__future__" all over the place to get at the new
> features they want right away.  Alas, I haven't seen evidence that it eases
> migration.  That could be because it works so well people don't feel a need
> to comment about it, but somehow that seems a tad unlikely <wink>.

Well, I can tell you that __future__ has definately worked. Without it, I
would not have been able to upgrade Python on many of our servers -- and
because we don't need newer versions ourselves, they would not likely be
installed at all. As it is, I can safely upgrade to the next major version
when they come out, instead of having to start a lengthy and very
energy-consuming customer-acceptance process. And this anal attitude towards
compatibility isn't just me, it's company policy ;)

I suspect all companies that maintain python on behalf of third parties
are grateful of future. Or at least, the good ones, the others merely
_should_ be grateful :)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From tim.one@comcast.net  Mon Oct  7 20:07:30 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 15:07:30 -0400
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <200210071853.g97IrMO08662@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOCBJAB.tim.one@comcast.net>

[Gerson Kurz]

> Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
>  b = {}
> > [3319 refs]
> > >>> for i in range(1000000):
> > ...     b[i]=[i]*60
> > ...
> > [62003414 refs]
> > >>> for k in range(1000000):
> > ...     del b[k]
> > ...
> > [1003419 refs]
> > >>> print b[0]
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > KeyError: 0
> > [1003458 refs]
> > >>> print len(b.keys())
> > 0
> > [1003621 refs]
> >
> > The funny thing is, the memory allocated by python_d.exe
> *INCREASES* when I
> > do the del b[k]-bit thingy. OK, so I guess its time to do a gc:

[Guido]
> No, there are no cycles in the data structures you create here.
>
> I suspect another CYGWIN malloc bug.

Unless I'm reading it wrong, Gerson wasn't using Cygwin there.  The int free
list is immortal and unbounded, and the structure of the loop interleaves
int allocations with list allocs and dict resizings.  So it's unlikely
memory will be "given back to the system" no matter which platform this is
done on -- it's too fragmented.

It doesn't make sense that sysreftotal isn't falling, though.  It's as if
the int free list were holding on to references ...



From guido@python.org  Mon Oct  7 20:11:23 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 15:11:23 -0400
Subject: AW: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 21:00:35 +0200."
 <KGEPLGONBPHFGIHOBLCKKEAMCKAA.gerson.kurz@t-online.de>
References: <KGEPLGONBPHFGIHOBLCKKEAMCKAA.gerson.kurz@t-online.de>
Message-ID: <200210071911.g97JBN008783@pcp02138704pcs.reston01.va.comcast.net>

> No, that was done with the DevStudio build. Note also this:
> 
> Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> 
> python_d mem usage : 2.340 K
> 
> >>> b = {}
> [3319 refs]
> >>> for i in range(1000000):
> ...     b[i] = [i] * 60
> ...
> [62003414 refs]
> 
> python_d mem usage : 376.320 K
> 
> >>> del b
> [3413 refs]
> >>>
> 
> python_d mem usage : 351.716 K
> 
> Again, with the devstudio build (the cygwin build will crash in the loop)

It's not a bug if freed memory is not given back to the system.
Fragmentation probably prevents this in this case.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Mon Oct  7 20:22:12 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Oct 2002 21:22:12 +0200
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: <200210071849.g97Inlt08581@pcp02138704pcs.reston01.va.comcast.net>
References: <20021004135948.GA30482@epoch.metaslash.com>
 <15777.54387.246026.24804@montanaro.dyndns.org>
 <200210071849.g97Inlt08581@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3adlq5a1n.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> That's strange -- that test is specifically designed to fail before it
> ever gets to allocating memory.  Can you track this down with a
> debugger?

Notice that this test does not strictly achieve that: with a four-byte
pointer, you attempt to allocate 0x20000000L items (due to
roundupsize), which is small enough to try a realloc. In turn, it
tries to allocate 0x80000000L bytes, i.e. 2GB. On a 32-bit machine, it
is possible to allocate that much memory.

Regards,
Martin


From tim.one@comcast.net  Mon Oct  7 20:22:42 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 15:22:42 -0400
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEOCBJAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEOEBJAB.tim.one@comcast.net>

[Tim]
> It doesn't make sense that sysreftotal isn't falling, though.  It's as
> if the int free list were holding on to references ...

It makes sense after all -- they're all references to the dummy key value we
plug into a dict slot when del'ing a key.



From guido@python.org  Mon Oct  7 20:29:59 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 15:29:59 -0400
Subject: AW: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: Your message of "Mon, 07 Oct 2002 15:07:30 EDT."
 <LNBBLJKPBEHFEDALKOLCMEOCBJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEOCBJAB.tim.one@comcast.net>
Message-ID: <200210071930.g97JU0G11290@pcp02138704pcs.reston01.va.comcast.net>

> [Gerson Kurz]
> 
> > Python 2.3a0 (#29, Oct  7 2002, 19:54:53) [MSC 32 bit (Intel)] on win32
> >  b = {}
> > > [3319 refs]
> > > >>> for i in range(1000000):
> > > ...     b[i]=[i]*60
> > > ...
> > > [62003414 refs]
> > > >>> for k in range(1000000):
> > > ...     del b[k]
> > > ...
> > > [1003419 refs]
> > > >>> print b[0]
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in ?
> > > KeyError: 0
> > > [1003458 refs]
> > > >>> print len(b.keys())
> > > 0
> > > [1003621 refs]
> > >
> > > The funny thing is, the memory allocated by python_d.exe
> > *INCREASES* when I
> > > do the del b[k]-bit thingy. OK, so I guess its time to do a gc:
> 
> [Guido]
> > No, there are no cycles in the data structures you create here.
> >
> > I suspect another CYGWIN malloc bug.
> 
> Unless I'm reading it wrong, Gerson wasn't using Cygwin there.  The int free
> list is immortal and unbounded, and the structure of the loop interleaves
> int allocations with list allocs and dict resizings.  So it's unlikely
> memory will be "given back to the system" no matter which platform this is
> done on -- it's too fragmented.
> 
> It doesn't make sense that sysreftotal isn't falling, though.  It's as if
> the int free list were holding on to references ...

I tried this with smaller numbers on Linux, and it's pretty much the
same (some of the [startup...] messages are from my $PYTHONSTARTUP):

Python 2.2.2b1 (#21, Oct  7 2002, 15:15:32) 
[GCC 2.96 20000731 (Red Hat Linux 7.2 2.96-108.7.2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
[startup.py ...]
[no rl completer: No module named readline]
[startup.py done]
>>> b = {}
[16738 refs]
>>> for i in range(100000): b[i]=[i]*60
    

[6216740 refs]
>>> [6216740 refs]
>>> for k in range(100000): del b[k]
    
[116742 refs]
>>> b
{}
[116749 refs]
>>> 1   
1
[116749 refs]
>>> del b
[16748 refs]
>>> ^D
[16748 refs]
[exithandler()]
[5704 refs]

The extra references must be refs to the dummy value that the dict
implementation inserts when it deletes a value.

So this is not something to worry about.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Oct  7 20:42:14 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 15:42:14 -0400
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <20021007190631.GD5116@xs4all.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEOGBJAB.tim.one@comcast.net>

[Thomas Wouters]
> Well, I can tell you that __future__ has definately worked. Without
> it, I would not have been able to upgrade Python on many of our
> servers -- and because we don't need newer versions ourselves, they
> would not likely be installed at all. As it is, I can safely upgrade
> to the next major version when they come out, instead of having to
> start a lengthy and very energy-consuming customer-acceptance process.
> And this anal attitude towards compatibility isn't just me, it's
> company policy ;)
>
> I suspect all companies that maintain python on behalf of third parties
> are grateful of future. Or at least, the good ones, the others merely
> _should_ be grateful :)

I'm not sure this counts as a success:  it doesn't sound like __future__ is
helping you migrate, it sounds like __future__ is helping you *not* migrate
so far.  What are you going to do when 2.3 comes out, and the things that
*were* future in 2.2 are no longer choices?  Some code will break if you
move to 2.3.  Will you be able to get away with saying "but Python 2.2
warned you about this all along, so we're moving to 2.3 regardless"?  If so,
then I'd count it a success.



From thomas.heller@ion-tof.com  Mon Oct  7 21:03:56 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 7 Oct 2002 22:03:56 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
References: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
Message-ID: <053501c26e3c$ab4871e0$e000a8c0@thomasnotebook>

> [Mark Hammond]
> > While this case seems quite trivial, I am starting to face this issue
> > more and more, especially as I am seeing these lovely "FutureWarnings"
> > from all my lovely 32 bit hexadecimal constants <wink/frown>
> 
> [Tim]
> > Sticking "L" at the end is usually all it takes.
> 
> [Thomas Heller]
> > That removes the warnings for 'x = 0x80000000L'.
> 
> What else do you think Mark may have meant by "32 bit hexadecimal
> constants"?
> 
> > Is there a way (other than the -w command-line arg)
> > to suppress the warnings when doing 'hex(-1)'?
> 
> The result of that is also currently platform-dependent, so you need to say
> what you mean in a platform-independent way.  It's likely that you meant
> 
>     hex(-1 & 0xffffffffL)
> 
> or, more directly,
> 
>     hex(0xffffffffL)
> 
> but that someone on a 64-bit box meant something other than that.
> 
> > Shouldn't there be a __future__ option?
> 
> You seem to want most of all to avoid warning msgs, but __future__ options
> don't accomplish that:  it's the *point* of __future__ thingies to spit out
> warnings about semantics that are going to change.
Yes, I want to avoid the warnings.

I've reread pep237 (is this the relevant one?) again.
Do I understand this correctly, that all the following expressions
will be 'true' and will compile/execute without any warnings (on
a 32bit box):

Python 2.2:
0x80000000 == -2147483648

Python 2.3:
0x80000000L == 2147483648

Python 2.x (Phase B1?):
0x80000000L == 2147483648

Python 2.y (Phase B2 and later):
0x80000000 == 2147483648

Regards,

Thomas


From guido@python.org  Mon Oct  7 21:12:17 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 16:12:17 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Sun, 06 Oct 2002 13:13:30 EDT."
 <Pine.LNX.4.44.0210061231560.28570-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210061231560.28570-100000@penguin.theopalgroup.com>
Message-ID: <200210072012.g97KCIS11560@pcp02138704pcs.reston01.va.comcast.net>

> Python 2.2.2 beta 1 is coming out, and I was hoping to float a final
> request to have the fix for bug #537450 applied to the 2.2 maint
> branch (it is a trivial backport, since the 2.3 patch applies just
> fine).  A back-port was originally passed over, because it changes
> semantics from Python 2.2.1.  I argue that the changed semantics are
> desirable and constitute a clear bug fix, which closes holes that
> can allow very obscure errors.  It would be very surprising to me if
> this change breaks any extant code, since it is virtually impossible
> to take meaningful or safe advantage of the original semantics.
[...]
> I assert that no sensible developer would ever write code that
> relied on the pre-2.3 CVS semantics.  I've done a survey of much of
> the meta-class code posted to or discussed on Python-dev and
> comp.lang.python, and none rely on this behavior.  Plus, even if
> they did, we have already decided they will have to change it when
> 2.3 comes out.
> 
> If you are not entirely convinced by the above argument, please
> consider a compromise solution: put the fix in for 2.2.1 Beta 1.  If
> anyone has complaints based on extant code, then we can always back
> it out for Beta 2, or the final release.

OK, that's a deal.  It's in 2.2.2b1.  If there are problems with it I
may back it out before 2.2.2final though.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Mon Oct  7 21:18:05 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 7 Oct 2002 16:18:05 -0400 (EDT)
Subject: [Python-Dev] Bug #537450 Improper object initialization for
 type.__new__
In-Reply-To: <200210072012.g97KCIS11560@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0210071614430.15953-100000@penguin.theopalgroup.com>

On Mon, 7 Oct 2002, Guido van Rossum wrote:
> > If you are not entirely convinced by the above argument, please
> > consider a compromise solution: put the fix in for 2.2.1 Beta 1.  If
> > anyone has complaints based on extant code, then we can always back
> > it out for Beta 2, or the final release.
> 
> OK, that's a deal.  It's in 2.2.2b1.  If there are problems with it I
> may back it out before 2.2.2final though.

Thanks -- I appreciate it.  This will indeed make life easier for me.  I'll
bet you a beer that nobody (who doesn't read this list) will complain.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Mon Oct  7 21:32:42 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 16:32:42 -0400
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: Your message of "Mon, 07 Oct 2002 22:03:56 +0200."
 <053501c26e3c$ab4871e0$e000a8c0@thomasnotebook>
References: <LNBBLJKPBEHFEDALKOLCCEMABJAB.tim.one@comcast.net>
 <053501c26e3c$ab4871e0$e000a8c0@thomasnotebook>
Message-ID: <200210072032.g97KWgA11625@pcp02138704pcs.reston01.va.comcast.net>

> I've reread pep237 (is this the relevant one?) again.

Yes.

> Do I understand this correctly, that all the following expressions
> will be 'true' and will compile/execute without any warnings (on
> a 32bit box):
> 
> Python 2.2:
> 0x80000000 == -2147483648

Yes.

> Python 2.3:
> 0x80000000L == 2147483648

Also in 2.2.  Before that, 0x80000000L had the same value, but
2147483648 raised OverflowError.

> Python 2.x (Phase B1?):
> 0x80000000L == 2147483648
> 
> Python 2.y (Phase B2 and later):
> 0x80000000 == 2147483648

Yes.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Mon Oct  7 22:22:03 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 7 Oct 2002 23:22:03 +0200
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEOGBJAB.tim.one@comcast.net>
References: <20021007190631.GD5116@xs4all.nl> <LNBBLJKPBEHFEDALKOLCIEOGBJAB.tim.one@comcast.net>
Message-ID: <20021007212203.GR621@xs4all.nl>

On Mon, Oct 07, 2002 at 03:42:14PM -0400, Tim Peters wrote:
> [Thomas Wouters]
> > Well, I can tell you that __future__ has definately worked. Without
> > it, I would not have been able to upgrade Python on many of our
> > servers -- and because we don't need newer versions ourselves, they
> > would not likely be installed at all. As it is, I can safely upgrade
> > to the next major version when they come out, instead of having to
> > start a lengthy and very energy-consuming customer-acceptance process.
> > And this anal attitude towards compatibility isn't just me, it's
> > company policy ;)

> I'm not sure this counts as a success:  it doesn't sound like __future__ is
> helping you migrate, it sounds like __future__ is helping you *not* migrate
> so far.  What are you going to do when 2.3 comes out, and the things that
> *were* future in 2.2 are no longer choices?  Some code will break if you
> move to 2.3.  Will you be able to get away with saying "but Python 2.2
> warned you about this all along, so we're moving to 2.3 regardless"?  If so,
> then I'd count it a success.

We've already been there (2.0 -> 2.1 -> 2.2) and that's exactly
what we did. Customers had several months, some more, to notice warnings
about deprecated features and the like, and to use the __future__ import if
they wanted newer features. That gives us a very justified 'too bad' if they
come complaining afterwards anyway. It also inspires more trust from the
more, uhm, 'traditional' companies if you have a clear and documented
upgrade path for software. They often have to hire outside programmers to do
the work, and they want to know about upkeep as well.

I could talk for hours about how we are switching from an old
hysterical-raisin[*] setup that was hard to update and practically
impossible to upgrade major portions of, where it would take months to
migrate customers from BSDI 3.1 to BSDI 4.0, to our current 'ideal' setup
where everything is maintained on a small set of 'master' servers that
distribute their changes to hundreds of other servers automatically (which
www.python.org is part of, by the way) -- but I won't bore you with the
details. Rest assured, __future__ imports (or at least 'no breakage without
a warning one major release before', which doesn't strictly require
__future__) make it a heck of a lot easier.

Backwards-doesn't-mean-retarded'ly y'rs,

[*] For some reason the Jargon file lists this as 'hysterical reasons', in
case you're wondering what it is. Must be a hysterical raisin.
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Oct  7 22:45:59 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 17:45:59 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
Message-ID: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>

I've committed the last changes for Python 2.2.2b1 -- the Misc/NEWS
file.  I'm now going to follow the rest of the PEP 102 script until
the release is done.  In the mean time, please no more checkins to the
2.2 branch!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Mon Oct  7 22:54:15 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 7 Oct 2002 16:54:15 -0500
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
Message-ID: <15778.647.849183.55911@montanaro.dyndns.org>

When building from CVS head, I get this warning for each file compiled:

    gcc -c -DNDEBUG -g -I. -I../Include  -DPy_BUILD_CORE -o Objects/rangeobject.o ../Objects/rangeobject.c
    In file included from ../Include/Python.h:70,
                     from ../Objects/rangeobject.c:3:
    ../Include/objimpl.h:252: warning: use of `long double' type; its size may change in a future release
    ../Include/objimpl.h:252: warning: (Long double usage is reported only once for each file.
    ../Include/objimpl.h:252: warning: To disable this warning, use -Wno-long-double.)

The culprit is the long double alignment field in:

    /* GC information is stored BEFORE the object structure. */
    typedef union _gc_head {
            struct {
                    union _gc_head *gc_next;
                    union _gc_head *gc_prev;
                    int gc_refs;
            } gc;
            long double dummy;  /* force worst-case alignment */
    } PyGC_Head;

I don't know what the best way to fix this is.  Should I look at making
configure add -Wno-long-double in the proper situations or should that
alignment field be something else in situations where the use of long double
would generate a warning?

Skip



From tim.one@comcast.net  Mon Oct  7 23:02:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 18:02:37 -0400
Subject: [Python-Dev] Unclear on the way forward with unsigned integers
In-Reply-To: <20021007212203.GR621@xs4all.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPABJAB.tim.one@comcast.net>

[Thomas Wouters]
> We've already been there (2.0 -> 2.1 -> 2.2) and that's exactly
> what we did.

Cool!  Then it's exactly as I expected:  it works so well, nobody thought to
mention it <wink>.

> Customers had several months, some more, to notice warnings about
> deprecated features and the like, and to use the __future__  import
> if they wanted newer features. That gives us a very justified 'too
> bad' if they come complaining afterwards anyway.

That was all the intent.

> It also inspires more trust from the more, uhm, 'traditional'
> companies if you have a clear and documented upgrade path for
> software.

Let's quit here while we're ahead <0.9 wink>.



From greg@cosc.canterbury.ac.nz  Mon Oct  7 23:38:29 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Oct 2002 11:38:29 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <yu99d6qmxsvy.fsf@europa.research.att.com>
Message-ID: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>

Andrew> In that case, wouldn't you have a situation where a==b, a and
Andrew> b are the same type, but x[a]!=x[b]?

Yes. Not such a good idea, I suppose.

I've just had another idea:

  {2/3}

This can't be confused with a dict, because there's
no ":" in it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From dave@boost-consulting.com  Mon Oct  7 23:56:09 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 07 Oct 2002 18:56:09 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
Message-ID: <u8z19uad2.fsf@boost-consulting.com>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Andrew> In that case, wouldn't you have a situation where a==b, a and
> Andrew> b are the same type, but x[a]!=x[b]?
> 
> Yes. Not such a good idea, I suppose.
> 
> I've just had another idea:
> 
>   {2/3}

2r/3 would be nicer IMO.

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From guido@python.org  Tue Oct  8 00:05:11 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 19:05:11 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Tue, 08 Oct 2002 11:38:29 +1300."
 <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
Message-ID: <200210072305.g97N5Bq20760@pcp02138704pcs.reston01.va.comcast.net>

> I've just had another idea:
> 
>   {2/3}
> 
> This can't be confused with a dict, because there's
> no ":" in it.

On the slight chance that you're serious, I'd like to reserve {...}
without colons for set notation, see PEP 218.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Oct  8 00:39:49 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 7 Oct 2002 18:39:49 -0500
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
References: <20021006115713.GB22400@lysator.liu.se>
 <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15778.6981.362515.852029@montanaro.dyndns.org>

    >> Should the configure script try with /usr/ccs/bin/ar and ld if none
    >> is explicitly supplied?

I think you should add /usr/ccs/{lib,bin,include} and /sw/{lib,bin,include}
to places you automatically search for "useful stuff".  As Anders pointed
out, /usr/ccs is something of a standard among the more proprietary of the
Unix platforms.  /sw/... is where the fink gang installs all the various GNU
goodies on MacOS X.  While you won't need to search /sw/bin for ar and ld,
you might well find useful third-party libraries there which various wrapper
modules can use.  When I configured XEmacs without asking it to look in
/sw/...  it came up with a fairly feeble configuration (no jpeg or png
support).  That said, perhaps the /sw thing will work with a simple change
to setup.py.  I'm testing a change right now to see if my build picks up
readline or not.

Skip


From skip@pobox.com  Tue Oct  8 00:56:23 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 7 Oct 2002 18:56:23 -0500
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
 <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15778.7975.178103.577482@montanaro.dyndns.org>

    Guido> BTW, the Wiki is here: http://manatee.mojam.com/py222wiki/

    Guido> I tried to set up an email subscription but the Wiki isn't
    Guido> configured to support that...

That would be my mistake.  I didn't even know you could do that.  Under the
rubric of "better late than never" I'll see what I can do to enable whatever
email subscription capabilities MoinMoin supports.

Skip


From jacobs@penguin.theopalgroup.com  Tue Oct  8 01:02:35 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 7 Oct 2002 20:02:35 -0400 (EDT)
Subject: [Python-Dev] Bug #537450 Improper object initialization for
 type.__new__
In-Reply-To: <15778.7975.178103.577482@montanaro.dyndns.org>
Message-ID: <Pine.LNX.4.44.0210071958510.24102-100000@penguin.theopalgroup.com>

On Mon, 7 Oct 2002, Skip Montanaro wrote:
>     Guido> BTW, the Wiki is here: http://manatee.mojam.com/py222wiki/
> 
>     Guido> I tried to set up an email subscription but the Wiki isn't
>     Guido> configured to support that...
> 
> That would be my mistake.  I didn't even know you could do that.  Under the
> rubric of "better late than never" I'll see what I can do to enable whatever
> email subscription capabilities MoinMoin supports.

On a somewhat tangental note, MoinMoin breaks with Python 2.3 CVS -- it
expects integers from some of the os/filesystem time routines were changed
to return floats.  The fix is trivial, of course, but it is interesting to
note where real code breaks.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From greg@cosc.canterbury.ac.nz  Tue Oct  8 01:18:10 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Oct 2002 13:18:10 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210072305.g97N5Bq20760@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>

Guido:

> On the slight chance that you're serious, I'd like to reserve {...}
> without colons for set notation, see PEP 218.

I have to agree that would be a better use for them.

So, I'm now reduced to suggesting

  <2/3>

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tdelaney@avaya.com  Tue Oct  8 01:18:16 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Tue, 8 Oct 2002 10:18:16 +1000
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A626@natasha.auslabs.avaya.com>

> From: Guido van Rossum [mailto:guido@python.org]
> 
> I don't understand.  How many Unix emulations on Windows do we need?
> We've already absorbed a sheer endless set of patches to make it work
> on CYGWIN.  How does Mingw differ?

None at all. MinGW is not a Unix emulation. There is a minimal shell
environment (MSYS) which works with MinGW (allowing use of the autoconf/make
toolchain) but the produced binaries have no reliance on anything except the
native win32 libraries.

I was recently trying to use MinGW to compile Python CVS at home - I don't
own, not will I pay for MSVC++. I occasionally have to use it at work, but
only from the command line ;) I gave up when it got into posixmodule.c and
there was lots of stuff there I didn't know about.

I personally am very pleased that Gerhard is working so hard to make this a
viable compilation option on Windows.

http://www.mingw.org/
http://www.mingw.org/msys.shtml

Tim Delaney


From brian@sweetapp.com  Tue Oct  8 01:32:39 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Mon, 07 Oct 2002 17:32:39 -0700
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>
Message-ID: <002301c26e62$353cc2a0$df7e4e18@brianspiv1700>

Greg Ewing wrote:
> I have to agree that would be a better use for them.
> 
> So, I'm now reduced to suggesting
> 
>   <2/3>

Is that going to be easy to parse?

>>> 1 <5/2> -3
1

Cheers,
Brian



From tdelaney@avaya.com  Tue Oct  8 01:39:58 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Tue, 8 Oct 2002 10:39:58 +1000
Subject: [Python-Dev] test_pep277 vs Win98
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A627@natasha.auslabs.avaya.com>

> From: martin@v.loewis.de [mailto:martin@v.loewis.de]
> 
> Tim Peters <tim.one@comcast.net> writes:
> 
> > > Furthermore, how would I implement 
> os.path.supports_unicode_filenames
> > > for ntpath? It seems that something must be exported from os.
> > 
> > This sounds like, for now, whether a platform supports this 
> or not is a
> > matter of hand-forcing a config symbol, where only a few 
> Windows flavors are
> > currently justified in saying "yup, works here".
> 
> Repeating my question: How can I hand-force a config symbol to True on
> NT+ and to False on W9x?

Sounds like an exceptional situation to me.

try:
    try to use problem filename(s)
    yup_works_here = True
except IOError: # or whatever is appropriate
    yup_works_here = False

The test cases then simply test that it works for all cases

Tim Delaney


From guido@python.org  Tue Oct  8 01:59:59 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 20:59:59 -0400
Subject: [Python-Dev] RELEASED: Python 2.2.2b1
Message-ID: <200210080100.g98104x22506@pcp02138704pcs.reston01.va.comcast.net>

I'm happy to announce the release of Python 2.2.2 beta 1.  Python
2.2.2 will be a fully backwards compatible bugfix release for Python
2.2.1.  I'm shooting for a short beta test cycle: hopefully I can make
the final 2.2.2 release next week with minimal changes from 2.2.2b1.

Python 2.2.2 has a large number of bug fixes in all areas of the
interpreter and the library; for a full account see the release notes,
available via the link below.

The new release is available here:

    http://www.python.org/2.2.2/

As usual, a Windows installer and a Unix/Linux source tarball are made
available, as well as tarballs of the documentation in various forms.
At the moment, no Mac version is available, though I expect that Jack
Jansen will start working on one soon.  Likewise, no Linux RPMs are
available; I expect that Sean Reifschneider will prepare Linux RPMs
for 2.2.2 final.

I'm especially grateful to Michael Hudson, Raymond Hettinger and Neal
Norwitz for their hard work on backporting the large volume of patches
from Python 2.3.  Without them this release would not have been
possible.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct  8 02:08:12 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 21:08:12 -0400
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: Your message of "Mon, 07 Oct 2002 20:02:35 EDT."
 <Pine.LNX.4.44.0210071958510.24102-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210071958510.24102-100000@penguin.theopalgroup.com>
Message-ID: <200210080108.g9818CJ22598@pcp02138704pcs.reston01.va.comcast.net>

> On a somewhat tangental note, MoinMoin breaks with Python 2.3 CVS --
> it expects integers from some of the os/filesystem time routines
> were changed to return floats.  The fix is trivial, of course, but
> it is interesting to note where real code breaks.

I wonder if perhaps we should be more conservative here and stick to
ints.  This was discussed here and we didn't know of any software that
would break.  Now we do...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Tue Oct  8 02:12:06 2002
From: barry@python.org (Barry A. Warsaw)
Date: Mon, 7 Oct 2002 21:12:06 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <200210072305.g97N5Bq20760@pcp02138704pcs.reston01.va.comcast.net>
 <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>
Message-ID: <15778.12518.911824.400047@gargle.gargle.HOWL>

>>>>> "GE" =3D=3D Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

    GE> I have to agree that would be a better use for them.

    GE> So, I'm now reduced to suggesting

    GE>   <2/3>

Everyone's missing the obvious:

2=F73

:)
-Barry


From guido@python.org  Tue Oct  8 02:12:53 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 21:12:53 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Mon, 07 Oct 2002 18:56:09 EDT."
 <u8z19uad2.fsf@boost-consulting.com>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
 <u8z19uad2.fsf@boost-consulting.com>
Message-ID: <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>

> 2r/3 would be nicer IMO.

Or 2/3r.  (The r binds only to the 3, but of course the binary
operator rules kick in with the same effect.)

Frankly, this is the only sane notation for rationals I've seen so far
in this discussion.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Tue Oct  8 02:23:44 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Oct 2002 14:23:44 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <002301c26e62$353cc2a0$df7e4e18@brianspiv1700>
Message-ID: <200210080123.g981Nie07039@oma.cosc.canterbury.ac.nz>

Brian Quinlan <brian@sweetapp.com>:

> Is that going to be easy to parse?
> 
> >>> 1 <5/2> -3
> 1

Yes, but

>>> 1 + <5/2> * -3
  File "<string>", line 1
    1 + <5/2> * -3
        ^
SyntaxError: invalid syntax

When used as a rational literal, the <...> will always
have some operator separating it from any adjacent
expression.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Tue Oct  8 02:25:11 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Oct 2002 14:25:11 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <15778.12518.911824.400047@gargle.gargle.HOWL>
Message-ID: <200210080125.g981PBC07059@oma.cosc.canterbury.ac.nz>

barry@python.org (Barry A. Warsaw):

> Everyone's missing the obvious:
>=20
> 2=F73
>=20
> :)

Well, I was trying to limit myself to ASCII characters.

If/when we get Unicode source, though...

Greg Ewing, Computer Science Dept, +---------------------------------=
-----+
University of Canterbury,=09   | A citizen of NewZealandCorp, a=09  |
Christchurch, New Zealand=09   | wholly-owned subsidiary of USA Inc. =
 |
greg@cosc.canterbury.ac.nz=09   +------------------------------------=
--+



From dave@boost-consulting.com  Tue Oct  8 02:20:39 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 07 Oct 2002 21:20:39 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
 <u8z19uad2.fsf@boost-consulting.com>
 <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <u3crhsp3s.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

> > 2r/3 would be nicer IMO.
> 
> Or 2/3r.  (The r binds only to the 3, but of course the binary
> operator rules kick in with the same effect.)
> 
> Frankly, this is the only sane notation for rationals I've seen so far
> in this discussion.

I liked 2r/3 because it gives the sense that r/ is the rational
division operator, where // is the whatever-the-hell-it-is division
operator. I don't know if it works in the grammar to be able to say
  
  x r/ y

though. Does it?

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From greg@cosc.canterbury.ac.nz  Tue Oct  8 02:27:50 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Oct 2002 14:27:50 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210080127.g981Ro807070@oma.cosc.canterbury.ac.nz>

Guido:

> Or 2/3r.  (The r binds only to the 3, but of course the binary
> operator rules kick in with the same effect.)

That's actually not too bad! The only trouble is I keep
wanting to pronounce it as "two-thirds raw"...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Tue Oct  8 02:37:58 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 07 Oct 2002 21:37:58 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Mon, 07 Oct 2002 21:20:39 EDT."
 <u3crhsp3s.fsf@boost-consulting.com>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz> <u8z19uad2.fsf@boost-consulting.com> <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
 <u3crhsp3s.fsf@boost-consulting.com>
Message-ID: <200210080137.g981bwU23754@pcp02138704pcs.reston01.va.comcast.net>

> I liked 2r/3 because it gives the sense that r/ is the rational
> division operator, where // is the whatever-the-hell-it-is division
> operator. I don't know if it works in the grammar to be able to say
>   
>   x r/ y
> 
> though. Does it?

That would require changes to the tokenizer.

But I am against r/ on different grounds: it's not the kind of
grouping of symbols that one would expect.  People are used to 12L, 1j
and then it's a small step to 2r.  There were also precedents for
r"..." and u"...": C's w"...".  If you want a precedent for 2/, you'd
have to search in Lisp or Forth or other (nearly) grammar-less
languages.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jacobs@penguin.theopalgroup.com  Tue Oct  8 02:49:55 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 7 Oct 2002 21:49:55 -0400 (EDT)
Subject: [Python-Dev] Bug #537450 Improper object initialization for
 type.__new__
In-Reply-To: <200210080108.g9818CJ22598@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0210072143220.24577-100000@penguin.theopalgroup.com>

On Mon, 7 Oct 2002, Guido van Rossum wrote:
> > On a somewhat tangental note, MoinMoin breaks with Python 2.3 CVS --
> > it expects integers from some of the os/filesystem time routines
> > were changed to return floats.  The fix is trivial, of course, but
> > it is interesting to note where real code breaks.
> 
> I wonder if perhaps we should be more conservative here and stick to
> ints.  This was discussed here and we didn't know of any software that
> would break.  Now we do...

I don't really care much either way, and I suspect that the author(s) of
MoinMoin don't either.  I can see the theoretical benefit of floating point
timestamps, though I don't really need them (yet).  Once I do (if ever),
then I'll definitly be stumping for them.  On the other hand, the code that
broke in MoinMoin was ugly, and deserved what it got.

So maybe I do care a little...

+1 for leaving it in
+1 for getting some sleep since I'm clearly starting to hallucinate

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From dave@boost-consulting.com  Tue Oct  8 03:29:21 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 07 Oct 2002 22:29:21 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210080137.g981bwU23754@pcp02138704pcs.reston01.va.comcast.net>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
 <u8z19uad2.fsf@boost-consulting.com>
 <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
 <u3crhsp3s.fsf@boost-consulting.com>
 <200210080137.g981bwU23754@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <ur8f1r7cu.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

> > I liked 2r/3 because it gives the sense that r/ is the rational
> > division operator, where // is the whatever-the-hell-it-is division
> > operator. I don't know if it works in the grammar to be able to say
> >   
> >   x r/ y
> > 
> > though. Does it?
> 
> That would require changes to the tokenizer.
> 
> But I am against r/ on different grounds: it's not the kind of
> grouping of symbols that one would expect.  People are used to 12L, 1j
> and then it's a small step to 2r.  

You're right. And now that I look at it, if 2r is a rational with
value 2, and if you can divide ints by rationals, then 1/2r makes a
lot of sense. I wasn't looking at it that way (but I am now, and
liking it).

> There were also precedents for
> r"..." and u"...": C's w"...".  If you want a precedent for 2/, you'd
> have to search in Lisp or Forth or other (nearly) grammar-less
> languages.

Oh, please, don't remind me about those funky Forth symbols. I guess
it has less to do with grammar than with lexemes, though.

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From tim.one@comcast.net  Tue Oct  8 03:51:55 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 07 Oct 2002 22:51:55 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <200210080112.g981Crr22726@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEABBKAB.tim.one@comcast.net>

Noting that Scheme has two sets of optional numeric literal prefixes:

    #b #o #d #h  number is binary, octal, decimal (the default), or hex
    #e #i        number is exact or inexact

This avoids conflating exactness with representation, and, e.g.,

    #e#b1.001

is exactly 9/8 (although that Dr. Scheme allows a radix point in a binary
literal appears to be an extension of the Scheme std).  By default, a
numeric literal is inexect iff it contains a radix point or an exponent, but
#e or #i can override that.

> (exact? 2)
#t
> (exact? #i2)
#f


> (exact? 6.02e23)
#f
> (exact? #e6.02e23)
#t

I think this works very well.  The same rule about default exactness would
be appropriate for Python too, and an r suffix meaning what a #e prefix
means in Scheme would be a fine idea by my lights (the effect of an #i
prefix can be gotten via including a decimal point for decimal literals, and
inexact literals in other bases are rarely useful).



From Raymond Hettinger" <python@rcn.com  Tue Oct  8 05:19:15 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Tue, 8 Oct 2002 00:19:15 -0400
Subject: [Python-Dev] RELEASED: Python 2.2.2b1 -- Arrrggggggghhhh!
References: <200210080100.g98104x22506@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <005301c26e81$de469780$870fa044@oemcomputer>

Traceback (most recent call last):
  File "idle.py", line 3, in ?
    import PyShell
  File "C:\PYTHON22\Tools\idle\PyShell.py", line 16, in ?
    from EditorWindow import EditorWindow, fixwordbreaks
  File "C:\PYTHON22\Tools\idle\EditorWindow.py", line 13, in ?
    from IdleConf import idleconf
  File "C:\PYTHON22\Tools\idle\IdleConf.py", line 5, in ?
    from ConfigParser import ConfigParser, NoOptionError, NoSectionError
  File "C:\PYTHON22\lib\ConfigParser.py", line 172, in ?
    class ConfigParser:
  File "C:\PYTHON22\lib\ConfigParser.py", line 308, in ConfigParser
    _boolean_states = {'1': True, 'yes': True, 'true': True, 'on': True,
NameError: name 'True' is not defined



From Raymond Hettinger" <python@rcn.com  Tue Oct  8 05:28:05 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Tue, 8 Oct 2002 00:28:05 -0400
Subject: [Python-Dev] RELEASED: Python 2.2.2b1 -- Arrrggggggghhhh!
References: <200210080100.g98104x22506@pcp02138704pcs.reston01.va.comcast.net> <005301c26e81$de469780$870fa044@oemcomputer>
Message-ID: <005b01c26e83$19ac7960$870fa044@oemcomputer>

It looks like the windows build includes an updated
python_d.dll but leaves an out-of-date python.dll:

PYTHON22 DLL       843,833  05-08-02  2:34a python22.dll
PYTHON~1 DLL     1,744,943  08-02-02  8:55p python22_d.dll


Raymond Hettinger

----- Original Message ----- 
From: "Raymond Hettinger" <raymond.hettinger@verizon.net>
To: <python-dev@python.org>; "Guido van Rossum" <guido@python.org>
Sent: Tuesday, October 08, 2002 12:19 AM
Subject: [Python-Dev] RELEASED: Python 2.2.2b1 -- Arrrggggggghhhh!


> Traceback (most recent call last):
>   File "idle.py", line 3, in ?
>     import PyShell
>   File "C:\PYTHON22\Tools\idle\PyShell.py", line 16, in ?
>     from EditorWindow import EditorWindow, fixwordbreaks
>   File "C:\PYTHON22\Tools\idle\EditorWindow.py", line 13, in ?
>     from IdleConf import idleconf
>   File "C:\PYTHON22\Tools\idle\IdleConf.py", line 5, in ?
>     from ConfigParser import ConfigParser, NoOptionError, NoSectionError
>   File "C:\PYTHON22\lib\ConfigParser.py", line 172, in ?
>     class ConfigParser:
>   File "C:\PYTHON22\lib\ConfigParser.py", line 308, in ConfigParser
>     _boolean_states = {'1': True, 'yes': True, 'true': True, 'on': True,
> NameError: name 'True' is not defined
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 


From martin@v.loewis.de  Tue Oct  8 07:53:14 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Oct 2002 08:53:14 +0200
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: <15778.647.849183.55911@montanaro.dyndns.org>
References: <15778.647.849183.55911@montanaro.dyndns.org>
Message-ID: <m3k7ktxvz9.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> When building from CVS head, I get this warning for each file compiled:
> 
>     gcc -c -DNDEBUG -g -I. -I../Include  -DPy_BUILD_CORE -o Objects/rangeobject.o ../Objects/rangeobject.c
[...]
> I don't know what the best way to fix this is.  Should I look at making
> configure add -Wno-long-double in the proper situations or should that
> alignment field be something else in situations where the use of long double
> would generate a warning?

As a starting point, explain the problem :-) What architecture, what
compiler, and why is sizeof long double going to change?

Regards,
Martin



From mhammond@skippinet.com.au  Tue Oct  8 08:22:23 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 8 Oct 2002 17:22:23 +1000
Subject: [Python-Dev] Snapshot win32all builds of interest?
In-Reply-To: <200209301500.g8UF0mL18039@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LCEPIIGDJPKCOIHOBJEPCEFIGMAA.mhammond@skippinet.com.au>

[Me]

> > I'm wondering if there is any interest in me making regular,
> > basically untested win32all builds against the current Python CVS
> > tree?

[Guido]

> I think that would be useful, especially if the current 2.2-compatible
> win32all does not work with 2.3, or if you have added features since
> that was last released.

OK - first such snapshot done.  See the link from
starship.python.net/crew/mhammond

This version installs no shortcuts, does not compile .pyc files etc - you
are pretty much on your own.  Pythonwin\start_pythonwin.pyw is installed to
start Pythonwin, but you must do so manually.

Please let me know if you try it.

Mark.



From martin@v.loewis.de  Tue Oct  8 08:42:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Oct 2002 09:42:40 +0200
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <200210080108.g9818CJ22598@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.44.0210071958510.24102-100000@penguin.theopalgroup.com>
 <200210080108.g9818CJ22598@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3fzvhxtov.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> I wonder if perhaps we should be more conservative here and stick to
> ints.  This was discussed here and we didn't know of any software that
> would break.  Now we do...

What convinced me the last time is that time stamps were already
floats on the Mac. 

Regards,
Martin


From martin@v.loewis.de  Tue Oct  8 08:44:39 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Oct 2002 09:44:39 +0200
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <B43D149A9AB2D411971300B0D03D7E8BF0A626@natasha.auslabs.avaya.com>
References: <B43D149A9AB2D411971300B0D03D7E8BF0A626@natasha.auslabs.avaya.com>
Message-ID: <m3bs65xtlk.fsf@mira.informatik.hu-berlin.de>

"Delaney, Timothy" <tdelaney@avaya.com> writes:

> None at all. MinGW is not a Unix emulation. There is a minimal shell
> environment (MSYS) which works with MinGW (allowing use of the autoconf/make
> toolchain) but the produced binaries have no reliance on anything except the
> native win32 libraries.

To me, this is a clear indication that there is a bug in mingw
here. If they want to be a Windows compiler, they have to provide MSVC
compatibility (and they do, to some degree); this also means they have
to provide _alloca, since this is the Windows name of that feature.

Regards,
Martin



From mwh@python.net  Tue Oct  8 10:58:26 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Oct 2002 10:58:26 +0100
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: Skip Montanaro's message of "Mon, 7 Oct 2002 16:54:15 -0500"
References: <15778.647.849183.55911@montanaro.dyndns.org>
Message-ID: <2m4rbxntfh.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> When building from CVS head, I get this warning for each file compiled:

On Mac OS X, right?

>     gcc -c -DNDEBUG -g -I. -I../Include  -DPy_BUILD_CORE -o Objects/rangeobject.o ../Objects/rangeobject.c
>     In file included from ../Include/Python.h:70,
>                      from ../Objects/rangeobject.c:3:
>     ../Include/objimpl.h:252: warning: use of `long double' type; its size may change in a future release
>     ../Include/objimpl.h:252: warning: (Long double usage is reported only once for each file.
>     ../Include/objimpl.h:252: warning: To disable this warning, use -Wno-long-double.)
> 
> The culprit is the long double alignment field in:
> 
>     /* GC information is stored BEFORE the object structure. */
>     typedef union _gc_head {
>             struct {
>                     union _gc_head *gc_next;
>                     union _gc_head *gc_prev;
>                     int gc_refs;
>             } gc;
>             long double dummy;  /* force worst-case alignment */
>     } PyGC_Head;
> 
> I don't know what the best way to fix this is.  Should I look at making
> configure add -Wno-long-double in the proper situations or should that
> alignment field be something else in situations where the use of long double
> would generate a warning?

I thought that's what configure already did:

    case $ac_sys_system in
    Darwin*)
        OPT="$OPT -Wno-long-double -no-cpp-precomp";;
    esac

If I've guessed wrong and you're not on OS X, give us some help :)

Cheers,
M.

-- 
  I'm sorry, was my bias showing again? :-)
                                      -- William Tanksley, 13 May 2000


From mwh@python.net  Tue Oct  8 11:02:09 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Oct 2002 11:02:09 +0100
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: Skip Montanaro's message of "Mon, 7 Oct 2002 18:39:49 -0500"
References: <20021006115713.GB22400@lysator.liu.se> <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net> <15778.6981.362515.852029@montanaro.dyndns.org>
Message-ID: <2m1y71nt9a.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     >> Should the configure script try with /usr/ccs/bin/ar and ld if none
>     >> is explicitly supplied?
> 
> I think you should add /usr/ccs/{lib,bin,include} and /sw/{lib,bin,include}
> to places you automatically search for "useful stuff".  As Anders pointed
> out, /usr/ccs is something of a standard among the more proprietary of the
> Unix platforms.  /sw/... is where the fink gang installs all the various GNU
> goodies on MacOS X.  While you won't need to search /sw/bin for ar and ld,
> you might well find useful third-party libraries there which various wrapper
> modules can use.  When I configured XEmacs without asking it to look in
> /sw/...  it came up with a fairly feeble configuration (no jpeg or png
> support).  That said, perhaps the /sw thing will work with a simple change
> to setup.py.  I'm testing a change right now to see if my build picks up
> readline or not.

If this is your change:

Index: setup.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/setup.py,v
retrieving revision 1.111
diff -c -r1.111 setup.py
*** setup.py    25 Sep 2002 14:59:57 -0000      1.111
--- setup.py    8 Oct 2002 10:01:00 -0000
***************
*** 229,234 ****
--- 229,236 ----
          # Ensure that /usr/local is always used
          add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
          add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
+         add_dir_to_list(self.compiler.library_dirs, '/sw/lib')
+         add_dir_to_list(self.compiler.include_dirs, '/sw/include')
  
          if os.path.normpath(sys.prefix) != '/usr':
              add_dir_to_list(self.compiler.library_dirs,

It will.

This is so easy, I don't think it's worth bothering with in the
core...

Cheers,
M.

-- 
  Considering that this thread is completely on-topic in the way only
  c.l.py threads can be, I think I can say that you should replace
  "Oblivion" with "Gravity", and increase your Radiohead quotient.
                                      -- Ben Wolfson, comp.lang.python


From hughett@mercur.uphs.upenn.edu  Tue Oct  8 13:41:31 2002
From: hughett@mercur.uphs.upenn.edu (Paul Hughett)
Date: Tue, 8 Oct 2002 08:41:31 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> (message
 from Greg Ewing on Tue, 08 Oct 2002 13:18:10 +1300 (NZDT))
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>
Message-ID: <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>

Greg Ewing wrote:

> So, I'm now reduced to suggesting

>   <2/3>

How about 2r3, which could be pronounced "2 rational 3" and is syntactically
very similar to the universal 2e3?

Paul Hughett


From mal@lemburg.com  Tue Oct  8 13:51:49 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 08 Oct 2002 14:51:49 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation
 and new issues
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>
Message-ID: <3DA2D4E5.3020301@lemburg.com>

Paul Hughett wrote:
> Greg Ewing wrote:
> 
> 
>>So, I'm now reduced to suggesting
> 
> 
>>  <2/3>
> 
> 
> How about 2r3, which could be pronounced "2 rational 3" and is syntactically
> very similar to the universal 2e3?

What's so hard about rat(2,3) ?

And if that doesn't look right, simply do:

R = rat(1,1)

2/R/3

This works without any changes to the language.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From Norman Vine" <nhv@cape.com  Tue Oct  8 13:47:24 2002
From: Norman Vine" <nhv@cape.com (Norman Vine)
Date: Tue, 8 Oct 2002 08:47:24 -0400
Subject: [Python-Dev] Re: [win32] Killing MSVC's _alloca
References: <B43D149A9AB2D411971300B0D03D7E8BF0A626@natasha.auslabs.avaya.com> <m3bs65xtlk.fsf@mira.informatik.hu-berlin.de>
Message-ID: <anuk3t$rb7$1@main.gmane.org>

"Martin v. Loewis" <martin@v.loewis.de> wrote in message
news:m3bs65xtlk.fsf@mira.informatik.hu-berlin.de...
> "Delaney, Timothy" <tdelaney@avaya.com> writes:
>
> > None at all. MinGW is not a Unix emulation. There is a minimal shell
> > environment (MSYS) which works with MinGW (allowing use of the
autoconf/make
> > toolchain) but the produced binaries have no reliance on anything except
the
> > native win32 libraries.
>
> To me, this is a clear indication that there is a bug in mingw
> here. If they want to be a Windows compiler, they have to provide MSVC
> compatibility (and they do, to some degree); this also means they have
> to provide _alloca, since this is the Windows name of that feature.

#ifdef __MINGW32__
 #ifndef _alloca
  #define _alloca(x) __builtin_alloca(x)
 #endif
#endif

HTH

Norman







From guido@python.org  Tue Oct  8 14:01:16 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 09:01:16 -0400
Subject: [Python-Dev] RELEASED: Python 2.2.2b1 -- Arrrggggggghhhh!
In-Reply-To: Your message of "Tue, 08 Oct 2002 00:28:05 EDT."
 <005b01c26e83$19ac7960$870fa044@oemcomputer>
References: <200210080100.g98104x22506@pcp02138704pcs.reston01.va.comcast.net> <005301c26e81$de469780$870fa044@oemcomputer>
 <005b01c26e83$19ac7960$870fa044@oemcomputer>
Message-ID: <200210081301.g98D1H725141@pcp02138704pcs.reston01.va.comcast.net>

> It looks like the windows build includes an updated
> python_d.dll but leaves an out-of-date python.dll:
> 
> PYTHON22 DLL       843,833  05-08-02  2:34a python22.dll
> PYTHON~1 DLL     1,744,943  08-02-02  8:55p python22_d.dll

Neither of those was created by the installer (look at the dates!).
Perhaps somehow the installer couldn't write your python.dll because
it was in use?  The python22_d.dll is never written by the installer
-- we don't distribute any debug-mode files.  I'm sure the installer
is fine, the problem must be in your setup.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From list-python@ccraig.org  Tue Oct  8 14:28:27 2002
From: list-python@ccraig.org (Christopher A. Craig)
Date: 08 Oct 2002 09:28:27 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <3DA2D4E5.3020301@lemburg.com>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>
 <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>
 <3DA2D4E5.3020301@lemburg.com>
Message-ID: <whit0drres.fsf@olorin.ts.gatech.edu>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> What's so hard about rat(2,3) ?
> 
> And if that doesn't look right, simply do:
> 
> R = rat(1,1)
> 
> 2/R/3
> 
> This works without any changes to the language.

My thoughts are moving towards that.  I also wouldn't especially mind
seeing rationals added to the standard library instead of the core,
though I'm not sure how that effects Numeric.

The two best suggestions (IMHO) I've seen so far are the above and
Guido's 2/3r where that is actually the int 2 over the rational 3r.

-- 
Christopher A. Craig <list-python@ccraig.org>
"Going to school make a person educated, any more than going to a 
garage makes a person a car" Slashdot


From mwh@python.net  Tue Oct  8 14:36:28 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Oct 2002 14:36:28 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils sysconfig.py,1.44,1.44.6.1
In-Reply-To: mwh@users.sourceforge.net's message of "Mon, 07 Oct 2002 03:38:35 -0700"
References: <E17yVHX-0000pn-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2m4rbxrr1f.fsf@starship.python.net>

mwh@users.sourceforge.net writes:

> Index: sysconfig.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v
> retrieving revision 1.44
> retrieving revision 1.44.6.1
> diff -C2 -d -r1.44 -r1.44.6.1
> *** sysconfig.py	6 Dec 2001 20:51:35 -0000	1.44
> --- sysconfig.py	7 Oct 2002 10:38:33 -0000	1.44.6.1
> ***************
> *** 24,40 ****
>   EXEC_PREFIX = os.path.normpath(sys.exec_prefix)
>   
> ! # Boolean; if it's true, we're still building Python, so
> ! # we use different (hard-wired) directories.
> ! 
> ! python_build = 0
> ! 
> ! def set_python_build():
> !     """Set the python_build flag to true.
>   
> !     This means that we're building Python itself.  Only called from
> !     the setup.py script shipped with Python.
> !     """
> !     global python_build
>       python_build = 1

Hmm, I wonder if I should put a do-nothing set_python_build back in
for 2.2.2.  Does anyone call this from their build scripts?  I guess
it's possible...

Cheers,
M.

-- 
  GET   *BONK*
  BACK  *BONK*
  IN    *BONK*
  THERE *BONK*             -- Naich using the troll hammer in cam.misc


From thomas@xs4all.net  Tue Oct  8 14:46:10 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 8 Oct 2002 15:46:10 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <3DA2D4E5.3020301@lemburg.com>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <3DA2D4E5.3020301@lemburg.com>
Message-ID: <20021008134610.GE5116@xs4all.nl>

On Tue, Oct 08, 2002 at 02:51:49PM +0200, M.-A. Lemburg wrote:

> What's so hard about rat(2,3) ?

I have to agree with MAL. I know, rationally, why a rational literal (or is
it literal rational ?) is desirable, but it feels like clutter and all of
the proposed syntactic solutions strike me as bad ideas. I'd much rather
have the above (which deals with everyone else's favorite data type and
their requests to have builtin support for it at the same time) than a
not-quite-perfect way to spell a rational literally.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Tue Oct  8 14:55:49 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 8 Oct 2002 15:55:49 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>
Message-ID: <20021008135549.GF5116@xs4all.nl>

On Tue, Oct 08, 2002 at 08:41:31AM -0400, Paul Hughett wrote:

> Greg Ewing wrote:

> > So, I'm now reduced to suggesting
> 
> >   <2/3>

> How about 2r3, which could be pronounced "2 rational 3" and is syntactically
> very similar to the universal 2e3?

2e3 may be universal, I wouldn't say it's very universally used. I
personally never use the e-notation, and I can't say I find it very
readable. Who do we expect to use rationals ? Is it something we want
newbies to learn ? Will high/grade-school students with not compsci/math
background to recognize the e-notation be using rationals ? I'm not going
all happy-dayzy about rationals in the first place, but I know that is
because I understand the implementation and tradeoffs of floating point, and
they are what I want :)

I'd also say rational-literals are not that important. Looking at my own
Python code, I very rarely need a floating-point literal to start with.
Strings, plenty, dicts and lists fairly often, integers every now and then,
but floating point numbers very rarely, and almost all of them are just
'0.0' or an integer expressed as float to force float-division. Most of my
float objects come from (library) functions that return them.

If we expect newbies to prefer rationals over fp, we need syntax that is
clear to them (which may very well be just 'rat()'.) If we expect hard-core
mathers like Tim and Moshe and Chris Tismer and even Guido to use them, I'd
say we don't need syntax support for them and can just live with rat().

And I have to say some of the proposals for literal rationals I saw were
very disturbingly sick. Some of you need help, and some need to spend less
time using XML :-)

Beautiful-is-better-than-ug'ly y'rs,
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From neal@metaslash.com  Tue Oct  8 14:58:32 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 8 Oct 2002 09:58:32 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
In-Reply-To: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>
References: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021008135832.GA22587@epoch.metaslash.com>

On Mon, Oct 07, 2002 at 05:45:59PM -0400, Guido van Rossum wrote:
> I've committed the last changes for Python 2.2.2b1 -- the Misc/NEWS
> file.  I'm now going to follow the rest of the PEP 102 script until
> the release is done.  In the mean time, please no more checkins to the
> 2.2 branch!

I know of 2 problems.  Both are simple and low-risk to fix.  In NEWS,
alrady is misspelled.  tstate->tick_counter is not initialized in
Python/pystate.c line 138.

If I remember correct, there is some inconsistent use/doc of
True/False vs 1/0 in ConfigParser.py.  I don't think this is worth
fixing in 2.2.2.

I valgrind'ed 2.2.2 on Sept 26 and all was clean.

Neal


From skip@pobox.com  Tue Oct  8 14:49:18 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 08:49:18 -0500
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: <m3k7ktxvz9.fsf@mira.informatik.hu-berlin.de>
References: <15778.647.849183.55911@montanaro.dyndns.org>
 <m3k7ktxvz9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15778.57950.590904.335471@montanaro.dyndns.org>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> Skip Montanaro <skip@pobox.com> writes:
    >> When building from CVS head, I get this warning for each file compiled:

    Martin> As a starting point, explain the problem :-) What architecture,
    Martin> what compiler, and why is sizeof long double going to change?

Sorry.  MacOS 10.2.  Apple's gcc version of 3.1.  I have no idea why it's
going to change.  I'm just the messenger. ;-)

Skip




From guido@python.org  Tue Oct  8 15:43:27 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 10:43:27 -0400
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils sysconfig.py,1.44,1.44.6.1
In-Reply-To: Your message of "Tue, 08 Oct 2002 14:36:28 BST."
 <2m4rbxrr1f.fsf@starship.python.net>
References: <E17yVHX-0000pn-00@usw-pr-cvs1.sourceforge.net>
 <2m4rbxrr1f.fsf@starship.python.net>
Message-ID: <200210081443.g98EhRj12368@odiug.zope.com>

> Hmm, I wonder if I should put a do-nothing set_python_build back in
> for 2.2.2.  Does anyone call this from their build scripts?  I guess
> it's possible...

Good idea.  better safe than sorry.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Tue Oct  8 15:45:02 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 10:45:02 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz>
 <200210081241.g98CfVA23876@mercur.uphs.upenn.edu>
 <3DA2D4E5.3020301@lemburg.com>
 <20021008134610.GE5116@xs4all.nl>
Message-ID: <15778.61294.171669.963839@gargle.gargle.HOWL>

>>>>> "TW" == Thomas Wouters <thomas@xs4all.net> writes:

    TW> I have to agree with MAL. I know, rationally, why a rational
    TW> literal (or is it literal rational ?) is desirable, but it
    TW> feels like clutter and all of the proposed syntactic solutions
    TW> strike me as bad ideas. I'd much rather have the above (which
    TW> deals with everyone else's favorite data type and their
    TW> requests to have builtin support for it at the same time) than
    TW> a not-quite-perfect way to spell a rational literally.

It seems to me that Python has a tradition of deferring syntax
decisions until way after the more important issues have been
worked out.  Perhaps we should do the same here, IOW, get the rational
library into the core and see if a rational literal makes that big a
difference for readability or maintainability.  It may not, but then
at least we'll still have rationals.

-Barry


From jeremy@alum.mit.edu  Tue Oct  8 16:16:51 2002
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 8 Oct 2002 11:16:51 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
In-Reply-To: <20021008135832.GA22587@epoch.metaslash.com>
References: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>
 <20021008135832.GA22587@epoch.metaslash.com>
Message-ID: <15778.63203.959350.840343@slothrop.zope.com>

>>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:

  NN> I valgrind'ed 2.2.2 on Sept 26 and all was clean.

Shouldn't that be valground?

Jeremy



From hughett@mercur.uphs.upenn.edu  Tue Oct  8 15:56:34 2002
From: hughett@mercur.uphs.upenn.edu (Paul Hughett)
Date: Tue, 8 Oct 2002 10:56:34 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021008135549.GF5116@xs4all.nl> (message from Thomas Wouters on
 Tue, 8 Oct 2002 15:55:49 +0200)
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <20021008135549.GF5116@xs4all.nl>
Message-ID: <200210081456.g98EuYx24034@mercur.uphs.upenn.edu>

Paul Hughett wrote:

>> How about 2r3, which could be pronounced "2 rational 3" and is syntactically
>> very similar to the universal 2e3?

Thomas Wouters wrote:

> 2e3 may be universal, I wouldn't say it's very universally used. I
> personally never use the e-notation, and I can't say I find it very
>readable

I think I was a bit misleading there.  I mentioned the similarity to 2e3
as an argument that it would be easy to modify the lexer and parser to
handle 2r3.  I doubt that we want to inflict that analogy on the learner;
if nothing else, they'd keep trying to interpret the 3 as an exponent.


> I'd also say rational-literals are not that important.

This may be the key decision.  Do we really need rational literals, or
can we live with constructors?  Using rat(2,3) seems fine to me as the
constructor, and I'm not all that enamoured of rational literals.  On
the other hand, I would hardly ever use rationals; I regard them as a
specialized tool akin to symbolic algebra (to which they are arguably
essential).  If we do want rational literals, well then 2r3 seems a
reasonably painless way to get them.


> If we expect newbies to prefer rationals over fp...

I don't.  I'd say the right thing for newbies is decimal floating point,
rather than rationals.  Not that I object to adding rationals to Python;
they're a specialized tool, but a powerful one within their niche.


Paul Hughett


From thomas@xs4all.net  Tue Oct  8 15:56:30 2002
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 8 Oct 2002 16:56:30 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <15778.61294.171669.963839@gargle.gargle.HOWL>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <3DA2D4E5.3020301@lemburg.com> <20021008134610.GE5116@xs4all.nl> <15778.61294.171669.963839@gargle.gargle.HOWL>
Message-ID: <20021008145630.GX621@xs4all.nl>

On Tue, Oct 08, 2002 at 10:45:02AM -0400, Barry A. Warsaw wrote:

> It seems to me that Python has a tradition of deferring syntax
> decisions until way after the more important issues have been
> worked out.

I'd agree, except that I don't think there are any issues more important
than syntax :) I know what you mean, though, and I do agree with that.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Tue Oct  8 15:52:22 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 10:52:22 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
In-Reply-To: Your message of "Tue, 08 Oct 2002 09:58:32 EDT."
 <20021008135832.GA22587@epoch.metaslash.com>
References: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>
 <20021008135832.GA22587@epoch.metaslash.com>
Message-ID: <200210081452.g98EqNL14643@odiug.zope.com>

> I know of 2 problems.  Both are simple and low-risk to fix.  In NEWS,
> alrady is misspelled.  tstate->tick_counter is not initialized in
> Python/pystate.c line 138.

Fixed -- in time for the final 2.2.2 release.

> If I remember correct, there is some inconsistent use/doc of
> True/False vs 1/0 in ConfigParser.py.  I don't think this is worth
> fixing in 2.2.2.

Definitely not.

> I valgrind'ed 2.2.2 on Sept 26 and all was clean.

Thanks!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct  8 16:09:36 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 11:09:36 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Mon, 07 Oct 2002 22:51:55 EDT."
 <LNBBLJKPBEHFEDALKOLCOEABBKAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEABBKAB.tim.one@comcast.net>
Message-ID: <200210081509.g98F9aU15528@odiug.zope.com>

[Tim]
> Noting that Scheme has two sets of optional numeric literal prefixes:
[...]
> This avoids conflating exactness with representation, and, e.g.,
{...]
> By default, a numeric literal is inexect iff it contains a radix
> point or an exponent, but #e or #i can override that.
[...]
> I think this works very well.  The same rule about default exactness
> would be appropriate for Python too, and an r suffix meaning what a
> #e prefix means in Scheme would be a fine idea by my lights (the
> effect of an #i prefix can be gotten via including a decimal point
> for decimal literals, and inexact literals in other bases are rarely
> useful).

I don't think Python needs the full matrix of exact and inexact
versions of all kind of numbers.  I see not enough need for inexact
ints or rationals, nor for exact floats or complex numbers.  So I'd
like to continue our partition of numeric types as follows:

         exact       |     inexact
 ------------------------------------
  int/long  rational | float  complex

But we can do this, which is pretty much what Tim proposes in the end:

  1   ->  int
  1.0 ->  float

  1r  ->  rational
  1.0r -> rational

If we ever add a fixed-point decimal type, that could use 'f' for a
suffix.  If we ever add a floating-point decimal type (like the one
Aahz is working on -- I mistakenly called it a fixed-point type
before) then it could use 'd' as a suffix, or it could become the
default and we could use 'b' as a suffix to get binary floating point.

But I also agree with a recent trend in this thread: let's not rush
to add syntax.  Let's first add rationals to the library.

I hereby declare this thread closed.  (Ha, ha. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Oct  8 16:10:36 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 10:10:36 -0500
Subject: [Python-Dev] Bug #537450 Improper object initialization for type.__new__
In-Reply-To: <15778.7975.178103.577482@montanaro.dyndns.org>
References: <Pine.LNX.4.44.0210061626450.29004-100000@penguin.theopalgroup.com>
 <200210062057.g96KvaG29958@pcp02138704pcs.reston01.va.comcast.net>
 <15778.7975.178103.577482@montanaro.dyndns.org>
Message-ID: <15778.62828.970616.101696@montanaro.dyndns.org>

    Guido> I tried to set up an email subscription but the Wiki isn't
    Guido> configured to support that...

    Skip> That would be my mistake.  

Now enabled.

Skip


From skip@pobox.com  Tue Oct  8 16:14:20 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 10:14:20 -0500
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <u8z19uad2.fsf@boost-consulting.com>
References: <200210072238.g97McTk06649@oma.cosc.canterbury.ac.nz>
 <u8z19uad2.fsf@boost-consulting.com>
Message-ID: <15778.63052.576188.619350@montanaro.dyndns.org>

    David> 2r/3 would be nicer IMO.

I like this the best of those I've seen so far.  I don't see any reason that
rational literals need to contain a division symbol, so "2r" makes a fine
rational.  What about "2.5r"?  Can that just be expanded to "25r/10" by the
lexical analyzer?

Skip


From barry@python.org  Tue Oct  8 16:15:19 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 11:15:19 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
References: <200210072145.g97LjxG12739@pcp02138704pcs.reston01.va.comcast.net>
 <20021008135832.GA22587@epoch.metaslash.com>
 <15778.63203.959350.840343@slothrop.zope.com>
Message-ID: <15778.63111.912539.824214@gargle.gargle.HOWL>

>>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:

    NN> I valgrind'ed 2.2.2 on Sept 26 and all was clean.

>>>>> "JH" == Jeremy Hylton <jeremy@alum.mit.edu> writes:

    JH> Shouldn't that be valground?

The technical term is "valgrunged".


From Richie Hindle" <richie@entrian.com  Tue Oct  8 16:19:11 2002
From: Richie Hindle" <richie@entrian.com (Richie Hindle" <richie@entrian.com)
Date: Tue, 08 Oct 2002 16:19:11 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
Message-ID: <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net>

Hi,

First a brief introduction, since this is my first post to python-dev: I'm a professional software engineer, mostly using C/C++ and Python.  I use Python a lot for my own projects and some of you will know me from the spambayes project, as the author of pop3proxy.py.

I was interested to see Armin's patch [617312] to "allow the C profile and trace functions to alter some fields of the current PyFrameObject".

I'm writing a Python debugger, and as Armin says, his patch "could also be used by advanced debuggers to allow the execution point to be modified by the user."  A "Set-next-statement" feature is on my list of nice-to-have-but-hard-to-do jobs, and this patch could make it easy.  (I know the blockstack presents a problem, but if it's insurmountable I'm happy to ignore that and to only allow skipping to lines in the same block - 90% of my use of the equivalent feature in MSVC is to skip back one line or forward one line.)

There's one fly in the ointment - I'm trying to keep my debugger in pure Python, and Armin's patch only applies to C trace functions.  If frame.f_lasti were writable by Python trace functions, pure Python debuggers (including pdb) could implement Set-Next-Statement.

Here's a small script that demonstrates what I mean, and the one-line patch to frameobject.c that enables it:

-----------------------------------------------------------------------

"""Demonstration of debugger 'Set-next-statement' feature.  Requires
Python 2.2.2 with writeable frame.f_lasti.  Prints "1, 2, 2, 3"."""

import sys, pprint

def example():
    print 1,
    print 2,
    print 3    # Line 9; we skip back to 8 the first time we hit this.

class DebuggerHook:
    def __init__(self):
        self.done = False   # Have we done the Set-next-statement yet?
        self.lasti = 0      # The frame.f_lasti of the previous line.
        sys.settrace(self.trace)

    def trace(self, frame, action, arg):
        if not self.done and frame.f_lineno == 9:
            # This is the first time we've hit line 9; skip back to 8.
            frame.f_lasti = self.lasti
            self.done = True
        else:
            # Store the instruction offset of this line.
            self.lasti = frame.f_lasti
        return self.trace

debugger = DebuggerHook()
example()  # Prints "1, 2, 2, 3"

-----------------------------------------------------------------------

*** frameobject.c       Tue Oct  8 08:16:39 2002
--- frameobject-222b1.c Tue Oct  8 08:16:12 2002
***************
*** 16,20 ****
        {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
        {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
!       {"f_lasti",     T_INT,          OFF(f_lasti)},
        {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
        {"f_restricted",T_INT,          OFF(f_restricted),RO},
--- 16,20 ----
        {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
        {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
!       {"f_lasti",     T_INT,          OFF(f_lasti),   RO},
        {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
        {"f_restricted",T_INT,          OFF(f_restricted),RO},

-----------------------------------------------------------------------

For completeness, f_lineno should probably be writable as well, but I'm keeping the changes minimal for the purposes of discussion.

Is this a reasonable suggestion for 2.3, or is it giving people too much rope?  A nasty consequence is that you can write Python code that causes Python to seg-fault, but you have to be doing some fairly advanced stuff for that to happen.  I would say that the power of a Set-next-statement feature in pdb and other debuggers is worth the price, but others may disagree...?

-- 
Richie Hindle
richie@entrian.com




From mwh@python.net  Tue Oct  8 16:27:16 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Oct 2002 16:27:16 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: richie@entrian.com's message of "Tue, 08 Oct 2002 16:19:11 +0100"
References: <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net>
Message-ID: <2m4rbxne7f.fsf@starship.python.net>

richie@entrian.com writes:

> I was interested to see Armin's patch [617312] to "allow the C
> profile and trace functions to alter some fields of the current
> PyFrameObject".
> 
> I'm writing a Python debugger, and as Armin says, his patch "could
> also be used by advanced debuggers to allow the execution point to
> be modified by the user."  A "Set-next-statement" feature is on my
> list of nice-to-have-but-hard-to-do jobs, and this patch could make
> it easy.  (I know the blockstack presents a problem, but if it's
> insurmountable I'm happy to ignore that and to only allow skipping
> to lines in the same block - 90% of my use of the equivalent feature
> in MSVC is to skip back one line or forward one line.)
> 
> There's one fly in the ointment - I'm trying to keep my debugger in
> pure Python, and Armin's patch only applies to C trace functions.
> If frame.f_lasti were writable by Python trace functions, pure
> Python debuggers (including pdb) could implement Set-Next-Statement.
> 
> Here's a small script that demonstrates what I mean, and the
> one-line patch to frameobject.c that enables it:
[...]
> For completeness, f_lineno should probably be writable as well, but
> I'm keeping the changes minimal for the purposes of discussion.

I'd prefer a different approach; make f_lasti a descriptor so you can
do some sanity checking on the values (i.e. fits in the
f_code.co_code, actually is an opcode boundary and sets f_lineno).

> Is this a reasonable suggestion for 2.3, or is it giving people too
> much rope?  A nasty consequence is that you can write Python code
> that causes Python to seg-fault, but you have to be doing some
> fairly advanced stuff for that to happen.  
> 
> I would say that the power of a Set-next-statement feature in pdb
> and other debuggers is worth the price, but others may disagree...?

Hmm.  Not sure.  Given all the ways Armin's just found to make the
interpreter crap out, one more doesn't seem that exciting -- but
perhaps this is the wrong attitude...

Cheers,
M.

-- 
  I'm not sure that the ability to create routing diagrams 
  similar to pretzels with mad cow disease is actually a 
  marketable skill.                                     -- Steve Levin
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From Richie Hindle" <richie@entrian.com  Tue Oct  8 16:31:49 2002
From: Richie Hindle" <richie@entrian.com (Richie Hindle" <richie@entrian.com)
Date: Tue, 08 Oct 2002 16:31:49 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net>
Message-ID: <E17ywKr-0003UJ-0U@anchor-post-39.mail.demon.net>

Hi,

[Hell's bells, sorry, here's this again with the line-breaks fixed!
How embarrassing...]

First a brief introduction, since this is my first post to python-dev:
I'm a professional software engineer, mostly using C/C++ and Python.  I
use Python a lot for my own projects and some of you will know me from
the spambayes project, as the author of pop3proxy.py.

I was interested to see Armin's patch [617312] to "allow the C profile
and trace functions to alter some fields of the current PyFrameObject".

I'm writing a Python debugger, and as Armin says, his patch "could also
be used by advanced debuggers to allow the execution point to be
modified by the user."  A "Set-next-statement" feature is on my list of
nice-to-have-but-hard-to-do jobs, and this patch could make it easy.  (I
know the blockstack presents a problem, but if it's insurmountable I'm
happy to ignore that and to only allow skipping to lines in the same
block - 90% of my use of the equivalent feature in MSVC is to skip back
one line or forward one line.)

There's one fly in the ointment - I'm trying to keep my debugger in pure
Python, and Armin's patch only applies to C trace functions.  If
frame.f_lasti were writable by Python trace functions, pure Python
debuggers (including pdb) could implement Set-Next-Statement.

Here's a small script that demonstrates what I mean, and the one-line
patch to frameobject.c that enables it:

-----------------------------------------------------------------------

"""Demonstration of debugger 'Set-next-statement' feature.  Requires
Python 2.2.2 with writeable frame.f_lasti.  Prints "1, 2, 2, 3"."""

import sys, pprint

def example():
    print 1,
    print 2,
    print 3    # Line 9; we skip back to 8 the first time we hit this.

class DebuggerHook:
    def __init__(self):
        self.done = False   # Have we done the Set-next-statement yet?
        self.lasti = 0      # The frame.f_lasti of the previous line.
        sys.settrace(self.trace)

    def trace(self, frame, action, arg):
        if not self.done and frame.f_lineno == 9:
            # This is the first time we've hit line 9; skip back to 8.
            frame.f_lasti = self.lasti
            self.done = True
        else:
            # Store the instruction offset of this line.
            self.lasti = frame.f_lasti
        return self.trace

debugger = DebuggerHook()
example()  # Prints "1, 2, 2, 3"

-----------------------------------------------------------------------

*** frameobject.c       Tue Oct  8 08:16:39 2002
--- frameobject-222b1.c Tue Oct  8 08:16:12 2002
***************
*** 16,20 ****
        {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
        {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
!       {"f_lasti",     T_INT,          OFF(f_lasti)},
        {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
        {"f_restricted",T_INT,          OFF(f_restricted),RO},
--- 16,20 ----
        {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
        {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
!       {"f_lasti",     T_INT,          OFF(f_lasti),   RO},
        {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
        {"f_restricted",T_INT,          OFF(f_restricted),RO},

-----------------------------------------------------------------------

For completeness, f_lineno should probably be writable as well, but I'm
keeping the changes minimal for the purposes of discussion.

Is this a reasonable suggestion for 2.3, or is it giving people too much
rope?  A nasty consequence is that you can write Python code that causes
Python to seg-fault, but you have to be doing some fairly advanced stuff
for that to happen.  I would say that the power of a Set-next-statement
feature in pdb and other debuggers is worth the price, but others may
disagree...?

-- 
Richie Hindle
richie@entrian.com




From guido@python.org  Tue Oct  8 16:42:18 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 11:42:18 -0400
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: Your message of "Tue, 08 Oct 2002 16:19:11 BST."
 <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net>
References: <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net>
Message-ID: <200210081542.g98FgIn15739@odiug.zope.com>

> *** frameobject.c       Tue Oct  8 08:16:39 2002
> --- frameobject-222b1.c Tue Oct  8 08:16:12 2002
> ***************
> *** 16,20 ****
>         {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
>         {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
> !       {"f_lasti",     T_INT,          OFF(f_lasti)},
>         {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
>         {"f_restricted",T_INT,          OFF(f_restricted),RO},
> --- 16,20 ----
>         {"f_builtins",  T_OBJECT,       OFF(f_builtins),RO},
>         {"f_globals",   T_OBJECT,       OFF(f_globals), RO},
> !       {"f_lasti",     T_INT,          OFF(f_lasti),   RO},
>         {"f_lineno",    T_INT,          OFF(f_lineno),  RO},
>         {"f_restricted",T_INT,          OFF(f_restricted),RO},

A backwards diff!  He posted a backwards diff!  Aargh! :-)

> Is this a reasonable suggestion for 2.3, or is it giving people too
> much rope?  A nasty consequence is that you can write Python code
> that causes Python to seg-fault, but you have to be doing some
> fairly advanced stuff for that to happen.  I would say that the
> power of a Set-next-statement feature in pdb and other debuggers is
> worth the price, but others may disagree...?

I can't approve a Python-level feature that explicitly opens up a
loophole of this kind.  But if you add a C "getset" wrapper that
allows setting this only to safe values (i.e. within the same block
and pointing at the beginning of a bytecode) it might be deemed safe
enough.  It should probably silently update f_lineno as well.  Spare
no cost.  It's up to you to write the patch, submit it to SF, and
prove that it's safe.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From dave@boost-consulting.com  Tue Oct  8 16:37:46 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 08 Oct 2002 11:37:46 -0400
Subject: [Python-Dev] read-only properties
Message-ID: <u3crhkkl1.fsf@boost-consulting.com>

I was just working on some docs for Boost.Python and thought I'd write
down what happens when you try to write to a read-only property (I use
properties to expose C++ data members among other things). So:

    >>> class X(object):
    ...    def get(self): return 'x'
    ...    l = property(get)
    ...
    >>> x = X()
    >>> x.l
    'x'
    >>> x.l = 1
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    AttributeError: can't set attribute

This struck me as a very poor error message, given that mechanism in
Boost.Python v1 used to say:

    Traceback (innermost last):
      File "<string>", line 1, in ?
    AttributeError: 'l' attribute is read-only

Is it worth doing something about this?

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From tim@zope.com  Tue Oct  8 16:55:24 2002
From: tim@zope.com (Tim Peters)
Date: Tue, 8 Oct 2002 11:55:24 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210081509.g98F9aU15528@odiug.zope.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEIHDLAA.tim@zope.com>

[Guido]
> I don't think Python needs the full matrix of exact and inexact
> versions of all kind of numbers.  I see not enough need for inexact
> ints or rationals, nor for exact floats or complex numbers.

#i3 is "an inexact integer" in Scheme, but that says nothing about how it's
*implemented*.  In fact, it's almost certainly implemented as a float, much
as #e3.0 is almost certainly implemented as an integer.  Scheme hides the
internal implementation, so that the programmer doesn't need to care; the
flip side is that programmers who do care can't force the issue in a
portable way.

> So I'd like to continue our partition of numeric types as follows:
>
>          exact       |     inexact
>  ------------------------------------
>   int/long  rational | float  complex
>
> But we can do this, which is pretty much what Tim proposes in the end:
>
>   1   ->  int
>   1.0 ->  float
>
>   1r  ->  rational
>   1.0r -> rational

Yes, that's exactly what I was proposing.  s/r/#i/ and it's the same as
Scheme's rules, although Scheme doesn't promise anything about internal
representation.

> If we ever add a fixed-point decimal type, that could use 'f' for a
> suffix.  If we ever add a floating-point decimal type (like the one
> Aahz is working on -- I mistakenly called it a fixed-point type
> before) then it could use 'd' as a suffix, or it could become the
> default and we could use 'b' as a suffix to get binary floating point.

Etc.

> But I also agree with a recent trend in this thread: let's not rush
> to add syntax.  Let's first add rationals to the library.
>
> I hereby declare this thread closed.  (Ha, ha. :-)

You should have put that sentence first -- my editor still doesn't go
backwards <wink>.



From skip@pobox.com  Tue Oct  8 16:59:21 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 10:59:21 -0500
Subject: [Python-Dev] Re: [snake-farm] Autodetect /usr/ccs/bin?
In-Reply-To: <2m1y71nt9a.fsf@starship.python.net>
References: <20021006115713.GB22400@lysator.liu.se>
 <200210061329.g96DTOs14718@pcp02138704pcs.reston01.va.comcast.net>
 <15778.6981.362515.852029@montanaro.dyndns.org>
 <2m1y71nt9a.fsf@starship.python.net>
Message-ID: <15779.217.866575.83673@montanaro.dyndns.org>

    Michael> If this is your change:

    ...
    Michael> +         add_dir_to_list(self.compiler.library_dirs, '/sw/lib')
    Michael> +         add_dir_to_list(self.compiler.include_dirs, '/sw/include')
  
yes, but prefaced with

    if sys.platform == "darwin":

    Michael> This is so easy, I don't think it's worth bothering with in the
    Michael> core...

What's "the core" mean in this context?  Do you mean that people should be
expected to add this to setup.py themselves?

Skip


From oren-py-d@hishome.net  Tue Oct  8 17:03:51 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Tue, 8 Oct 2002 12:03:51 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021008135549.GF5116@xs4all.nl>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <20021008135549.GF5116@xs4all.nl>
Message-ID: <20021008160351.GA20990@hishome.net>

On Tue, Oct 08, 2002 at 03:55:49PM +0200, Thomas Wouters wrote:
> I'd also say rational-literals are not that important. Looking at my own
> Python code, I very rarely need a floating-point literal to start with.
> Strings, plenty, dicts and lists fairly often, integers every now and then,
> but floating point numbers very rarely, and almost all of them are just
> '0.0' or an integer expressed as float to force float-division. Most of my
> float objects come from (library) functions that return them.

Having nice-looking literals is important even if they are not actually
typed in the source code too often.  The literal form is also the repr()
for all built-in numeric types so far. I don't think we should break that.

What would your like to see as the repr() of a rational number? The answer
to this will also determine what you type in your source.

Note that repr(n) is not necessarily str(n):

>>> repr(f)
'0.59999999999999998'
>>> str(f)
'0.6'

So we could have:

>>> repr(r)
'3/5r'    # or 'rat(3, 6)'
>>> str(r)
'3/5'

	Oren


From guido@python.org  Tue Oct  8 17:09:57 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 12:09:57 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Tue, 08 Oct 2002 12:03:51 EDT."
 <20021008160351.GA20990@hishome.net>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <20021008135549.GF5116@xs4all.nl>
 <20021008160351.GA20990@hishome.net>
Message-ID: <200210081609.g98G9v516159@odiug.zope.com>

> So we could have:
> 
> >>> repr(r)
> '3/5r'    # or 'rat(3, 6)'

Surely you meant 'rat(3, 5)'. :-)

> >>> str(r)
> '3/5'

I'd like at least one of those return '0.6' or '0.6r'.  I think str(r)
should return '0.6', and then repr(r) could return '3/5r'.  For values
that require approximation as decimal, I'd say use the same number of
digits that str() of a float currently uses (about 12 I believe).  So
str(1/3r) should be '0.333333333333'.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jepler@unpythonic.net  Tue Oct  8 17:47:28 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 8 Oct 2002 11:47:28 -0500
Subject: [Python-Dev] read-only properties
In-Reply-To: <u3crhkkl1.fsf@boost-consulting.com>
References: <u3crhkkl1.fsf@boost-consulting.com>
Message-ID: <20021008164723.GA22918@unpythonic.net>

On Tue, Oct 08, 2002 at 11:37:46AM -0400, David Abrahams wrote:
[...]
>     AttributeError: can't set attribute
> 
> This struck me as a very poor error message, given that mechanism in
> Boost.Python v1 used to say:
[...]
>     AttributeError: 'l' attribute is read-only
> 
> Is it worth doing something about this?

The following comes close, in Python, but requires a setter object for
each attribute:

    class SetROProperty:
        def __init__(self, name):
            self.name = name
        def __call__(self, obj, val):
            raise AttributeError, "%r attribute is read-only" % self.name

    class X(object):
        def get(self): return 'x'
        l = property(get, SetROProperty('l'))

    >>> x = X()
    >>> x.l
    'x'
    >>> x.l = 1
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
      File "srop.py", line 5, in __call__
        raise AttributeError, "%r attribute is read-only" % self.name
    AttributeError: 'l' attribute is read-only

The key seems to be that property() doesn't know the name the property
has.

Jeff


From richie@entrian.com  Tue Oct  8 17:56:15 2002
From: richie@entrian.com (Richie Hindle)
Date: Tue, 08 Oct 2002 17:56:15 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: <200210081542.g98FgIn15739@odiug.zope.com>
References: <E17yw8d-0006du-0U@anchor-post-35.mail.demon.net> <200210081542.g98FgIn15739@odiug.zope.com>
Message-ID: <n936qu06gl2bighkih1abf88sl8l31pnab@4ax.com>

> A backwards diff!  He posted a backwards diff!  Aargh! :-)

Oops, mental note well and truly made...

> if you add a C "getset" wrapper that
> allows setting this only to safe values (i.e. within the same block
> and pointing at the beginning of a bytecode) it might be deemed safe
> enough.  It should probably silently update f_lineno as well.

OK, I'll have a look at doing that, and at creating a suitably convincing
test suite.

-- 
Richie Hindle
richie@entrian.com



From skip@pobox.com  Tue Oct  8 18:18:55 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 12:18:55 -0500
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: <2m4rbxntfh.fsf@starship.python.net>
References: <15778.647.849183.55911@montanaro.dyndns.org>
 <2m4rbxntfh.fsf@starship.python.net>
Message-ID: <15779.4991.649997.288990@montanaro.dyndns.org>

    Michael> I thought that's what configure already did:

    Michael>     case $ac_sys_system in
    Michael>     Darwin*)
    Michael>         OPT="$OPT -Wno-long-double -no-cpp-precomp";;
    Michael>     esac

    Michael> If I've guessed wrong and you're not on OS X, give us some help :)

You guessed right.  In my situation, I generally configure with OPT=-O3 or
OPT=-g, both of which prevent execution of this bit of code.  Perhaps we
should modify it so this is executed unconditionally.

Skip


From guido@python.org  Tue Oct  8 18:27:35 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 13:27:35 -0400
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: Your message of "Tue, 08 Oct 2002 12:18:55 CDT."
 <15779.4991.649997.288990@montanaro.dyndns.org>
References: <15778.647.849183.55911@montanaro.dyndns.org> <2m4rbxntfh.fsf@starship.python.net>
 <15779.4991.649997.288990@montanaro.dyndns.org>
Message-ID: <200210081727.g98HRZJ16526@odiug.zope.com>

>     Michael> I thought that's what configure already did:
> 
>     Michael>     case $ac_sys_system in
>     Michael>     Darwin*)
>     Michael>         OPT="$OPT -Wno-long-double -no-cpp-precomp";;
>     Michael>     esac
> 
>     Michael> If I've guessed wrong and you're not on OS X, give us some help :)
> 
> You guessed right.  In my situation, I generally configure with OPT=-O3 or
> OPT=-g, both of which prevent execution of this bit of code.  Perhaps we
> should modify it so this is executed unconditionally.

That's a longstanding nit: $OPT is used to specify some standard
things as well as giving you a hook to override.  I think we should
introduce a separate variable.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Oct  8 18:38:24 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 12:38:24 -0500
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: <200210081727.g98HRZJ16526@odiug.zope.com>
References: <15778.647.849183.55911@montanaro.dyndns.org>
 <2m4rbxntfh.fsf@starship.python.net>
 <15779.4991.649997.288990@montanaro.dyndns.org>
 <200210081727.g98HRZJ16526@odiug.zope.com>
Message-ID: <15779.6160.714341.875991@montanaro.dyndns.org>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    Guido> That's a longstanding nit: $OPT is used to specify some standard
    Guido> things as well as giving you a hook to override.  I think we
    Guido> should introduce a separate variable.

That makes good sense to me.  For the purposes of backward compatibility, I
suppose we shouldn't change OPT as the variable people set to tweak
settings.  I'll create a new BASEOPT variable and submit a patch.

Skip


From guido@python.org  Tue Oct  8 18:51:30 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 13:51:30 -0400
Subject: [Python-Dev] Warning about use of long double - what's the correct approach?
In-Reply-To: Your message of "Tue, 08 Oct 2002 12:38:24 CDT."
 <15779.6160.714341.875991@montanaro.dyndns.org>
References: <15778.647.849183.55911@montanaro.dyndns.org> <2m4rbxntfh.fsf@starship.python.net> <15779.4991.649997.288990@montanaro.dyndns.org> <200210081727.g98HRZJ16526@odiug.zope.com>
 <15779.6160.714341.875991@montanaro.dyndns.org>
Message-ID: <200210081751.g98HpUU16615@odiug.zope.com>

>     Guido> That's a longstanding nit: $OPT is used to specify some
>     Guido> standard things as well as giving you a hook to override.
>     Guido> I think we should introduce a separate variable.
> 
> That makes good sense to me.  For the purposes of backward
> compatibility, I suppose we shouldn't change OPT as the variable
> people set to tweak settings.  I'll create a new BASEOPT variable
> and submit a patch.

OK!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct  8 19:25:59 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 14:25:59 -0400
Subject: [Python-Dev] read-only properties
In-Reply-To: Your message of "Tue, 08 Oct 2002 11:47:28 CDT."
 <20021008164723.GA22918@unpythonic.net>
References: <u3crhkkl1.fsf@boost-consulting.com>
 <20021008164723.GA22918@unpythonic.net>
Message-ID: <200210081826.g98IPx516833@odiug.zope.com>

> The key seems to be that property() doesn't know the name the
> property has.

Oops, you're right.  This makes it practically impossible to improve
the error message (without the kind of trick that Jeff shows).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim@zope.com  Tue Oct  8 21:15:16 2002
From: tim@zope.com (Tim Peters)
Date: Tue, 8 Oct 2002 16:15:16 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: <m3adlq5a1n.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEJHDLAA.tim@zope.com>

[Guido, on the list(xrange(maxint // 4)) test]
> That's strange -- that test is specifically designed to fail before it
> ever gets to allocating memory.  Can you track this down with a
> debugger?

[Martin]
> Notice that this test does not strictly achieve that:

This is true, although I'd say it plain doesn't achieve that, and leave
"strictly" out of it.

> with a four-byte pointer, you attempt to allocate 0x20000000L items (due
> to roundupsize), which is small enough to try a realloc. In turn, it
> tries to allocate 0x80000000L bytes, i.e. 2GB. On a 32-bit machine, it
> is possible to allocate that much memory.

Depending on platform, of course.  Under MSVC6, it does call the platform
realloc(), and it's the latter that returns a NULL pointer (the max you can
alloc under Win32 is 0x7ffdefff).

If we changed the test to use maxint // 2 instead, it would (just barely)
trigger the

    _new_size <= ((~(size_t)0) / sizeof(type))

test in NRESIZE instead, and keep the platform realloc() out of it.

Good enough?



From guido@python.org  Tue Oct  8 21:25:06 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 16:25:06 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: Your message of "Tue, 08 Oct 2002 16:15:16 EDT."
 <BIEJKCLHCIOIHAGOKOLHMEJHDLAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHMEJHDLAA.tim@zope.com>
Message-ID: <200210082025.g98KP6m24416@odiug.zope.com>

> [Guido, on the list(xrange(maxint // 4)) test]
> > That's strange -- that test is specifically designed to fail before it
> > ever gets to allocating memory.  Can you track this down with a
> > debugger?
> 
> [Martin]
> > Notice that this test does not strictly achieve that:

[Tim]
> This is true, although I'd say it plain doesn't achieve that, and leave
> "strictly" out of it.
> 
> > with a four-byte pointer, you attempt to allocate 0x20000000L items (due
> > to roundupsize), which is small enough to try a realloc. In turn, it
> > tries to allocate 0x80000000L bytes, i.e. 2GB. On a 32-bit machine, it
> > is possible to allocate that much memory.
> 
> Depending on platform, of course.  Under MSVC6, it does call the platform
> realloc(), and it's the latter that returns a NULL pointer (the max you can
> alloc under Win32 is 0x7ffdefff).
> 
> If we changed the test to use maxint // 2 instead, it would (just barely)
> trigger the
> 
>     _new_size <= ((~(size_t)0) / sizeof(type))
> 
> test in NRESIZE instead, and keep the platform realloc() out of it.
> 
> Good enough?

[Guido] I'd say so, except I fear that it might cause different
problems (not that I know of any).  I tried it on Linux and on Win98
and it works fine.  I presume you've tested it on Win2K.

But wasn't it specifically designed to catch a condition that was only
true when the size was (approximately) maxint/4?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Oct  8 21:33:30 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 08 Oct 2002 16:33:30 -0400
Subject: [Python-Dev] tests failing in 2.2.2
Message-ID: <36d7e3d7e1.3d7e136d7e@icomcast.net>

[Tim]
>> If we changed the test to use maxint // 2 instead, it would 
>> (just barely) trigger the
>>
>>     _new_size <= ((~(size_t)0) / sizeof(type))
>>
>> test in NRESIZE instead, and keep the platform realloc() out of it.
>>
>> Good enough?

[Guido]
> I'd say so, except I fear that it might cause different problems
> (not that I know of any).  I tried it on Linux and on Win98
> and it works fine.  I presume you've tested it on Win2K.

Not really, but I watched what it *would* have done <wink> in the 
debugger.

> But wasn't it specifically designed to catch a condition that was
> only true when the size was (approximately) maxint/4?

If that was the intent, then Martin was right that it's not catching 
what it intended to catch:  as written, it goes to the platform 
realloc.  The only place it *could* get caught before going to the 
platform realloc is in the snippet of code from NRESIZE I included 
above.  The suggested change would (just barely) trigger that code 
instead, and we wouldn't invoked the platform realloc() at all then.

There are no other interesting conditions I know of, or can find in the 
code.



From Jack.Jansen@oratrix.com  Tue Oct  8 21:37:41 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 8 Oct 2002 22:37:41 +0200
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
In-Reply-To: <20021008135832.GA22587@epoch.metaslash.com>
Message-ID: <CAF2E1F0-DAFD-11D6-B250-003065517236@oratrix.com>

> On Mon, Oct 07, 2002 at 05:45:59PM -0400, Guido van Rossum wrote:
>> I've committed the last changes for Python 2.2.2b1 -- the Misc/NEWS
>> file.  I'm now going to follow the rest of the PEP 102 script until
>> the release is done.  In the mean time, please no more checkins to the
>> 2.2 branch!

Sigh, just got around to giving 2.2.2b1 a first spin on MacOS9 
(yes, I'm back since saturday but didn't have email access until 
yesterday, so it's only since this afternoon I'm aware of the 
2.2.2 schedule), and it crashes the machine hard in the test 
suite after gobbling up all memory. The problem may be in 
test_class, or it could be the victim of an earlier memory 
problem, I'm still investigating. Ah, found it! It's the stack 
limit problem I ran into with 2.3a0 too. So, this particular fix 
should be okay to check in (it's in the mac-only .mcp project) 
but I'll wait for an okay from you anyway.

If I find more problems I think the options are (I think)
- I refrain from checking anything in until 2.2.2 for unix/win 
has been released, then I grab hold of the maint branch and 
check fixes in and we release MacPython 2.2.2 with a slightly 
different set of sources, or
- You hold up the release process until I've found the problem 
and fixed it, and then we do 2.2.2b2 to make sure the fix 
doesn't break any other platforms.

It's your call,
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From guido@python.org  Tue Oct  8 21:48:48 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 16:48:48 -0400
Subject: [Python-Dev] Python 2.2.2b1 checkin freeze
In-Reply-To: Your message of "Tue, 08 Oct 2002 22:37:41 +0200."
 <CAF2E1F0-DAFD-11D6-B250-003065517236@oratrix.com>
References: <CAF2E1F0-DAFD-11D6-B250-003065517236@oratrix.com>
Message-ID: <200210082048.g98Kmmh03773@odiug.zope.com>

> > On Mon, Oct 07, 2002 at 05:45:59PM -0400, Guido van Rossum wrote:
> >> I've committed the last changes for Python 2.2.2b1 -- the Misc/NEWS
> >> file.  I'm now going to follow the rest of the PEP 102 script until
> >> the release is done.  In the mean time, please no more checkins to the
> >> 2.2 branch!

Sorry, the freeze is over, since I did the beta release last night.  I
should've mentioned that.

> Sigh, just got around to giving 2.2.2b1 a first spin on MacOS9 
> (yes, I'm back since saturday but didn't have email access until 
> yesterday, so it's only since this afternoon I'm aware of the 
> 2.2.2 schedule), and it crashes the machine hard in the test 
> suite after gobbling up all memory. The problem may be in 
> test_class, or it could be the victim of an earlier memory 
> problem, I'm still investigating. Ah, found it! It's the stack 
> limit problem I ran into with 2.3a0 too. So, this particular fix 
> should be okay to check in (it's in the mac-only .mcp project) 
> but I'll wait for an okay from you anyway.

Go ahead.  You may also run into a problem in test_b1.py (allocating a
list of size sys.maxint/4); Tim will fix in a change there that should
fix it by making that sys.maxint/2 (!).

> If I find more problems I think the options are (I think)
> - I refrain from checking anything in until 2.2.2 for unix/win 
> has been released, then I grab hold of the maint branch and 
> check fixes in and we release MacPython 2.2.2 with a slightly 
> different set of sources, or
> - You hold up the release process until I've found the problem 
> and fixed it, and then we do 2.2.2b2 to make sure the fix 
> doesn't break any other platforms.
> 
> It's your call,

Please do all the checkins on the Mac branch you want.

For checkins in the rest of the code, I think you should use your
normal caution but please fix things that are broken!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Tue Oct  8 22:06:39 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 8 Oct 2002 17:06:39 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: <200210082025.g98KP6m24416@odiug.zope.com>
References: <BIEJKCLHCIOIHAGOKOLHMEJHDLAA.tim@zope.com> <200210082025.g98KP6m24416@odiug.zope.com>
Message-ID: <20021008210639.GB22587@epoch.metaslash.com>

On Tue, Oct 08, 2002 at 04:25:06PM -0400, Guido van Rossum wrote:
> [Tim]
> > > with a four-byte pointer, you attempt to allocate 0x20000000L items (due
> > > to roundupsize), which is small enough to try a realloc. In turn, it
> > > tries to allocate 0x80000000L bytes, i.e. 2GB. On a 32-bit machine, it
> > > is possible to allocate that much memory.
> > 
> > Depending on platform, of course.  Under MSVC6, it does call the platform
> > realloc(), and it's the latter that returns a NULL pointer (the max you can
> > alloc under Win32 is 0x7ffdefff).
> > 
> > If we changed the test to use maxint // 2 instead, it would (just barely)
> > trigger the
> > 
> >     _new_size <= ((~(size_t)0) / sizeof(type))
> > 
> > test in NRESIZE instead, and keep the platform realloc() out of it.
> > 
> > Good enough?
> 
> [Guido] I'd say so, except I fear that it might cause different
> problems (not that I know of any).  I tried it on Linux and on Win98
> and it works fine.  I presume you've tested it on Win2K.
> 
> But wasn't it specifically designed to catch a condition that was only
> true when the size was (approximately) maxint/4?

The original problem was actually 1e9.  The test is somewhat dubious,
so anything to reduce the problems would be good.  Perhaps the test
should just be removed?

Neal


From tim.one@comcast.net  Tue Oct  8 22:13:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 08 Oct 2002 17:13:19 -0400
Subject: [Python-Dev] tests failing in 2.2.2
Message-ID: <39c003b550.3b55039c00@icomcast.net>

[Neal Norwitz]
> The original problem was actually 1e9.

Which is about maxint/2, which I checked in.

> The test is somewhat dubious, so anything to reduce the problems
> would be good.  Perhaps the test should just be removed?

So long as it's testing Python's overflow-checking code instead of the 
platform realloc(), it should be fine.  It's probably still dubious on 
Win64 (where sys.maxint is still 2**31-1), but should be fine 
everywhere else).  I'll worry about Win64 when somebody runs it <1/64 
wink>.



From neal@metaslash.com  Tue Oct  8 22:16:33 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 8 Oct 2002 17:16:33 -0400
Subject: [Python-Dev] tests failing in 2.2.2
In-Reply-To: <200210041500.g94F0XV25271@pcp02138704pcs.reston01.va.comcast.net>
References: <20021004135948.GA30482@epoch.metaslash.com> <200210041500.g94F0XV25271@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021008211633.GA30812@epoch.metaslash.com>

On Fri, Oct 04, 2002 at 11:00:33AM -0400, Guido van Rossum wrote:
> [Neal Norwitz]
> > Is anyone else having problems with 2.2.2 tests?
> > 
> > minidom & sax fail for me on Linux:
> > 
> > test_sax
> > test test_sax skipped -- cannot import name quoteattr
> > test_minidom
> > Traceback (most recent call last):
> >   File "Lib/test/test_minidom.py", line 636, in ?
> >     func()
> >   File "Lib/test/test_minidom.py", line 208, in testAAA
> >     "setAttribute() sets ownerDocument")
> >   File "Lib/test/test_minidom.py", line 22, in confirm
> >     raise Exception
> > Exception
> > 
> > I'm off for a long weekend.  I'll be back late Monday.
> 
> Both tests work fine for me.  I expect there's a problem with your
> expat installation.  Maybe Fred has a suggestion?

I had an installed version of 2.2.2 from a while ago, in addition to
the latest 2.2.2.  I removed the installed version and all the tests
pass now.

Neal



From neal@metaslash.com  Tue Oct  8 22:27:15 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 8 Oct 2002 17:27:15 -0400
Subject: [Python-Dev] segmentation fault with python2.3a0 from cvs
In-Reply-To: <KGEPLGONBPHFGIHOBLCKIEAKCKAA.gerson.kurz@t-online.de>
References: <20021007173834.GA10088@codesourcery.com> <KGEPLGONBPHFGIHOBLCKIEAKCKAA.gerson.kurz@t-online.de>
Message-ID: <20021008212715.GB30812@epoch.metaslash.com>

On Mon, Oct 07, 2002 at 08:03:56PM +0200, Gerson Kurz wrote:
> Zack Weinberg wrote:
> 
> > For what it's worth, this may be system-specific.  I was able to
> > import your test case under both 2.2.1 and 2.3a0 with no problem, on
> > linux.  The 2.3 compiler is a lot faster, which is nice.
> 
> It seems like that. I compiled with DevStudio 6, and the import works (at
> least in the debug build, but thats the only one I checked). I recompiled
> python unter cygwin using "--with-pydebug", but to the same effect: cygwins'
> python.exe raises a segmentation fault. So maybe, just maybe, its a fault of
> cygwin ?
> 
> $ uname -a
> CYGWIN_NT-5.0 DREIZEHN 1.3.12(0.54/3/2) 2002-07-06 02:16 i686 unknown

This may be relevant, from Lib/test/test_b1.py:

        # Note: This test is expected to SEGV under Cygwin 1.3.12 or
        # earlier due to a newlib bug.  See the following mailing list
        # thread for the details:

        #     http://sources.redhat.com/ml/newlib/2002/msg00369.html

In Cygwin, malloc() was treating the size as signed which caused a
crash.  Perhaps you are experiencing something similar?  Have you
tried with the latest Cygwin?

Neal


From Jack.Jansen@oratrix.com  Tue Oct  8 22:46:59 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 8 Oct 2002 23:46:59 +0200
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
Message-ID: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>

Aside from the usual suspects I have three tests failing in 
MacPython 2.2.2: test_atexit, test_email and test_httplib.

The first one should be easy to fix: backport the revision 1.7 
fix. Would that cause any problems on other platforms?

The other two I have no idea of what could be causing them, so I 
need suggestions. I can't copy/paste, so I'll retype a bit.

test_email: assertion failed, 2042 != 2. from line 1793 in 
test_crlf_separation.
Could this have something to do with MacPython 2.2's "poor man's 
universal newlines"? (MacPython 2.2.X will read both \r and \n 
as \n for files opened in text mode, but not \r\n. This quick 
hack was put in to make 2.2 usable on OSX).

test_httplib: mismatch in lines 9-12 of expected output. It 
wanted two header: Set-Cookie: lines, but got nothing.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From skip@pobox.com  Tue Oct  8 22:54:40 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 16:54:40 -0500
Subject: [Python-Dev] Where is this flag coming from?
Message-ID: <15779.21536.368403.957238@montanaro.dyndns.org>

On my MacOS 10.2 system an unexpected flag is added to the link line for the
bsddb module:

    gcc -bundle -bundle_loader python.exe \
      build/temp.darwin-6.1-Power\ Macintosh-2.3/bsddbmodule.o \
      -L/sw/lib -L/sw/lib -L/usr/local/lib -Wl,-R/sw/lib -ldb-4.1 \
      -o build/lib.darwin-6.1-Power\ Macintosh-2.3/bsddb.so

None of the other extension modules are linked with "-Wl,-R/sw/lib".  I
don't find "-R" in either setup.py or configure.  Any idea where this is
coming from and how to get rid of it?

Skip



From barry@python.org  Tue Oct  8 23:25:26 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 18:25:26 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
Message-ID: <15779.23382.921583.645228@gargle.gargle.HOWL>

>>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:

    JJ> test_email: assertion failed, 2042 != 2. from line 1793 in
    JJ> test_crlf_separation.  Could this have something to do with
    JJ> MacPython 2.2's "poor man's universal newlines"? (MacPython
    JJ> 2.2.X will read both \r and \n as \n for files opened in text
    JJ> mode, but not \r\n. This quick hack was put in to make 2.2
    JJ> usable on OSX).

Undoubtably.  Guido and I had an inkling about that this morning but
didn't follow up.  I suppose I should fire up MacOSX and see what I
can work out, but if you have suggestions I'm all ears.

-Barry


From barry@python.org  Tue Oct  8 23:30:25 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 18:30:25 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15779.21536.368403.957238@montanaro.dyndns.org>
Message-ID: <15779.23681.462147.252222@gargle.gargle.HOWL>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> On my MacOS 10.2 system an unexpected flag is added to the
    SM> link line for the bsddb module:

    |    gcc -bundle -bundle_loader python.exe \
    |      build/temp.darwin-6.1-Power\ Macintosh-2.3/bsddbmodule.o \
    |      -L/sw/lib -L/sw/lib -L/usr/local/lib -Wl,-R/sw/lib -ldb-4.1 \
    |      -o build/lib.darwin-6.1-Power\ Macintosh-2.3/bsddb.so

    SM> None of the other extension modules are linked with
    SM> "-Wl,-R/sw/lib".  I don't find "-R" in either setup.py or
    SM> configure.  Any idea where this is coming from and how to get
    SM> rid of it?

Don't you remember, Skip!?  I think we worked this out when we were
looking at the linking problems with bsddb, since the from-source
Berkeley install doesn't put its libs in a normally searched
location.

These flags get added because of the runtime_library_dirs arg in the
Extension constructor for bsddbmodule.c.

Are you saying that the flags are wrong, the paths are wrong, the
flags are unnecessary, or something else?

-Barry


From jrw@pobox.com  Tue Oct  8 23:34:05 2002
From: jrw@pobox.com (John Williams)
Date: Tue, 08 Oct 2002 17:34:05 -0500
Subject: [Python-Dev] read-only properties
References: <u3crhkkl1.fsf@boost-consulting.com>              <20021008164723.GA22918@unpythonic.net> <200210081826.g98IPx516833@odiug.zope.com>
Message-ID: <3DA35D5D.9010503@pobox.com>

Guido van Rossum wrote:
>>The key seems to be that property() doesn't know the name the
>>property has.
> 
> 
> Oops, you're right.  This makes it practically impossible to improve
> the error message (without the kind of trick that Jeff shows).
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

Why not have the property find its name in the class's __dict__, 
something like this:

(in class property)
   def __set__(self, obj, value):
     if self.fset is not None:
       self.fset(obj, value)
     else:
       for key, value in type(obj).__dict__.iteritems():
         if value is self:
           raise AttributeError, "Property '" + key + "' is read-only."
       # No name?  Strange put possible I guess.
       raise AttributeError, "can't set property"



From greg@cosc.canterbury.ac.nz  Wed Oct  9 00:36:23 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Oct 2002 12:36:23 +1300 (NZDT)
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <20021008160351.GA20990@hishome.net>
Message-ID: <200210082336.g98NaNp18660@oma.cosc.canterbury.ac.nz>

Oren Tirosh <oren-py-d@hishome.net>:

> What would your like to see as the repr() of a rational number? The answer
> to this will also determine what you type in your source.

I think it's the other way around -- what you type in the
source determines what repr() should return.

If a constructor, e.g. rat(2,3) is used, then the
repr() should be likewise.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From barry@python.org  Wed Oct  9 00:50:12 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 19:50:12 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
Message-ID: <15779.28468.831725.1313@gargle.gargle.HOWL>

>>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:

    JJ> Aside from the usual suspects I have three tests failing in 
    JJ> MacPython 2.2.2: test_atexit, test_email and test_httplib.

Oops, looks like I'm also getting a segfault in test_re.

-Barry


From oren-py-d@hishome.net  Wed Oct  9 01:02:12 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Wed, 9 Oct 2002 02:02:12 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <200210081609.g98G9v516159@odiug.zope.com>; from guido@python.org on Tue, Oct 08, 2002 at 12:09:57PM -0400
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <20021008135549.GF5116@xs4all.nl> <20021008160351.GA20990@hishome.net> <200210081609.g98G9v516159@odiug.zope.com>
Message-ID: <20021009020212.A31472@hishome.net>

On Tue, Oct 08, 2002 at 12:09:57PM -0400, Guido van Rossum wrote:
>
> > '3/5'
> 
> I'd like at least one of those return '0.6' or '0.6r'.  

Why should any of them go through a potetially lossy transformation?
If you need to approximate a ratio as a finite decimal fraction it should 
be explicit: str(float(r)). 

I think it will also make a good visual cue to always use rational notation 
for rationals and decimal fractions for floats.

The result of repr() should eval() back to exactly the same object. The 
result of str() should be the 'pretty' representation because this is the 
form displayed by print statements and %s formatting. I find '2/3' prettier 
than '0.666666666667'

pretty-is-in-the-eyes-of-the-beholder-ly yours,

	Oren



From barry@python.org  Wed Oct  9 01:04:31 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 20:04:31 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
 <15779.28468.831725.1313@gargle.gargle.HOWL>
Message-ID: <15779.29327.201832.109788@gargle.gargle.HOWL>

>>>>> "BAW" == Barry A Warsaw <barry@python.org> writes:

    BAW> Oops, looks like I'm also getting a segfault in test_re.

Let's see if cut-n-paste works.  This is under gdb on OSX 10.2 with a
simple "configure; make".  Jack, you're not seeing this are you?
-Barry

@freewill[~/Desktop/src/python22:217]% gdb python.exe
GNU gdb 5.1-20020408 (Apple version gdb-231) (Tue Aug 13 21:37:39 GMT 2002)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-macos10".
Reading symbols for shared libraries .... done
(gdb) run Lib/test/test_re.py
Starting program: /Users/barry/Desktop/src/python22/python.exe Lib/test/test_re.py
[Switching to process 5135 thread 0xb03]
Reading symbols for shared libraries ............. done
Reading symbols for shared libraries . done
Running tests on re.search and re.match
Running tests on re.sub
Running tests on symbolic references
Running tests on re.subn
Running tests on re.split
Running tests on re.findall
Running tests on re.match
Running tests on re.escape
Pickling a RegexObject instance
Reading symbols for shared libraries . done
Reading symbols for shared libraries . done
Test engine limitations

Program received signal EXC_BAD_ACCESS, Could not access memory.
0x0009cfb8 in mark_save (state=0xbffff1c0, lo=1, hi=1) at Modules/_sre.c:290
290     {
(gdb) where
#0  0x0009cfb8 in mark_save (state=0xbffff1c0, lo=1, hi=1) at Modules/_sre.c:290
#1  0x0009c168 in sre_match (state=0xbffff240, pattern=0x3ea47a, level=4068) at Modules/_sre.c:1070
[... lots of frames in sre_match ... ]
#4067 0x0009c0b8 in sre_match (state=0xbffff240, pattern=0x3ea466, level=1) at Modules/_sre.c:1026
#4068 0x0009e928 in pattern_match (self=0x3ea440, args=0x0, kw=0x1) at Modules/_sre.c:1694
#4069 0x000591c0 in PyObject_Call (func=0xbffff240, arg=0x0, kw=0x1) at Objects/abstract.c:1688
#4070 0x00074c4c in do_call (func=0x3ea050, pp_stack=0xbffff6b4, na=1, nk=1528) at Python/ceval.c:3271
#4071 0x000725e0 in eval_frame (f=0x3ba3b0) at Python/ceval.c:2009
#4072 0x00073a50 in PyEval_EvalCodeEx (co=0x386cb0, globals=0x0, locals=0x1, args=0x3ba50c, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=1, closure=0x0) at Python/ceval.c:2595
#4073 0x00074b30 in fast_function (func=0xbffff240, pp_stack=0x54, n=1, na=4104272, nk=1) at Python/ceval.c:3170
#4074 0x000726fc in eval_frame (f=0x3ba3b0) at Python/ceval.c:2034
#4075 0x00073a50 in PyEval_EvalCodeEx (co=0x386cb0, globals=0x0, locals=0x1, args=0x3ba50c, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2595
#4076 0x00076574 in PyEval_EvalCode (co=0xbffff240, globals=0x0, locals=0x1) at Python/ceval.c:481
#4077 0x0002a650 in run_node (n=0x3ba50c, filename=0x0, globals=0x39bab2, locals=0x1, flags=0x9bad0) at Python/pythonrun.c:1079
#4078 0x00029dfc in PyRun_SimpleFileExFlags (fp=0xa0006950, filename=0xbffffcd5 "Lib/test/test_re.py", closeit=1, flags=0x386cb0) at Python/pythonrun.c:685
#4079 0x00006344 in Py_Main (argc=-2013265336, argv=0xbffffbe8) at Modules/main.c:364
#4080 0x00002254 in _start (argc=2, argv=0xbffffbe8, envp=0xbffffbf4) at /SourceCache/Csu/Csu-45/crt.c:267
#4081 0x000020d4 in start ()

Yes, that stack trace is that deep.
-Barry


From barry@python.org  Wed Oct  9 01:11:33 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 8 Oct 2002 20:11:33 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
Message-ID: <15779.29749.886457.639973@gargle.gargle.HOWL>

>>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:

    JJ> Aside from the usual suspects I have three tests failing in 
    JJ> MacPython 2.2.2: test_atexit, test_email and test_httplib.

    JJ> test_email: assertion failed, 2042 != 2. from line 1793 in
    JJ> test_crlf_separation.  Could this have something to do with
    JJ> MacPython 2.2's "poor man's universal newlines"? (MacPython
    JJ> 2.2.X will read both \r and \n as \n for files opened in text
    JJ> mode, but not \r\n. This quick hack was put in to make 2.2
    JJ> usable on OSX).

Re-reading this, are you saying that this failure is on MacOS9 only?
That test passes just fine on OSX 10.2 and the data looks just fine to
me, so this must be an OS9 failure only.

Maybe the thing to do is to open msg_26.txt in binary mode, but only
on MacOS9?  Maybe we should add a sys.platform test to
test_crlf_separation()?

-Barry


From guido@python.org  Wed Oct  9 01:34:29 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 20:34:29 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: Your message of "Tue, 08 Oct 2002 23:46:59 +0200."
 <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
Message-ID: <200210090034.g990YTx26342@pcp02138704pcs.reston01.va.comcast.net>

> Aside from the usual suspects I have three tests failing in 
> MacPython 2.2.2: test_atexit, test_email and test_httplib.
> 
> The first one should be easy to fix: backport the revision 1.7 
> fix. Would that cause any problems on other platforms?

No problemo.

> The other two I have no idea of what could be causing them, so I 
> need suggestions. I can't copy/paste, so I'll retype a bit.
> 
> test_email: assertion failed, 2042 != 2. from line 1793 in 
> test_crlf_separation.
> Could this have something to do with MacPython 2.2's "poor man's 
> universal newlines"? (MacPython 2.2.X will read both \r and \n 
> as \n for files opened in text mode, but not \r\n. This quick 
> hack was put in to make 2.2 usable on OSX).

Is in Barry's capable hands.

> test_httplib: mismatch in lines 9-12 of expected output. It 
> wanted two header: Set-Cookie: lines, but got nothing.

Probably a line ending problem.  I notice that those two lines end
with \r\n!  I don't understand why the test uses \r\n.  I propose to
change all the \r\n to \n in test_httplib.py and ditto in
output/test_httplib.  If you can get that working, feel free to check
it in; or I can do it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 01:38:48 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 08 Oct 2002 20:38:48 -0400
Subject: [Python-Dev] read-only properties
In-Reply-To: Your message of "Tue, 08 Oct 2002 17:34:05 CDT."
 <3DA35D5D.9010503@pobox.com>
References: <u3crhkkl1.fsf@boost-consulting.com> <20021008164723.GA22918@unpythonic.net> <200210081826.g98IPx516833@odiug.zope.com>
 <3DA35D5D.9010503@pobox.com>
Message-ID: <200210090038.g990cmK26359@pcp02138704pcs.reston01.va.comcast.net>

> Why not have the property find its name in the class's __dict__, 
> something like this:
> 
> (in class property)
>    def __set__(self, obj, value):
>      if self.fset is not None:
>        self.fset(obj, value)
>      else:
>        for key, value in type(obj).__dict__.iteritems():
>          if value is self:
>            raise AttributeError, "Property '" + key + "' is read-only."
>        # No name?  Strange put possible I guess.
>        raise AttributeError, "can't set property"

You'd have to search base classes too, in MRO order.  Doable, but
makes AttributeError very expensive.  (This can be a concern if it is
in fact being caught.)

It could also be misleading -- the same property object may occur more
than once, and you don't know which one was being set.

I think it's best to rely on the traceback.  If the traceback looks
like

    Traceback (most recent call last):
      File "foo.py", line 42, in bar
        self.foo = 42
    AttributeError: can't set attribute

the problem should be clear enough.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Oct  9 03:18:46 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 08 Oct 2002 22:18:46 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <20021009020212.A31472@hishome.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>

[Guido, on 3/5r]
> I'd like at least one of those return '0.6' or '0.6r'.

[Oren Tirosh]
> Why should any of them go through a potetially lossy transformation?

0.6r is exact, although it's hard to know whether Guido was hoping to
preserve that or not.

> If you need to approximate a ratio as a finite decimal fraction it should
> be explicit: str(float(r)).
> I think it will also make a good visual cue to always use
> rational notation  for rationals and decimal fractions for floats.

The trailing 'r' *is* rational notation, and stuff like this probably isn't
a good idea for str():

> #e6.02e-45
301/50000000000000000000000000000000000000000000000
>

6.02e-45r would be clearer to virtually anyone.

> (+ 312 (/ 1 3))
937/3
>

would likely be clearer to virtually anyone as 312+1/3r too.

But most of all, I agree that if you're working with rationals, you don't
want to lose information silently, not even in a "pleasant" string.

> The result of repr() should eval() back to exactly the same object. The
> result of str() should be the 'pretty' representation because this is the
> form displayed by print statements and %s formatting. I find
> '2/3' prettier than '0.666666666667'

2/3r works for both for me.  0.666...7 doesn't work for me at all as a
stringification of a rational (if I want to approximate, I'll ask for an
approximation).



From skip@pobox.com  Wed Oct  9 04:24:36 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 8 Oct 2002 22:24:36 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15779.23681.462147.252222@gargle.gargle.HOWL>
References: <15779.21536.368403.957238@montanaro.dyndns.org>
 <15779.23681.462147.252222@gargle.gargle.HOWL>
Message-ID: <15779.41332.251210.360418@montanaro.dyndns.org>

    SM> None of the other extension modules are linked with "-Wl,-R/sw/lib".
    SM> I don't find "-R" in either setup.py or configure.  Any idea where
    SM> this is coming from and how to get rid of it?

    BAW> Don't you remember, Skip!?  I think we worked this out when we were
    BAW> looking at the linking problems with bsddb, since the from-source
    BAW> Berkeley install doesn't put its libs in a normally searched
    BAW> location.

I remember having problems with bsddb, which I think were fixed.
-Wl,-R/sw/lib is clearly wrong on MacOS X though.  Deleting it and running
the link command manually works.

    BAW> Are you saying that the flags are wrong, the paths are wrong, the
    BAW> flags are unnecessary, or something else?

The flags are incorrect:

    ...
    building 'bsddb' extension
    gcc -bundle -bundle_loader python.exe \
      build/temp.darwin-6.1-Power\ Macintosh-2.3/bsddbmodule.o \
      -L/sw/lib -L/home/skip/local/lib -L/sw/lib -L/usr/local/lib \
      -Wl,-R/sw/lib -ldb-4.1 \
      -o build/lib.darwin-6.1-Power\ Macintosh-2.3/bsddb.so
    ld: unknown flag: -R/sw/lib
    ...

Skip


From tim@multitalents.net  Wed Oct  9 05:54:44 2002
From: tim@multitalents.net (Tim Rice)
Date: Tue, 8 Oct 2002 21:54:44 -0700 (PDT)
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15779.41332.251210.360418@montanaro.dyndns.org>
Message-ID: <Pine.UW2.4.44.0210082135290.19141-100000@ou8.int.multitalents.net>

On Tue, 8 Oct 2002, Skip Montanaro wrote:

>
>     SM> None of the other extension modules are linked with "-Wl,-R/sw/lib".
>     SM> I don't find "-R" in either setup.py or configure.  Any idea where
>     SM> this is coming from and how to get rid of it?
>
>     BAW> Don't you remember, Skip!?  I think we worked this out when we were
>     BAW> looking at the linking problems with bsddb, since the from-source
>     BAW> Berkeley install doesn't put its libs in a normally searched
>     BAW> location.
>
> I remember having problems with bsddb, which I think were fixed.
> -Wl,-R/sw/lib is clearly wrong on MacOS X though.  Deleting it and running
> the link command manually works.
>
>     BAW> Are you saying that the flags are wrong, the paths are wrong, the
>     BAW> flags are unnecessary, or something else?
>

The -R flag in not supported on all systems.
2.2.2b1 build is broken on SCO Open Server now too.

> The flags are incorrect:
>
>     ...
>     building 'bsddb' extension
>     gcc -bundle -bundle_loader python.exe \
>       build/temp.darwin-6.1-Power\ Macintosh-2.3/bsddbmodule.o \
>       -L/sw/lib -L/home/skip/local/lib -L/sw/lib -L/usr/local/lib \
>       -Wl,-R/sw/lib -ldb-4.1 \
>       -o build/lib.darwin-6.1-Power\ Macintosh-2.3/bsddb.so
>     ld: unknown flag: -R/sw/lib
>     ...
>
> Skip
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
>

-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net




From barry@python.org  Wed Oct  9 05:58:18 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 00:58:18 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15779.41332.251210.360418@montanaro.dyndns.org>
 <Pine.UW2.4.44.0210082135290.19141-100000@ou8.int.multitalents.net>
Message-ID: <15779.46954.672730.965163@gargle.gargle.HOWL>

>>>>> "TR" == Tim Rice <tim@multitalents.net> writes:

    TR> The -R flag in not supported on all systems.
    TR> 2.2.2b1 build is broken on SCO Open Server now too.

Shouldn't distutils be taught what to do with runtime_library_dirs for
SCO and MacOSX?

-Barry


From oren-py-d@hishome.net  Wed Oct  9 09:13:06 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Wed, 9 Oct 2002 10:13:06 +0200
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>; from tim.one@comcast.net on Tue, Oct 08, 2002 at 10:18:46PM -0400
References: <20021009020212.A31472@hishome.net> <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>
Message-ID: <20021009101306.A4307@hishome.net>

On Tue, Oct 08, 2002 at 10:18:46PM -0400, Tim Peters wrote:
> > If you need to approximate a ratio as a finite decimal fraction it should
> > be explicit: str(float(r)).
> > I think it will also make a good visual cue to always use
> > rational notation  for rationals and decimal fractions for floats.
> 
> The trailing 'r' *is* rational notation, and stuff like this probably isn't
> a good idea for str():

I'm not sure I follow. What 'stuff' is not a good idea for str() and why?  

By rational notation I meant the division form, not the trailing r or any 
other syntax used by the language to indicate the type.

> 
> > #e6.02e-45
> 301/50000000000000000000000000000000000000000000000
> >
> 
> 6.02e-45r would be clearer to virtually anyone.

Again, I'm not sure I follow your logic. How likely is an arbitrary
rational number to have a denominator ending with lots of zeros?

> > (+ 312 (/ 1 3))
> 937/3
> >
> 
> would likely be clearer to virtually anyone as 312+1/3r too.

Yes, in this case it is much clearer.

> But most of all, I agree that if you're working with rationals, you don't
> want to lose information silently, not even in a "pleasant" string.
> 
> > The result of repr() should eval() back to exactly the same object. The
> > result of str() should be the 'pretty' representation because this is the
> > form displayed by print statements and %s formatting. I find
> > '2/3' prettier than '0.666666666667'
> 
> 2/3r works for both for me.  0.666...7 doesn't work for me at all as a
> stringification of a rational (if I want to approximate, I'll ask for an
> approximation).

Works for repr, str, or both?  The str() of long omits the trailing L.
I don't think rationals should show the trailing r. 

	Oren


From mwh@python.net  Wed Oct  9 10:34:27 2002
From: mwh@python.net (Michael Hudson)
Date: 09 Oct 2002 10:34:27 +0100
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: barry@python.org's message of "Tue, 8 Oct 2002 19:50:12 -0400"
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com> <15779.28468.831725.1313@gargle.gargle.HOWL>
Message-ID: <2mfzvgnefw.fsf@starship.python.net>

barry@python.org (Barry A. Warsaw) writes:

> >>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:
> 
>     JJ> Aside from the usual suspects I have three tests failing in 
>     JJ> MacPython 2.2.2: test_atexit, test_email and test_httplib.
> 
> Oops, looks like I'm also getting a segfault in test_re.

This is normal; run "ulimit -s 2048" before running the test suite.

Cheers,
M.

-- 
  please realize that the Common  Lisp community is more than 40
  years old.  collectively, the community has already been where
  every clueless newbie  will be going for the next three years.
  so relax, please.                     -- Erik Naggum, comp.lang.lisp


From stephen@xemacs.org  Wed Oct  9 13:10:36 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: Wed, 09 Oct 2002 21:10:36 +0900
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Mon, 07 Oct 2002 08:20:27 -0400")
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de>
 <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <8765wbak3n.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Guido" =3D=3D Guido van Rossum <guido@python.org> writes:

    >> Did you try to open the file in IDLE? What other tools are you
    >> using?

    Guido> XEmacs.  The particular version or configuration I'm using
    Guido> apparently doesn't recogize the coding cookie.

No current XEmacs recognizes cookies in the second line.  I tried it,
it broke, I backed it out, had a bad attack of day job, and forgot to
fix the patch.  :-(

This may[1] work in 21.4.10.  The code is moderately complicated because
we have to deal with all the different linebreaking conventions in the
raw buffer.  I wish I could work on an editor that only needs to deal
with Lisp, Python, and XML....

You will need to use Mule-UCS (the package that provides un-define) to
get Unicode (any flavor of UTF) support in XEmacs 21.4.  You probably
don't want latin-unity if you're playing with cookies, because it
currently incorrectly complains about cookies that are OK.  (This is
also EOL breakage.  :-=FE)


Footnotes:=20
[1]  Assuming the new patch isn't broken in a new way, and has to be
backed out.

--=20
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac=
.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JA=
PAN
 My nostalgia for Icon makes me forget about any of the bad things.  I don't
have much nostalgia for Perl, so its faults I remember.  Scott Gilbert c.l.=
py


From stephen@xemacs.org  Wed Oct  9 13:19:32 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: Wed, 09 Oct 2002 21:19:32 +0900
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Mon, 07 Oct 2002 08:20:27 -0400")
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net>
 <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de>
 <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <873crfajor.fsf@tleepslib.sk.tsukuba.ac.jp>

Oops.

I wrote:

    >> No current XEmacs recognizes cookies in the second line.

and

    >> You will need to use Mule-UCS [for UTF-8]

I suspect this means that there will be glitches in cookie support for
UTF-8, even in l.1.

I'll keep lurking to see what people are saying on py-dev, or if you
really really want support in XEmacs, write me directly with bug
reports and example files and I'll see what I can do (I still hate
cookies but love Python more than ever!)  [No, Barry, I dunno nothing
about RFC 2047 + UTF-8, that's been on my list for a long time, too.]

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
 My nostalgia for Icon makes me forget about any of the bad things.  I don't
have much nostalgia for Perl, so its faults I remember.  Scott Gilbert c.l.py


From arigo@tunes.org  Wed Oct  9 13:57:26 2002
From: arigo@tunes.org (Armin Rigo)
Date: Wed, 9 Oct 2002 14:57:26 +0200 (CEST)
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: <E17ywKr-0003UJ-0U@anchor-post-39.mail.demon.net>
Message-ID: <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be>

Hello Richie,

On Tue, 8 Oct 2002 richie@entrian.com wrote:
> (...)  A nasty consequence is that you can write Python code that causes
> Python to seg-fault, but you have to be doing some fairly advanced stuff
> for that to happen.

You can already crash the interpreter with pure Python code, for example
via the new.code() constructor or by writing crappy .pyc files.  On Linux
you can also open("/proc/self/mem", "w").

I don't think that people get their hands on frame objects by pure chance,
but the possibility exists.  Moreover, conditionally allowing changes to
f_lasti is limiting and complex because of the stack and block stack.  For
safety I'd consider writing the frame-object-modifying code in a C
extension module, carefully documented as "don't use this". 


Armin



From neal@metaslash.com  Wed Oct  9 14:11:42 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 9 Oct 2002 09:11:42 -0400
Subject: [Python-Dev] Re: [snake-farm] Re: [farm-report] Daily Snake Farm report
In-Reply-To: <49c9447952.4795249c94@icomcast.net>
References: <49c9447952.4795249c94@icomcast.net>
Message-ID: <20021009131142.GG30812@epoch.metaslash.com>

On Wed, Oct 09, 2002 at 03:59:29AM -0400, Tim Peters wrote:
> [Anders Qvist]
> > ...
> >    check_all("ntpath")
> >  File "../../python/dist/src/Lib/test/test___all__.py", line 15, 
> > in check_all
> >    exec "import %s" % modname in names
> >  File "<string>", line 1, in ?
> >  File "/home/quest/tmp/compile-
> > python/python/dist/src/Lib/ntpath.py", line 482, in ?
> >    supports_unicode_filenames = sys.getwindowsversion()[3] >= 2
> > AttributeError: 'module' object has no attribute 'getwindowsversion'
> 
> Grrr.  I expect I just fixed this, but can't test it now on Linux for 
> you.  The person who checked this change in probably didn't realize 
> that ntpath was imported on any system other than Windows.  I'm 
> surprised the bug went uncaught for so long!  (Ya, it was less than a 
> day, but still ...)

:-) I've been running 2.2.2 only.

I just tried to run 2.3 and had some problems.  __all__ is ok, though.

The first problem is in regrtest.  This exception causes regrtest to crash.

test_pyexpat
Traceback (most recent call last):
  File "./Lib/test/regrtest.py", line 877, in ?
    main()
  File "./Lib/test/regrtest.py", line 229, in main
    ok = runtest(test, generate, verbose, quiet, testdir)
  File "./Lib/test/regrtest.py", line 392, in runtest
    output = cfp.getvalue()
  File "/home/neal/build/python/dist/src/Lib/StringIO.py", line 195, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 36: ordinal not in range(128)

I put a try/except around line 392 in regrtest and then had 2 failures:

test_pyexpat had no output (the unicode problem I guess) and test_rotor:

test test_rotor produced unexpected output:
**********************************************************************
*** mismatch between lines 2-3 of expected output and lines 2-3 of actual output:
- '\xb5q\x90\xa3\xe8IT\xd1\x07\xda\x1d\xdc\xdf'
+ 'qqqqq\x90
...

Some unicode problem complaining about these 2 chars:  \xcb and \xe2
which are printed as e umlaut and e circumflex.

Maybe this is just me again though?

Neal


From mwh@python.net  Wed Oct  9 14:20:50 2002
From: mwh@python.net (Michael Hudson)
Date: 09 Oct 2002 14:20:50 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib ntpath.py,1.50,1.51
References: <E17zBhP-0007dD-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2md6qjd9zh.fsf@starship.python.net>

tim_one@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv28728/python/Lib
> 
> Modified Files:
> 	ntpath.py 
> Log Message:
> Don't try to access sys.getwindowsversion unless it exists (ntpath is
> imported on systems other than Windows, and in particular is imported
> by test___all__; the compile farm reported that all Linux tests failed
> due to this; isn't anyone in PythonDevLand running CVS on Linux?!).

Yes, but I do sleep occasionally...

Cheers,
M.

-- 
  it's not that perl programmers are idiots, it's that the language
  rewards idiotic behavior in a  way that no other language or tool
  has ever done                         -- Erik Naggum, comp.lang.lisp


From skip@pobox.com  Wed Oct  9 14:54:49 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 9 Oct 2002 08:54:49 -0500
Subject: [Python-Dev] test_locale failing on MacOSX w/ 2.2.2
Message-ID: <15780.13609.477242.561460@montanaro.dyndns.org>

Just got this test failure this morning on MacOS 10.2 running w/ 2.2.2 from
CVS (or very nearly so):

    % ./python.exe ../Lib/test/regrtest.py test_locale
    test_locale
    test test_locale produced unexpected output:
    **********************************************************************
    *** lines 2-4 of actual output doesn't appear in expected output after line 1:
    + '%f' % 1024 == '1024.000000' != '1,024.000000'
    + '%+10.f' % -4200 == '     -4200' != '    -4,200'
    + '%-10.f' % 4200 == '4200      ' != '4,200     '
    **********************************************************************

Skip


From list-python@ccraig.org  Wed Oct  9 15:16:46 2002
From: list-python@ccraig.org (Christopher A. Craig)
Date: 09 Oct 2002 10:16:46 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>
Message-ID: <whu1jv8zox.fsf@olorin.ts.gatech.edu>

Tim Peters <tim.one@comcast.net> writes:

> 2/3r works for both for me.  0.666...7 doesn't work for me at all as
> a stringification of a rational (if I want to approximate, I'll ask
> for an approximation).

I agree.  I initially thought about returning a decimal with str(),
but for that reason chose not to.  It might be nice to have a method
that returned the best floating point decimal approximation to within
n decimal digits or shorter, but I don't want a builtin to make an
arbitrary decision on where to cut off precision.

I do like the idea of str() returning the form 5+3/2r, though.

-- 
Christopher A. Craig <list-python@ccraig.org>
Python is an excellent language for learning object orientation. (It also
happens to be my favorite OO scripting language.)
Sriram Srinivasan -- "Advanced Perl Programming"


From guido@python.org  Wed Oct  9 15:52:43 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 10:52:43 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Wed, 09 Oct 2002 02:02:12 +0200."
 <20021009020212.A31472@hishome.net>
References: <200210080018.g980IA706850@oma.cosc.canterbury.ac.nz> <200210081241.g98CfVA23876@mercur.uphs.upenn.edu> <20021008135549.GF5116@xs4all.nl> <20021008160351.GA20990@hishome.net> <200210081609.g98G9v516159@odiug.zope.com>
 <20021009020212.A31472@hishome.net>
Message-ID: <200210091452.g99EqiR10950@odiug.zope.com>

> > > '3/5'
> > 
> > I'd like at least one of those return '0.6' or '0.6r'.  
> 
> Why should any of them go through a potetially lossy transformation?

Human factors.  Same reason why printing a float rounds to about 12
digits.  The "accuracy" of 123456789/234567890 is lost on the eye of
the beholder -- I'd have to count digits to tell whether that's
smaller or larger than one!

> If you need to approximate a ratio as a finite decimal fraction it
> should be explicit: str(float(r)).

Nah, if you want to show the numerator and denominator you should use
repr(r).

> I think it will also make a good visual cue to always use rational
> notation for rationals and decimal fractions for floats.

What do you mean by that?  What's wrong with using 0.6r for a rational
number?

> The result of repr() should eval() back to exactly the same
> object. The result of str() should be the 'pretty' representation
> because this is the form displayed by print statements and %s
> formatting. I find '2/3' prettier than '0.666666666667'

But there are very few rationals where that's really true.  Most of
them look more like 1753/811.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Wed Oct  9 15:49:44 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 09 Oct 2002 10:49:44 -0400
Subject: [Python-Dev] Issue with 2.2.2b1 and MinGW-2.0
Message-ID: <u3crfvf93.fsf@boost-consulting.com>

So I thought I'd try testing the latests Boost.Python with 2.2.2 and
ALL of the compilers I have available to me this morning, and it
turned up this interesting data point:

What I get from the Windows installer differs from what I get when I
build Python using MS Visual Studio 6 in a way that affects whether
MinGW-2.0 can link extension modules with it out-of-the-box. 

The one that I get from the installer fails to link (errors below)
unless I make the magic incantations specified here:

http://www.python.org/doc/current/inst/non-ms-compilers.html#SECTION000312000000000000000

The one that I build myself works just fine "out-of-the-box".

---

These were the link errors I got from MinGW-2.0 g++ using the Python I
got from the Windows installer:

Creating library file: c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\boost_python.lib
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\numeric.obj: In function `ZN5boost6python7numeric53_GLOBAL__N_.._.._.._libs_python_src_numeric.cpp2dT8hb18throw_load_failureEv':
c:/boost/libs/python/test/../../../libs/python/src/numeric.cpp:29: undefined reference to `_imp__PyExc_ImportError'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\numeric.obj: In function `ZN5boost6python7numeric53_GLOBAL__N_.._.._.._libs_python_src_numeric.cpp2dT8hb4loadEb':
c:/boost/libs/python/test/../../../libs/python/src/numeric.cpp:57: undefined reference to `_imp__PyType_Type'
c:/boost/libs/python/test/../../../libs/python/src/numeric.cpp:57: undefined reference to `_imp__PyType_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\list.obj: In function `ZN5boost6python6detail9list_base4callERKNS0_3api6objectE':
c:/boost/libs/python/test/../../../libs/python/src/list.cpp:12: undefined reference to `_imp__PyList_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\list.obj: In function `ZN5boost6python6detail9list_base6appendERKNS0_3api6objectE':
c:/boost/libs/python/test/../../../libs/python/src/list.cpp:29: undefined reference to `_imp__PyList_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\list.obj: In function `ZN5boost6python6detail9list_base6insertEiRKNS0_3api6objectE':
c:/boost/libs/python/test/../../../libs/python/src/list.cpp:65: undefined reference to `_imp__PyList_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\list.obj: In function `ZN5boost6python6detail9list_base7reverseEv':
c:/boost/libs/python/test/../../../libs/python/src/list.cpp:106: undefined reference to `_imp__PyList_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\list.obj: In function `ZN5boost6python6detail9list_base4sortEv':
c:/boost/libs/python/test/../../../libs/python/src/list.cpp:119: undefined reference to `_imp__PyList_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\long.obj: In function `ZN5boost6python6detail9long_base4callERKNS0_3api6objectE':
c:/boost/libs/python/test/../../../libs/python/src/long.cpp:12: undefined reference to `_imp__PyLong_Type'
c:\build\libs\python\build\bin\boost_python.dll\mingw-2.0\debug\runtime-link-dynamic\long.obj: In function `ZN5boost6python6detail9long_base4callERKNS0_3api6objectES6_':


-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From guido@python.org  Wed Oct  9 16:36:31 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 11:36:31 -0400
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Your message of "Wed, 09 Oct 2002 00:58:18 EDT."
 <15779.46954.672730.965163@gargle.gargle.HOWL>
References: <15779.41332.251210.360418@montanaro.dyndns.org> <Pine.UW2.4.44.0210082135290.19141-100000@ou8.int.multitalents.net>
 <15779.46954.672730.965163@gargle.gargle.HOWL>
Message-ID: <200210091536.g99FaVa11272@odiug.zope.com>

>     TR> The -R flag in not supported on all systems.
>     TR> 2.2.2b1 build is broken on SCO Open Server now too.
> 
> Shouldn't distutils be taught what to do with runtime_library_dirs for
> SCO and MacOSX?

My thoughts exactly.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 16:55:13 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 11:55:13 -0400
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: Your message of "Wed, 09 Oct 2002 14:57:26 +0200."
 <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be>
References: <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be>
Message-ID: <200210091555.g99FtD011873@odiug.zope.com>

> On Tue, 8 Oct 2002 richie@entrian.com wrote:
> > (...)  A nasty consequence is that you can write Python code that causes
> > Python to seg-fault, but you have to be doing some fairly advanced stuff
> > for that to happen.

[Armin]
> You can already crash the interpreter with pure Python code, for example
> via the new.code() constructor or by writing crappy .pyc files.

Yes, and I occasionally lose sleep over those.  I definitely don't
want to add more loopholes like that, and I'd like to fix those.  In
the past, the new module was optional for precisely this reason (and
so is the dl module).  I would like to have a check on .pyc files (or
on unmarshalled code objects, really).

> On Linux you can also open("/proc/self/mem", "w").

That doesn't mean that python should abandon the policy "a core dump
is always considered Python's fault unless proven otherwise."
Protecting you against using *external* tools may not be possible; but
Python should not include features (like writable code object
attributes) that can cause crashes when used inexpertly.

> I don't think that people get their hands on frame objects by pure
> chance, but the possibility exists.  Moreover, conditionally
> allowing changes to f_lasti is limiting and complex because of the
> stack and block stack.  For safety I'd consider writing the
> frame-object-modifying code in a C extension module, carefully
> documented as "don't use this".

That's a reasonable solution: f_lasti should be read-only from Python
code, but you can write an extension that can write it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 17:13:09 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 12:13:09 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: Your message of "Wed, 09 Oct 2002 10:16:46 EDT."
 <whu1jv8zox.fsf@olorin.ts.gatech.edu>
References: <LNBBLJKPBEHFEDALKOLCAEEBBKAB.tim.one@comcast.net>
 <whu1jv8zox.fsf@olorin.ts.gatech.edu>
Message-ID: <200210091613.g99GD9812324@odiug.zope.com>

> Tim Peters <tim.one@comcast.net> writes:
> > 2/3r works for both for me.  0.666...7 doesn't work for me at all as
> > a stringification of a rational (if I want to approximate, I'll ask
> > for an approximation).
> 
> I agree.  I initially thought about returning a decimal with str(),
> but for that reason chose not to.  It might be nice to have a method
> that returned the best floating point decimal approximation to within
> n decimal digits or shorter, but I don't want a builtin to make an
> arbitrary decision on where to cut off precision.
> 
> I do like the idea of str() returning the form 5+3/2r, though.

Well, that would completely kills any possibility of ever making 1/3
return a rational (currently the main argument against it is the
expectation amongst the current user base that it will be a float).

I really don't think that this would be reasonable at all:

  Python 3.0 (#2345, Mar 12 2005, 11:32:10)
  Type "help", "copyright", "credits" or "license" for more information.
  >>> print 25/7
  25/7
  >>>

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim@multitalents.net  Wed Oct  9 17:15:15 2002
From: tim@multitalents.net (Tim Rice)
Date: Wed, 9 Oct 2002 09:15:15 -0700 (PDT)
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15779.46954.672730.965163@gargle.gargle.HOWL>
Message-ID: <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net>

On Wed, 9 Oct 2002, Barry A. Warsaw wrote:

>
> >>>>> "TR" == Tim Rice <tim@multitalents.net> writes:
>
>     TR> The -R flag in not supported on all systems.
>     TR> 2.2.2b1 build is broken on SCO Open Server now too.
>
> Shouldn't distutils be taught what to do with runtime_library_dirs for
> SCO and MacOSX?

It gets tricky trying to automagicly use the -R flag.
As an example, the default install dir for OpenSSL is /usr/local/ssl.
So on platforms that support -R you would think you could use
-L/usr/local/ssl/lib -R/usr/local/ssl/lib -lssl -lcrypto
OpenUNIX supports -R but if you did not build shared libraries for OpenSSL,
the -R/usr/local/ssl/lib will break the build.

Then there is the whole issue of what non system dirs to include.
Maybe you have OpenSSL 0.9.6g in /usr/local/ssl but you're testing
0.9.7 that's installed in some other dir. If setup.py automagicly
adds ssl to build the '_socket' extension, how does it know which
version to use? How would it even find the one not in /usr/local/ssl?
Normally you would configure --with-ssl-dir=/some_path_to_ssl

As one who has done ports to many Open Source projects, I found python
one of the more difficult ones because the build process uses python.
So for a new platform (or broken build) you have to know python
to build python.

>
> -Barry
>

-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net




From tim@zope.com  Wed Oct  9 17:42:27 2002
From: tim@zope.com (Tim Peters)
Date: Wed, 9 Oct 2002 12:42:27 -0400
Subject: [Python-Dev] Snapshot win32all builds of interest?
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPCEFIGMAA.mhammond@skippinet.com.au>
Message-ID: <BIEJKCLHCIOIHAGOKOLHIELIDLAA.tim@zope.com>

[Mark Hammond]
> OK - first such snapshot done.  See the link from
> starship.python.net/crew/mhammond
>
> This version installs no shortcuts, does not compile .pyc files etc - you
> are pretty much on your own.  Pythonwin\start_pythonwin.pyw is
> installed to start Pythonwin, but you must do so manually.
>
> Please let me know if you try it.

I tried it.  I'd say something useful, except as far as I could tell, it
worked <wink>.  Thanks!  Have you talked w/ the PBF about getting builds
tested?

FYI, the current PLabs Python uninstaller arranges to delete .pyc and .pyo
files via these WISE lines in the installer:

Add "File Tree: %MAINDIR%\*.pyc" to INSTALL.LOG
Add "File Tree: %MAINDIR%\*.pyo" to INSTALL.LOG

These arrange to (recursively) delete all .pyc and .pyo files at uninstall
time, and "do the right thing":  any directories left empty at the end are
magically removed.



From martin@v.loewis.de  Wed Oct  9 18:02:32 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 19:02:32 +0200
Subject: [Python-Dev] test_locale failing on MacOSX w/ 2.2.2
In-Reply-To: <15780.13609.477242.561460@montanaro.dyndns.org>
References: <15780.13609.477242.561460@montanaro.dyndns.org>
Message-ID: <m3y997tujb.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

>     **********************************************************************
>     *** lines 2-4 of actual output doesn't appear in expected output after line 1:
>     + '%f' % 1024 == '1024.000000' != '1,024.000000'
>     + '%+10.f' % -4200 == '     -4200' != '    -4,200'
>     + '%-10.f' % 4200 == '4200      ' != '4,200     '
>     **********************************************************************

That's most likely an OS X bug, for not properly supporting the en_US
locale. Various systems fail this test, we have always declared those
problems 3rdparty.

Regards,
Martin



From thomas.heller@ion-tof.com  Wed Oct  9 18:05:03 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Wed, 9 Oct 2002 19:05:03 +0200
Subject: [Python-Dev] Snapshot win32all builds of interest?
References: <BIEJKCLHCIOIHAGOKOLHIELIDLAA.tim@zope.com>
Message-ID: <03ba01c26fb6$02668780$e000a8c0@thomasnotebook>

From: "Tim Peters" <tim@zope.com>
> [Mark Hammond]
> > OK - first such snapshot done.  See the link from
> > starship.python.net/crew/mhammond
> >
> > This version installs no shortcuts, does not compile .pyc files etc - you
> > are pretty much on your own.  Pythonwin\start_pythonwin.pyw is
> > installed to start Pythonwin, but you must do so manually.
> >
> > Please let me know if you try it.
> 
> I tried it.  I'd say something useful, except as far as I could tell, it
> worked <wink>.  Thanks!  Have you talked w/ the PBF about getting builds
> tested?

Same for me (except that I see a lot of these FutureWarnings
about hex constants > sys.maxint, but you already knew this).

Thomas


From martin@v.loewis.de  Wed Oct  9 18:05:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 19:05:04 +0200
Subject: [Python-Dev] Issue with 2.2.2b1 and MinGW-2.0
In-Reply-To: <u3crfvf93.fsf@boost-consulting.com>
References: <u3crfvf93.fsf@boost-consulting.com>
Message-ID: <m3u1jvtuf3.fsf@mira.informatik.hu-berlin.de>

David Abrahams <dave@boost-consulting.com> writes:

> So I thought I'd try testing the latests Boost.Python with 2.2.2 and
> ALL of the compilers I have available to me this morning, and it
> turned up this interesting data point:

David,

Was this for information only, or did you want readers to infer some
conclusion?

Regards,
Martin



From martin@v.loewis.de  Wed Oct  9 18:10:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 19:10:54 +0200
Subject: [Python-Dev] Snapshot win32all builds of interest?
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHIELIDLAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHIELIDLAA.tim@zope.com>
Message-ID: <m3ptujtu5d.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim@zope.com> writes:

> I tried it.  I'd say something useful, except as far as I could tell, it
> worked <wink>.  Thanks!

I've had a few problems. Running it in a console gives the messages

Failed to connect to Pythonwin|System. Error 0

mh\Pythonwin\pywin\scintilla\view.py:18: DeprecationWarning: the regex
 module is deprecated; please use the re module
  import regex
E:\Pyinst\python\lib\regsub.py:15: DeprecationWarning: the regsub
module is deprecated; please use re.sub()
  DeprecationWarning)

which appear to be harmless; I'll report them anyway in case you are
unaware of them.

More troubling is that Find (Ctrl-F) does not work: I get

Traceback (most recent call last):
  File "mh\Pythonwin\pywin\scintilla\view.py", line 349, in OnCmdEditFind
ImportError: No module named find

Any suggestions?

Regards,
Martin


From dave@boost-consulting.com  Wed Oct  9 18:21:55 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 09 Oct 2002 13:21:55 -0400
Subject: [Python-Dev] Issue with 2.2.2b1 and MinGW-2.0
In-Reply-To: <m3u1jvtuf3.fsf@mira.informatik.hu-berlin.de>
References: <u3crfvf93.fsf@boost-consulting.com>
 <m3u1jvtuf3.fsf@mira.informatik.hu-berlin.de>
Message-ID: <uy997o7d8.fsf@boost-consulting.com>

martin@v.loewis.de (Martin v. Loewis) writes:

> David Abrahams <dave@boost-consulting.com> writes:
> 
> > So I thought I'd try testing the latests Boost.Python with 2.2.2 and
> > ALL of the compilers I have available to me this morning, and it
> > turned up this interesting data point:
> 
> David,
> 
> Was this for information only, or did you want readers to infer some
> conclusion?

It's mostly informational. I don't think it's worth holding up the
2.2.2 release over, but somebody else might think it was especially
worth investigating.

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From guido@python.org  Wed Oct  9 19:17:51 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 14:17:51 -0400
Subject: [Python-Dev] Re: [snake-farm] Re: [farm-report] Daily Snake Farm report
In-Reply-To: Your message of "Wed, 09 Oct 2002 09:11:42 EDT."
 <20021009131142.GG30812@epoch.metaslash.com>
References: <49c9447952.4795249c94@icomcast.net>
 <20021009131142.GG30812@epoch.metaslash.com>
Message-ID: <200210091817.g99IHqW16465@odiug.zope.com>

> The first problem is in regrtest.  This exception causes regrtest to crash.
> 
> test_pyexpat
> Traceback (most recent call last):
>   File "./Lib/test/regrtest.py", line 877, in ?
>     main()
>   File "./Lib/test/regrtest.py", line 229, in main
>     ok = runtest(test, generate, verbose, quiet, testdir)
>   File "./Lib/test/regrtest.py", line 392, in runtest
>     output = cfp.getvalue()
>   File "/home/neal/build/python/dist/src/Lib/StringIO.py", line 195, in getvalue
>     self.buf += ''.join(self.buflist)
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 36: ordinal not in range(128)
> 
> I put a try/except around line 392 in regrtest and then had 2 failures:
> 
> test_pyexpat had no output (the unicode problem I guess) and test_rotor:
> 
> test test_rotor produced unexpected output:
> **********************************************************************
> *** mismatch between lines 2-3 of expected output and lines 2-3 of actual output:
> - '\xb5q\x90\xa3\xe8IT\xd1\x07\xda\x1d\xdc\xdf'
> + 'qqqqq\x90
> ...
> 
> Some unicode problem complaining about these 2 chars:  \xcb and \xe2
> which are printed as e umlaut and e circumflex.
> 
> Maybe this is just me again though?

I saw this too.  The fix was to add a setlocale() call to
test___all__.py.  Read my checkin comments and weep.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Oct  9 19:18:40 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 14:18:40 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15779.46954.672730.965163@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net>
Message-ID: <15780.29440.641827.692637@gargle.gargle.HOWL>

>>>>> "TR" == Tim Rice <tim@multitalents.net> writes:

    TR> It gets tricky trying to automagicly use the -R flag.  As an
    TR> example, the default install dir for OpenSSL is
    TR> /usr/local/ssl.  So on platforms that support -R you would
    TR> think you could use -L/usr/local/ssl/lib -R/usr/local/ssl/lib
    TR> -lssl -lcrypto OpenUNIX supports -R but if you did not build
    TR> shared libraries for OpenSSL, the -R/usr/local/ssl/lib will
    TR> break the build.

Since there seems to be so little agreement among *nixes about how to
do this, it seems like the only/best place to codify all this
knowledge is in distutils.  I still think specifying
runtime_library_dirs to the Extension constructor is the right thing
to do.  But maybe on some systems distutils ought to ignore it, or at
least check to make sure that the dynamic libraries it wants to link
against are actually present.

    TR> Then there is the whole issue of what non system dirs to
    TR> include.  Maybe you have OpenSSL 0.9.6g in /usr/local/ssl but
    TR> you're testing 0.9.7 that's installed in some other dir. If
    TR> setup.py automagicly adds ssl to build the '_socket'
    TR> extension, how does it know which version to use? How would it
    TR> even find the one not in /usr/local/ssl?  Normally you would
    TR> configure --with-ssl-dir=/some_path_to_ssl

I would hope that setup.py would match the library version with the
header version, but I'm sure it's not easy or foolproof.  Look at the
machinations setup.py goes through to build bsddbmodule.  I still
think it's best to do this so that Python has a chance of building out
of the box for the widest possible audience.  It's impossible to get
100% right.

    TR> As one who has done ports to many Open Source projects, I
    TR> found python one of the more difficult ones because the build
    TR> process uses python.  So for a new platform (or broken build)
    TR> you have to know python to build python.

Python's also a pretty complicated piece of software, what with each
extension module linking against non-standard libraries presenting its
own set of complications.  But what tool that you can portably count
on being present would you use instead?

-Barry


From quest@lysator.liu.se  Wed Oct  9 19:34:33 2002
From: quest@lysator.liu.se (Anders Qvist)
Date: Wed, 9 Oct 2002 20:34:33 +0200
Subject: [Python-Dev] Re: [snake-farm] Re: [farm-report] Daily Snake Farm report
In-Reply-To: <200210091817.g99IHqW16465@odiug.zope.com>
References: <49c9447952.4795249c94@icomcast.net> <20021009131142.GG30812@epoch.metaslash.com> <200210091817.g99IHqW16465@odiug.zope.com>
Message-ID: <20021009183432.GQ22400@lysator.liu.se>

On Wed, Oct 09, 2002 at 02:17:51PM -0400, Guido van Rossum wrote:
> I saw this too.  The fix was to add a setlocale() call to
> test___all__.py.  Read my checkin comments and weep.

The intel/linux-boxen in the green again:

  http://www.lysator.liu.se/~sfarmer/xeno/latest.html
-- 
Anders "Quest" Qvist

"We've all heard that a million monkeys banging on a million typewriters
will eventually reproduce the entire works of Shakespeare. Now, thanks 
to the Internet, we know this is not true." -- Robert Wilensky 


From guido@python.org  Wed Oct  9 19:50:54 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 14:50:54 -0400
Subject: [Python-Dev] Multibyte repr()
In-Reply-To: Your message of "Wed, 09 Oct 2002 14:17:51 EDT."
 <200210091817.g99IHqW16465@odiug.zope.com>
References: <49c9447952.4795249c94@icomcast.net> <20021009131142.GG30812@epoch.metaslash.com>
 <200210091817.g99IHqW16465@odiug.zope.com>
Message-ID: <200210091850.g99Iosb17345@odiug.zope.com>

> I saw this too.  The fix was to add a setlocale() call to
> test___all__.py.  Read my checkin comments and weep.

I told this all to Tim, and he had one comment.  The repr() function
of an 8-bit string can now return characters with the high bit set.
This was the direct cause of the failures.  It was introduced in the
following patch:

----------------------------
revision 2.190
date: 2002/10/07 13:55:50;  author: loewis;  state: Exp;  lines: +68 -15
Patch #479898: Use multibyte C library for printing strings if available.
----------------------------

Was this really a good idea???

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 20:04:08 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 15:04:08 -0400
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: Your message of "Wed, 09 Oct 2002 14:50:54 EDT."
Message-ID: <200210091904.g99J48O17461@odiug.zope.com>

> I told this all to Tim, and he had one comment.  The repr() function
> of an 8-bit string can now return characters with the high bit set.
> This was the direct cause of the failures.  It was introduced in the
> following patch:
> 
> ----------------------------
> revision 2.190
> date: 2002/10/07 13:55:50;  author: loewis;  state: Exp;  lines: +68 -15
> Patch #479898: Use multibyte C library for printing strings if available.
> ----------------------------
> 
> Was this really a good idea???

Here's an example of what I mean.

Python 2.2:

  >>> u = u'\u1f40'
  >>> s = u.encode('utf8')
  >>> s
  '\xe1\xbd\x80'
  >>> 

Python 2.3:

  >>> u = u'\u1f40'
  >>> s = u.encode('utf8')
  >>> s
  'á½\x80'
  >>>

The latter output is not helpful, because the encoding of s is not the
locale's encoding.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Oct  9 20:14:20 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 15:14:20 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
 <15779.28468.831725.1313@gargle.gargle.HOWL>
 <2mfzvgnefw.fsf@starship.python.net>
Message-ID: <15780.32780.635036.756155@gargle.gargle.HOWL>

>>>>> "MH" == Michael Hudson <mwh@python.net> writes:

    >> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:
    >> JJ> Aside from the usual suspects I have three tests failing in
    >> JJ> MacPython 2.2.2: test_atexit, test_email and test_httplib.
    >> Oops, looks like I'm also getting a segfault in test_re.

    MH> This is normal; run "ulimit -s 2048" before running the test
    MH> suite.

Thanks, that works.  Would it make sense to add this to the test suite
some how, either by adding a ulimit call to the makefile, or adding
some resource.setrlimit() call to site.py or test_{,s}re.py?

-Barry


From fredrik@pythonware.com  Wed Oct  9 20:19:21 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 9 Oct 2002 21:19:21 +0200
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com><15779.28468.831725.1313@gargle.gargle.HOWL><2mfzvgnefw.fsf@starship.python.net> <15780.32780.635036.756155@gargle.gargle.HOWL>
Message-ID: <095701c26fc8$c8bf5580$ced241d5@hagrid>

barry wrote:

>     MH> This is normal; run "ulimit -s 2048" before running the test
>     MH> suite.
> 
> Thanks, that works.  Would it make sense to add this to the test suite
> some how, either by adding a ulimit call to the makefile, or adding
> some resource.setrlimit() call to site.py or test_{,s}re.py?

or s?re.py...

</F>



From fredrik@pythonware.com  Wed Oct  9 20:21:34 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 9 Oct 2002 21:21:34 +0200
Subject: [Python-Dev] Multibyte repr()
References: <49c9447952.4795249c94@icomcast.net> <20021009131142.GG30812@epoch.metaslash.com>              <200210091817.g99IHqW16465@odiug.zope.com>  <200210091850.g99Iosb17345@odiug.zope.com>
Message-ID: <095a01c26fc9$162a8470$ced241d5@hagrid>

guido wrote:

> Was this really a good idea???

first impression: it's a really bad idea.

</F>



From martin@v.loewis.de  Wed Oct  9 20:58:27 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 21:58:27 +0200
Subject: [Python-Dev] Multibyte repr()
In-Reply-To: <200210091850.g99Iosb17345@odiug.zope.com>
References: <49c9447952.4795249c94@icomcast.net>
 <20021009131142.GG30812@epoch.metaslash.com>
 <200210091817.g99IHqW16465@odiug.zope.com>
 <200210091850.g99Iosb17345@odiug.zope.com>
Message-ID: <m3adlntme4.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> revision 2.190
> date: 2002/10/07 13:55:50;  author: loewis;  state: Exp;  lines: +68 -15
> Patch #479898: Use multibyte C library for printing strings if available.
> ----------------------------
> 
> Was this really a good idea???

I think the submitter wanted to get "proper" output in interactive
mode for strings. For most cases, it might be sufficient to active the
code for tp_print only, not for tp_repr.

Would that be better?

Regards,
Martin



From guido@python.org  Wed Oct  9 21:13:43 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 16:13:43 -0400
Subject: [Python-Dev] Multibyte repr()
In-Reply-To: Your message of "Wed, 09 Oct 2002 21:58:27 +0200."
 <m3adlntme4.fsf@mira.informatik.hu-berlin.de>
References: <49c9447952.4795249c94@icomcast.net> <20021009131142.GG30812@epoch.metaslash.com> <200210091817.g99IHqW16465@odiug.zope.com> <200210091850.g99Iosb17345@odiug.zope.com>
 <m3adlntme4.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210092013.g99KDhe26705@odiug.zope.com>

> > revision 2.190
> > date: 2002/10/07 13:55:50;  author: loewis;  state: Exp;  lines: +68 -15
> > Patch #479898: Use multibyte C library for printing strings if available.
> > ----------------------------
> > 
> > Was this really a good idea???
> 
> I think the submitter wanted to get "proper" output in interactive
> mode for strings. For most cases, it might be sufficient to active the
> code for tp_print only, not for tp_repr.
> 
> Would that be better?

Not really, because then it would break the equivalence between repr()
and entering a value at the command line.  It would also cause
surprises because sometimes repr() is invoked anyway (when there's a
container that doesn't have a tp_print function).

I wouldn't mind this so much if GNU readline didn't do an evil thing
by calling setlocale().  Maybe the readline C extension  should
restore the locale?

This patch seems to take care of it, except I'm not sure if we can
assume that <locale.h> and setlocale() are always available.  This
doesn't seem to have any ill effects on the operation of GNU readline:

//////////////////////////////////////////////////////////////////////
*** readline.c	2 Aug 2002 02:27:13 -0000	2.52
--- readline.c	9 Oct 2002 20:10:33 -0000
***************
*** 11,16 ****
--- 11,17 ----
  #include <setjmp.h>
  #include <signal.h>
  #include <errno.h>
+ #include <locale.h>
  
  /* GNU readline definitions */
  #undef HAVE_CONFIG_H /* Else readline/chardefs.h includes strings.h */
***************
*** 538,543 ****
--- 539,545 ----
  static void
  setup_readline(void)
  {
+ 	char *saved_locale = NULL;
  	using_history();
  
  	rl_readline_name = "python";
***************
*** 570,576 ****
--- 572,580 ----
  	 * XXX: A bug in the readline-2.2 library causes a memory leak
  	 * inside this function.  Nothing we can do about it.
  	 */
+ 	saved_locale = setlocale(LC_CTYPE, NULL);
  	rl_initialize();
+ 	setlocale(LC_CTYPE, saved_locale); /* Restore locale */
  }
  
  
//////////////////////////////////////////////////////////////////////

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Oct  9 21:34:06 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 09 Oct 2002 16:34:06 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference
 Implementation and new issues
In-Reply-To: <whu1jv8zox.fsf@olorin.ts.gatech.edu>
Message-ID: <BIEJKCLHCIOIHAGOKOLHAEMHDLAA.tim.one@comcast.net>

[Christopher A. Craig]
> I agree.  I initially thought about returning a decimal with str(),
> but for that reason chose not to.

I expect you'll have to in the end, though.  Getting an explicit ratio of
multi-thousand digit integers really does suck for a default ...

> It might be nice to have a method that returned the best floating point
> decimal approximation to within n decimal digits or shorter, but I don't
> want a builtin to make an arbitrary decision on where to cut off
> precision.

If you haven't yet, you should strive to understand Moshe Zadka's prototype
rational implementation in Python CVS nondist/sandbox/ (which has all the
approximation functions you could need).  Unfortunately, the SourceForge
Python Numerics list:

    https://sourceforge.net/mailarchive/forum.php?forum_id=2280

appears to have lost almost all the voluminous discussions that went into
that, retaining mostly a random sampling of spam.



From martin@v.loewis.de  Wed Oct  9 21:55:08 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 22:55:08 +0200
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15780.29440.641827.692637@gargle.gargle.HOWL>
References: <15779.46954.672730.965163@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net>
 <15780.29440.641827.692637@gargle.gargle.HOWL>
Message-ID: <m365wbtjrn.fsf@mira.informatik.hu-berlin.de>

barry@python.org (Barry A. Warsaw) writes:

> Since there seems to be so little agreement among *nixes about how to
> do this, it seems like the only/best place to codify all this
> knowledge is in distutils.  I still think specifying
> runtime_library_dirs to the Extension constructor is the right thing
> to do.  But maybe on some systems distutils ought to ignore it, or at
> least check to make sure that the dynamic libraries it wants to link
> against are actually present.

I think both is difficult: It is hard to find out what option to use
on what system, and how to find out whether a shared library is
present. As a starting point, you have to decide whether you want a
conservative strategy (white list of systems you know how to steer),
or an optimistic strategy (black list of systems on which the standard
strategy fails).

For 2.2.2, backing out any setup.py changes to add
runtime_library_dirs should be taking into consideration.

Regards,
Martin


From martin@v.loewis.de  Wed Oct  9 22:00:55 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 23:00:55 +0200
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <200210091904.g99J48O17461@odiug.zope.com>
References: <200210091904.g99J48O17461@odiug.zope.com>
Message-ID: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

>   >>> u = u'\u1f40'
>   >>> s = u.encode('utf8')
>   >>> s
>   'a=\x80'
>   >>>
> 
> The latter output is not helpful, because the encoding of s is not the
> locale's encoding.

[Somehow, the accents got lost in your message]

It isn't helpful, but it isn't strictly wrong, either. In this
specific case, people are used to see utf-8 being interpreted as
Latin-1 - that form of "mojibake" is very common, so they will know
what happened.

I question whether the hex representation is more helpful: it depends
on how you need to interpret the result you get.

Regards,
Martin



From martin@v.loewis.de  Wed Oct  9 22:07:23 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Oct 2002 23:07:23 +0200
Subject: [Python-Dev] Multibyte repr()
In-Reply-To: <200210092013.g99KDhe26705@odiug.zope.com>
References: <49c9447952.4795249c94@icomcast.net>
 <20021009131142.GG30812@epoch.metaslash.com>
 <200210091817.g99IHqW16465@odiug.zope.com>
 <200210091850.g99Iosb17345@odiug.zope.com>
 <m3adlntme4.fsf@mira.informatik.hu-berlin.de>
 <200210092013.g99KDhe26705@odiug.zope.com>
Message-ID: <m3r8ezs4ms.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> This patch seems to take care of it, except I'm not sure if we can
> assume that <locale.h> and setlocale() are always available.  This
> doesn't seem to have any ill effects on the operation of GNU readline:

The patch looks fine, but you need to wrap it with HAVE_LOCALE_H and
HAVE_SETLOCALE; both are already tested in configure.in.

Regards,
Martin



From guido@python.org  Wed Oct  9 22:07:33 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 17:07:33 -0400
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: Your message of "Wed, 09 Oct 2002 23:00:55 +0200."
 <m3zntns4xk.fsf@mira.informatik.hu-berlin.de>
References: <200210091904.g99J48O17461@odiug.zope.com>
 <m3zntns4xk.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210092107.g99L7XA26891@odiug.zope.com>

> Guido van Rossum <guido@python.org> writes:
> 
> >   >>> u = u'\u1f40'
> >   >>> s = u.encode('utf8')
> >   >>> s
> >   'a=\x80'
> >   >>>
> > 
> > The latter output is not helpful, because the encoding of s is not the
> > locale's encoding.

[Martin]
> [Somehow, the accents got lost in your message]
> 
> It isn't helpful, but it isn't strictly wrong, either. In this
> specific case, people are used to see utf-8 being interpreted as
> Latin-1 - that form of "mojibake" is very common, so they will know
> what happened.
> 
> I question whether the hex representation is more helpful: it depends
> on how you need to interpret the result you get.

Well, if you *want* to see the hex codes for all non-ASCII characters,
repr() used to be your friend.  No more.  If you *want* to see the
printable characters, you could always use print.

I'd be okay with this change if the default locale wasn't changed by
readline.  Did you see my patch for that?  Then people who want to see
their encoding from repr() can learn to put

    import locale
    locale.setlocale(locale.LC_CTYPE, "")

in their $PYTHONSTARTUP file (or in their app's main()).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 22:18:12 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 17:18:12 -0400
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Your message of "Wed, 09 Oct 2002 22:55:08 +0200."
 <m365wbtjrn.fsf@mira.informatik.hu-berlin.de>
References: <15779.46954.672730.965163@gargle.gargle.HOWL> <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net> <15780.29440.641827.692637@gargle.gargle.HOWL>
 <m365wbtjrn.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210092118.g99LICj26939@odiug.zope.com>

[Martin]
> For 2.2.2, backing out any setup.py changes to add
> runtime_library_dirs should be taking into consideration.

This sounds like a conservative solution.  There is exactly one such
option, for ssl.  The checkin message:

----------------------------
revision 1.73.4.9
date: 2002/09/30 14:42:29;  author: bwarsaw;  state: Exp;  lines: +2 -1
Add "runtime_library_dirs = ssl_libs" to the _socket Extension
specification so that the proper runtime ld.so path gets compiled into
the extension.  This fixes _socket for Solaris systems with libssl and
libcrypto in non-standard locations and should be fine for other
systems as well.  Closes SF bug #565710.

Forward port candidate for Python 2.3 (I'll work on that).
----------------------------

The diff (apart from a change to __version__) is:

***************
*** 364,369 ****
--- 364,370 ----
              exts.append( Extension('_socket', ['socketmodule.c'],
                                     include_dirs = ssl_incs,
                                     library_dirs = ssl_libs,
+                                    runtime_library_dirs = ssl_libs,
                                     libraries = ['ssl', 'crypto'],
                                     define_macros = [('USE_SSL',1)] ) )
          else:

Maybe we should be more conservative and add this option only for
Solaris?  How does one detect Solaris in setup.py?  How about this
patch (on top of the above one, not replacing it)?

*** setup.py	7 Oct 2002 10:38:33 -0000	1.73.4.10
--- setup.py	9 Oct 2002 21:15:04 -0000
***************
*** 361,370 ****
  
          if (ssl_incs is not None and
              ssl_libs is not None):
              exts.append( Extension('_socket', ['socketmodule.c'],
                                     include_dirs = ssl_incs,
                                     library_dirs = ssl_libs,
!                                    runtime_library_dirs = ssl_libs,
                                     libraries = ['ssl', 'crypto'],
                                     define_macros = [('USE_SSL',1)] ) )
          else:
--- 361,373 ----
  
          if (ssl_incs is not None and
              ssl_libs is not None):
+             rtlibs = None
+             if platform.startswith('sunos'):
+                 rtlibs = ssl_libs
              exts.append( Extension('_socket', ['socketmodule.c'],
                                     include_dirs = ssl_incs,
                                     library_dirs = ssl_libs,
!                                    runtime_library_dirs = rtlibs,
                                     libraries = ['ssl', 'crypto'],
                                     define_macros = [('USE_SSL',1)] ) )
          else:

I hope that Skip/Barry can test this on MacOS X and Martin on Solaris!

Note: the 2.3 setup.py contains other uses of runtime_library_dirs,
which probably need to be treated the same way.  Skip, can you cook up
patches there?  (The 2.2.2 fix for ssl also needs to be forward-ported!)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct  9 22:28:05 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 09 Oct 2002 17:28:05 -0400
Subject: [Python-Dev] Multibyte repr()
In-Reply-To: Your message of "Wed, 09 Oct 2002 23:07:23 +0200."
 <m3r8ezs4ms.fsf@mira.informatik.hu-berlin.de>
References: <49c9447952.4795249c94@icomcast.net> <20021009131142.GG30812@epoch.metaslash.com> <200210091817.g99IHqW16465@odiug.zope.com> <200210091850.g99Iosb17345@odiug.zope.com> <m3adlntme4.fsf@mira.informatik.hu-berlin.de> <200210092013.g99KDhe26705@odiug.zope.com>
 <m3r8ezs4ms.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210092128.g99LS6W27043@odiug.zope.com>

> > This patch seems to take care of it, except I'm not sure if we can
> > assume that <locale.h> and setlocale() are always available.  This
> > doesn't seem to have any ill effects on the operation of GNU readline:
> 
> The patch looks fine, but you need to wrap it with HAVE_LOCALE_H and
> HAVE_SETLOCALE; both are already tested in configure.in.

OK, done and checked in.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nhodgson@bigpond.net.au  Wed Oct  9 23:04:04 2002
From: nhodgson@bigpond.net.au (Neil Hodgson)
Date: Thu, 10 Oct 2002 08:04:04 +1000
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net><m3r8f3vum2.fsf@mira.informatik.hu-berlin.de><200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net> <8765wbak3n.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <005001c26fdf$c8609c40$3da48490@neil>

Stephen J. Turnbull:

> No current XEmacs recognizes cookies in the second line.  I tried it,
> it broke, I backed it out, had a bad attack of day job, and forgot to
> fix the patch.  :-(

   Where did the coding cookie originally come from? It seems to have
arrived in Python from somewhere else, probably Emacs but I couldn't find a
reference in PEP-263. Googling with a couple of sets of relevant terms
didn't reveal anything good within the top references. I have just
implemented the start of this feature in my SciTE editor and would like to
ensure it is compatible with other editors.

   Neil



From mats@laplaza.org  Thu Oct 10 00:48:15 2002
From: mats@laplaza.org (Mats Wichmann)
Date: Wed, 09 Oct 2002 17:48:15 -0600
Subject: [Python-Dev] 2.2.2b fails test_minidom on Itanium Linux
In-Reply-To: <20021009022002.15248.70185.Mailman@mail.python.org>
Message-ID: <5.1.0.14.1.20021009174456.01fce830@204.151.72.2>

Been tied up since the beta was announced;
the 2.2.2 CVS of this morning, which I
assume is very close if not identical to
the beta, fails with "lines 2-217 of output missing"
on the minidom test on Itanium (ia64) Linux.
I presume this is due to the test dying or
some resource missing so it doesn't get
started. No chance to check deeper just now.
This was fine just a bit ago when I sent
Guido some feedback pre-beta.

Mats



From barry@python.org  Thu Oct 10 01:30:35 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 20:30:35 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15779.46954.672730.965163@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net>
 <15780.29440.641827.692637@gargle.gargle.HOWL>
 <m365wbtjrn.fsf@mira.informatik.hu-berlin.de>
 <200210092118.g99LICj26939@odiug.zope.com>
Message-ID: <15780.51755.561632.395266@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Maybe we should be more conservative and add this option only
    GvR> for Solaris?

That was one suggestion I made when we talked about it last.  Sounds
like it's probably the right approach to whitelist the platforms we
know are broken without it.  I'm running a test on SF's compile farm
Solaris machine now.
    
    GvR> How does one detect Solaris in setup.py?  How about this
    GvR> patch (on top of the above one, not replacing it)?

That's what I'm trying.

    GvR> I hope that Skip/Barry can test this on MacOS X and Martin on
    GvR> Solaris!

I'll try OSX momentarily.

    GvR> Note: the 2.3 setup.py contains other uses of
    GvR> runtime_library_dirs, which probably need to be treated the
    GvR> same way.

But maybe not.  If you install any modern BerkeleyDB from source it
always sticks it in a non-standard location.  Hmm, maybe a blacklist
is more useful for this one?  Jeez this stuff sucks.
    
    GvR> Skip, can you cook up patches there?  (The 2.2.2
    GvR> fix for ssl also needs to be forward-ported!)

Right, sorry, I'll do that if Guido's changes pans out for Solaris,
Linux, and OSX.

(Side note: I think I will have a few more minor patches for the email
package, which I'll try to get into 2.2.2 tonight, if my cold will lay
low for a few hours.)

-Barry


From barry@python.org  Thu Oct 10 01:45:38 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 20:45:38 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15779.46954.672730.965163@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210090732590.1703-100000@ou8.int.multitalents.net>
 <15780.29440.641827.692637@gargle.gargle.HOWL>
 <m365wbtjrn.fsf@mira.informatik.hu-berlin.de>
 <200210092118.g99LICj26939@odiug.zope.com>
 <15780.51755.561632.395266@gargle.gargle.HOWL>
Message-ID: <15780.52658.412013.959026@gargle.gargle.HOWL>

>>>>> "BAW" == Barry A Warsaw <barry@python.org> writes:

    GvR> I hope that Skip/Barry can test this on MacOS X and Martin on
    GvR> Solaris!

    BAW> I'll try OSX momentarily.

The patch seems fine on RH7.3, and Solaris 8.  OSX build not yet done,
but I suspect it'll work.  If so, I'll commit the change to setup.py
for 2.2.2 and forward port it to 2.3.

-Barry


From tim@multitalents.net  Thu Oct 10 03:01:27 2002
From: tim@multitalents.net (Tim Rice)
Date: Wed, 9 Oct 2002 19:01:27 -0700 (PDT)
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210092118.g99LICj26939@odiug.zope.com>
Message-ID: <Pine.UW2.4.44.0210091756050.14949-100000@ou8.int.multitalents.net>

On Wed, 9 Oct 2002, Guido van Rossum wrote:

> [Martin]
> > For 2.2.2, backing out any setup.py changes to add
> > runtime_library_dirs should be taking into consideration.
>
> This sounds like a conservative solution.  There is exactly one such
> option, for ssl.  The checkin message:
>
[snip]
> +                                    runtime_library_dirs = ssl_libs,

The only real problem is -R seems to be added for all unix platforms.
This change would benefit all my platforms if it handled -R properly
In OpenSSH we only use -R on solaris and freebsd.
Perhaps runtime_library_dir_option() in Lib/distutils/unixccompiler.py
needs some work.


-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net





From barry@python.org  Thu Oct 10 03:09:33 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 22:09:33 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <200210092118.g99LICj26939@odiug.zope.com>
 <Pine.UW2.4.44.0210091756050.14949-100000@ou8.int.multitalents.net>
Message-ID: <15780.57693.291891.342258@gargle.gargle.HOWL>

>>>>> "TR" == Tim Rice <tim@multitalents.net> writes:

    TR> The only real problem is -R seems to be added for all unix
    TR> platforms.  This change would benefit all my platforms if it
    TR> handled -R properly In OpenSSH we only use -R on solaris and
    TR> freebsd.  Perhaps runtime_library_dir_option() in
    TR> Lib/distutils/unixccompiler.py needs some work.

Anybody care to try a build on SF's freebsd system?  If not, I'll try
it after the Solaris build finishes.

-Barry


From tim@multitalents.net  Thu Oct 10 03:25:15 2002
From: tim@multitalents.net (Tim Rice)
Date: Wed, 9 Oct 2002 19:25:15 -0700 (PDT)
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15780.57693.291891.342258@gargle.gargle.HOWL>
Message-ID: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>

On Wed, 9 Oct 2002, Barry A. Warsaw wrote:

>
> >>>>> "TR" == Tim Rice <tim@multitalents.net> writes:
>
>     TR> The only real problem is -R seems to be added for all unix
>     TR> platforms.  This change would benefit all my platforms if it
>     TR> handled -R properly In OpenSSH we only use -R on solaris and
>     TR> freebsd.  Perhaps runtime_library_dir_option() in
>     TR> Lib/distutils/unixccompiler.py needs some work.
>

I just tried Guido's setup.py patch on SCO Open Server
and it works there.  No more -R. :-)

Am I out to lunch on the idea that the logic should be in
Lib/distutils/unixccompiler.py where the -R is added?

> Anybody care to try a build on SF's freebsd system?  If not, I'll try
> it after the Solaris build finishes.
>
> -Barry
>

-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net




From barry@python.org  Thu Oct 10 03:35:42 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 9 Oct 2002 22:35:42 -0400
Subject: [Python-Dev] Where is this flag coming from?
References: <15780.57693.291891.342258@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
Message-ID: <15780.59262.401827.568672@gargle.gargle.HOWL>

>>>>> "TR" == Tim Rice <tim@multitalents.net> writes:

    TR> I just tried Guido's setup.py patch on SCO Open Server
    TR> and it works there.  No more -R. :-)

Cool.  I think we're squared away for Python 2.2.2.

    TR> Am I out to lunch on the idea that the logic should be in
    TR> Lib/distutils/unixccompiler.py where the -R is added?

That's what Skip checked in but only for Python 2.3 it seems.
I'm a little confused now about what should be ported from 2.2.2 to
2.3, but maybe I'm still foggied headed from my cold.

The SF freebsd build of 2.2.2 seemed to work fine, except that
test_sax failed for reasons I don't know and don't have the energy
right now to investigate.  2.2.2 works for me on all of RH7.3, Solaris
8, and OSX 10.2 (except for test_locale on the latter which has
already been brought up).

Python 2.3cvs is working fine for me on RH7.3.  The Solaris 8 and OSX
builds are still running, and I can't do the freebsd build until the
Solaris one ends (well, I could, but it's easier to just wait).

-Barry


From tim@multitalents.net  Thu Oct 10 05:26:33 2002
From: tim@multitalents.net (Tim Rice)
Date: Wed, 9 Oct 2002 21:26:33 -0700 (PDT)
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15780.59262.401827.568672@gargle.gargle.HOWL>
Message-ID: <Pine.UW2.4.44.0210092123470.14949-100000@ou8.int.multitalents.net>

On Wed, 9 Oct 2002, Barry A. Warsaw wrote:

>
> >>>>> "TR" == Tim Rice <tim@multitalents.net> writes:
>
>     TR> I just tried Guido's setup.py patch on SCO Open Server
>     TR> and it works there.  No more -R. :-)
>
> Cool.  I think we're squared away for Python 2.2.2.
>
>     TR> Am I out to lunch on the idea that the logic should be in
>     TR> Lib/distutils/unixccompiler.py where the -R is added?
>
> That's what Skip checked in but only for Python 2.3 it seems.

The runtime_library_dir_option function seems to be
identical in both 2.3 and 2.2.2

> I'm a little confused now about what should be ported from 2.2.2 to
> 2.3, but maybe I'm still foggied headed from my cold.
>
[snip]
> -Barry
>

-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net




From mhammond@skippinet.com.au  Thu Oct 10 05:42:16 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 10 Oct 2002 14:42:16 +1000
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib ntpath.py,1.50,1.51
In-Reply-To: <2md6qjd9zh.fsf@starship.python.net>
Message-ID: <LCEPIIGDJPKCOIHOBJEPOELBGMAA.mhammond@skippinet.com.au>

[Tim in a checkin message]

>> isn't anyone in PythonDevLand running CVS on Linux?!).
> 
> Yes, but I do sleep occasionally...

And my Linux box was stolen!

New-one-supposed-to-be-here-yesterday--damn-dodgy-retailers ly

Mark.



From ishimoto@gembook.org  Thu Oct 10 06:10:01 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Thu, 10 Oct 2002 14:10:01 +0900
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <200210092107.g99L7XA26891@odiug.zope.com>
References: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de> <200210092107.g99L7XA26891@odiug.zope.com>
Message-ID: <20021010114451.7791.ISHIMOTO@gembook.org>

Hello,

On Wed, 09 Oct 2002 17:07:33 -0400
Guido van Rossum <guido@python.org> wrote:

> Well, if you *want* to see the hex codes for all non-ASCII characters,
> repr() used to be your friend.  No more.  If you *want* to see the
> printable characters, you could always use print.

I'm happy with this. I'm distributing modified version of Python Win32
installer at http://www.python.jp/Zope/download/pythonjpdist. This
version of Python contains similar modifications for Japanese ShiftJIS
users.

But this patch has one problem. Because result of repr() depends on
locale setting, we cannot assume text-form pickle could be restored
everywhere. For example, under Japanese ShiftJIS locale, 

>>> s = '\x83\x5c'  # This is a multi-byte character, third letter of "Python"
>>>                 # in Japanese. Note that trailing character is '\'
>>> 
>>> pickle.dump(s, f)

I assume CVS version of Python fails to load this pickled object because
backslash followed by quote is illegal. This problem may happens for
Japanese ShiftJIS encoding, but I don't know whether there are another
encodings causes same problem or not.

I think this is not a major problem since we can avoid this by using
binary form pickle, or using Unicode for text form pickle. But to
eliminate this problem entirely, Python can have another slot to get a
string representation of object, may be named tp_dumps. 
tp_dumps always returns hex codes for codes for all non-ASCII characters
and is called whenever valid Python string literals are required.

Regards,

--------------------------
Atsuo Ishimoto
ishimoto@gembook.jp
Homepage:http://www.gembook.jp



From perky@fallin.lv  Thu Oct 10 07:42:37 2002
From: perky@fallin.lv (Hye-Shik Chang)
Date: Thu, 10 Oct 2002 15:42:37 +0900
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <20021010114451.7791.ISHIMOTO@gembook.org>
References: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de> <200210092107.g99L7XA26891@odiug.zope.com> <20021010114451.7791.ISHIMOTO@gembook.org>
Message-ID: <20021010064237.GA8526@fallin.lv>

Hi!

On Thu, Oct 10, 2002 at 02:10:01PM +0900, Atsuo Ishimoto wrote:
> Hello,
> 
> On Wed, 09 Oct 2002 17:07:33 -0400
> Guido van Rossum <guido@python.org> wrote:
> 
> > Well, if you *want* to see the hex codes for all non-ASCII characters,
> > repr() used to be your friend.  No more.  If you *want* to see the
> > printable characters, you could always use print.
> 
> I'm happy with this. I'm distributing modified version of Python Win32
> installer at http://www.python.jp/Zope/download/pythonjpdist. This
> version of Python contains similar modifications for Japanese ShiftJIS
> users.
> 
> But this patch has one problem. Because result of repr() depends on
> locale setting, we cannot assume text-form pickle could be restored
> everywhere. For example, under Japanese ShiftJIS locale, 

I realized that string_repr's depending on locale can be a problem
maker for many unexpected situations. What I wanted in this patch is
just to see _real_ string even in lists or dictionaries.
I and CJKV users may feel happy even without string_repr locale patch.

Regards,
-- 
Hye-Shik Chang <perky@fallin.lv>
Yonsei University, Seoul
^D


From ishimoto@gembook.org  Thu Oct 10 09:28:29 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Thu, 10 Oct 2002 17:28:29 +0900
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <20021010064237.GA8526@fallin.lv>
References: <20021010114451.7791.ISHIMOTO@gembook.org> <20021010064237.GA8526@fallin.lv>
Message-ID: <20021010165744.7795.ISHIMOTO@gembook.org>

Hi, Hye-Shik Chang. Thank you for submitting nice patch.

On Thu, 10 Oct 2002 15:42:37 +0900
Hye-Shik Chang <perky@fallin.lv> wrote:

> I and CJKV users may feel happy even without string_repr locale patch.

But this doesn't work for Idle or PythonWin. If sys.stdout is not real
File object, PyFile_WriteObject() calls PyObject_Str() to get string to
print. On printing list or dictionary, this will generate repr()ed
string.

Regards,
--------------------------
Atsuo Ishimoto
ishimoto@gembook.org
Homepage:http://www.gembook.jp



From Armin Rigo <arigo@tunes.org>  Thu Oct 10 09:52:19 2002
From: Armin Rigo <arigo@tunes.org> (Armin Rigo)
Date: Thu, 10 Oct 2002 10:52:19 +0200 (CEST)
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: <200210091555.g99FtD011873@odiug.zope.com>
Message-ID: <Pine.OSF.3.96.1021010104159.1623327A-100000@aster3.ulb.ac.be>

Hello Guido,

On Wed, 9 Oct 2002, Guido van Rossum wrote:
> [Armin]
> > You can already crash the interpreter with pure Python code, for example
> > via the new.code() constructor or by writing crappy .pyc files.
> 
> Yes, and I occasionally lose sleep over those.  I definitely don't
> want to add more loopholes like that, and I'd like to fix those.

Ok.  I already more or less committed to have a serious look over all the
existing overflow problems that could cause a core dump
(http://www.python.org/sf/618623).  I may add in my to-do list the checks
that no boggy data can crash unmarshall (after a first survey of the code
I'd say that such data might exist; e.g. r_string() has a memcpy() that
could be called with a negative size).

More importantly, if you think that there is use for a complete "code
object checker" that could catch any code doing silly things, then I'd
love to contribute.  Note however that this is something that will have to
be kept closely in sync with future compiler extensions. 


A bientot,

Armin.



From mwh@python.net  Thu Oct 10 09:55:17 2002
From: mwh@python.net (Michael Hudson)
Date: 10 Oct 2002 09:55:17 +0100
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Tim Rice's message of "Wed, 9 Oct 2002 19:25:15 -0700 (PDT)"
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
Message-ID: <2m3crey8p6.fsf@starship.python.net>

Tim Rice <tim@multitalents.net> writes:

> Am I out to lunch on the idea that the logic should be in
> Lib/distutils/unixccompiler.py where the -R is added?

No.

The code we have right now is dubious, to say the least.

Cheers,
M.

-- 
  ... so the notion that it is meaningful to pass pointers to memory
  objects into which any random function may write random values
  without having a clue where they point, has _not_ been debunked as
  the sheer idiocy it really is.        -- Erik Naggum, comp.lang.lisp


From mwh@python.net  Thu Oct 10 09:57:46 2002
From: mwh@python.net (Michael Hudson)
Date: 10 Oct 2002 09:57:46 +0100
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: barry@python.org's message of "Wed, 9 Oct 2002 15:14:20 -0400"
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com> <15779.28468.831725.1313@gargle.gargle.HOWL> <2mfzvgnefw.fsf@starship.python.net> <15780.32780.635036.756155@gargle.gargle.HOWL>
Message-ID: <2mzntmwu0l.fsf@starship.python.net>

barry@python.org (Barry A. Warsaw) writes:

> >>>>> "MH" == Michael Hudson <mwh@python.net> writes:
> 
>     >> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:
>     >> JJ> Aside from the usual suspects I have three tests failing in
>     >> JJ> MacPython 2.2.2: test_atexit, test_email and test_httplib.
>     >> Oops, looks like I'm also getting a segfault in test_re.
> 
>     MH> This is normal; run "ulimit -s 2048" before running the test
>     MH> suite.
> 
> Thanks, that works.  Would it make sense to add this to the test suite
> some how, either by adding a ulimit call to the makefile, or adding
> some resource.setrlimit() call to site.py or test_{,s}re.py?

There's a patch of sf to do this.  It's probably a good idea (I forget
to do this fairly often).  Or maybe it could be in
Modules/main.c:main()...

Cheers,
M.

-- 
39. Re graphics:  A picture is worth 10K words - but only those
    to describe the picture.  Hardly any sets of 10K words can be
    adequately described with pictures.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From mwh@python.net  Thu Oct 10 10:00:44 2002
From: mwh@python.net (Michael Hudson)
Date: 10 Oct 2002 10:00:44 +0100
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
In-Reply-To: "Neil Hodgson"'s message of "Thu, 10 Oct 2002 08:04:04 +1000"
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de> <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net> <8765wbak3n.fsf@tleepslib.sk.tsukuba.ac.jp> <005001c26fdf$c8609c40$3da48490@neil>
Message-ID: <2mwuoqwtvn.fsf@starship.python.net>

"Neil Hodgson" <nhodgson@bigpond.net.au> writes:

> Stephen J. Turnbull:
> 
> > No current XEmacs recognizes cookies in the second line.  I tried it,
> > it broke, I backed it out, had a bad attack of day job, and forgot to
> > fix the patch.  :-(
> 
>    Where did the coding cookie originally come from? It seems to have
> arrived in Python from somewhere else, probably Emacs but I couldn't find a
> reference in PEP-263. 

>From the FSF Emacs 21.1 info file (node "Recognizing Coding Systems"):

       You can specify the coding system for a particular file using
    the `-*-...-*-' construct at the beginning of a file, or a local
    variables list at the end (*note File Variables::).  You do this
    by defining a value for the "variable" named `coding'.  Emacs does
    not really have a variable `coding'; instead of setting a
    variable, this uses the specified coding system for the file.  For
    example, `-*-mode: C; coding: latin-1;-*-' specifies use of the
    Latin-1 coding system, as well as C mode.  When you specify the
    coding explicitly in the file, that overrides
    `file-coding-system-alist'.

Not sure if this existed in 20.x.

I think the emacs manual is online somewhere, but not sure where.

Cheers,
M.

-- 
  For their next act, they'll no doubt be buying a firewall 
  running under NT, which makes about as much sense as 
  building a prison out of meringue.                     -- -:Tanuki:-
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From fredrik@pythonware.com  Thu Oct 10 10:13:07 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 10 Oct 2002 11:13:07 +0200
Subject: [Python-Dev] Non-ASCII characters in test_pep277.py in 2.3
References: <200210062218.g96MItv30307@pcp02138704pcs.reston01.va.comcast.net> <m3r8f3vum2.fsf@mira.informatik.hu-berlin.de> <200210071220.g97CKRG06496@pcp02138704pcs.reston01.va.comcast.net> <8765wbak3n.fsf@tleepslib.sk.tsukuba.ac.jp> <005001c26fdf$c8609c40$3da48490@neil> <2mwuoqwtvn.fsf@starship.python.net>
Message-ID: <019601c2703d$40c58e40$ced241d5@hagrid>

michael wrote:

> >From the FSF Emacs 21.1 info file (node "Recognizing Coding Systems"):
> 
>        You can specify the coding system for a particular file...
> 
> I think the emacs manual is online somewhere, but not sure where.

tried googling for "You can specify the coding system for a particular file"?

here's the "I feel lucky" hit:

    http://www.delorie.com/gnu/docs/emacs/emacs_195.html

</F>



From mwh@python.net  Thu Oct 10 10:24:34 2002
From: mwh@python.net (Michael Hudson)
Date: 10 Oct 2002 10:24:34 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib os.py,1.62,1.63 dospath.py,1.30,NONE
In-Reply-To: loewis@users.sourceforge.net's message of "Wed, 09 Oct 2002 10:23:32 -0700"
References: <E17zKYW-0006MI-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2mvg4aeje5.fsf@starship.python.net>

loewis@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv24338
> 
> Modified Files:
> 	os.py 
> Removed Files:
> 	dospath.py 
> Log Message:
> Remove more DOS support.

I think you should remove test_dospath & friends, too?

Cheers,
M.


From guido@python.org  Thu Oct 10 13:05:26 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 08:05:26 -0400
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Your message of "Wed, 09 Oct 2002 19:25:15 PDT."
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
Message-ID: <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>

> I just tried Guido's setup.py patch on SCO Open Server
> and it works there.  No more -R. :-)
> 
> Am I out to lunch on the idea that the logic should be in
> Lib/distutils/unixccompiler.py where the -R is added?

Well, if we knew what to do on other systems instead of -R, that would
help...

I'm curious if perhaps Apple's loader (which started this) interprets
-L as meaning both -L and -R in traditional Unix?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct 10 13:21:30 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 08:21:30 -0400
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: Your message of "Thu, 10 Oct 2002 10:52:19 +0200."
 <Pine.OSF.3.96.1021010104159.1623327A-100000@aster3.ulb.ac.be>
References: <Pine.OSF.3.96.1021010104159.1623327A-100000@aster3.ulb.ac.be>
Message-ID: <200210101221.g9ACLVL06906@pcp02138704pcs.reston01.va.comcast.net>

> > [Armin]
> > > You can already crash the interpreter with pure Python code, for example
> > > via the new.code() constructor or by writing crappy .pyc files.

[Guido]
> > Yes, and I occasionally lose sleep over those.  I definitely don't
> > want to add more loopholes like that, and I'd like to fix those.

[Armin]
> Ok.  I already more or less committed to have a serious look over
> all the existing overflow problems that could cause a core dump
> (http://www.python.org/sf/618623).  I may add in my to-do list the
> checks that no boggy data can crash unmarshall (after a first survey
> of the code I'd say that such data might exist; e.g. r_string() has
> a memcpy() that could be called with a negative size).

Fixes for this and similar problems will be most welcome!

> More importantly, if you think that there is use for a complete
> "code object checker" that could catch any code doing silly things,
> then I'd love to contribute.  Note however that this is something
> that will have to be kept closely in sync with future compiler
> extensions.

Yes, the fact that our VM isn't standardized makes this a bit tricky.
But I think it ought to be done.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct 10 13:59:26 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 08:59:26 -0400
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: Your message of "Thu, 10 Oct 2002 14:10:01 +0900."
 <20021010114451.7791.ISHIMOTO@gembook.org>
References: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de> <200210092107.g99L7XA26891@odiug.zope.com>
 <20021010114451.7791.ISHIMOTO@gembook.org>
Message-ID: <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>

[Guido]
> > Well, if you *want* to see the hex codes for all non-ASCII characters,
> > repr() used to be your friend.  No more.  If you *want* to see the
> > printable characters, you could always use print.

[Atsuo Ishimoto]
> I'm happy with this.

"This" was ambiguous.  Are you happy with what's in current CVS, or
with the old repr()?

> I'm distributing modified version of Python Win32
> installer at http://www.python.jp/Zope/download/pythonjpdist. This
> version of Python contains similar modifications for Japanese ShiftJIS
> users.
> 
> But this patch has one problem. Because result of repr() depends on
> locale setting, we cannot assume text-form pickle could be restored
> everywhere. For example, under Japanese ShiftJIS locale, 
> 
> >>> s = '\x83\x5c'  # This is a multi-byte character, third letter of "Python"
> >>>                 # in Japanese. Note that trailing character is '\'
> >>> 
> >>> pickle.dump(s, f)
> 
> I assume CVS version of Python fails to load this pickled object because
> backslash followed by quote is illegal. This problem may happens for
> Japanese ShiftJIS encoding, but I don't know whether there are another
> encodings causes same problem or not.

I tried this, and I could not find any problems with the resulting
pickle.  The pickle looks like this:

"S'\\x83\\\\'\np0\n."

I couldn't get this to fail loading in Python 2.1, 2.2 or 2.3 (CVS);
I tried both pickle and cPickle.

> I think this is not a major problem since we can avoid this by using
> binary form pickle, or using Unicode for text form pickle. But to
> eliminate this problem entirely, Python can have another slot to get a
> string representation of object, may be named tp_dumps. 
> tp_dumps always returns hex codes for codes for all non-ASCII characters
> and is called whenever valid Python string literals are required.

I don't think this particular issue (pickling) is a problem.  But I
*do* continue to worry that making repr() depend on the locale may be
a bigger problem than what it attempts to solve.

[Hye-Shik Chang]
> I realized that string_repr's depending on locale can be a problem
> maker for many unexpected situations. What I wanted in this patch is
> just to see _real_ string even in lists or dictionaries.
> I and CJKV users may feel happy even without string_repr locale patch.

I'm not sure I follow.  What is the alternative that you propose?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Thu Oct 10 14:04:22 2002
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 10 Oct 2002 09:04:22 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
 <15779.28468.831725.1313@gargle.gargle.HOWL>
 <2mfzvgnefw.fsf@starship.python.net>
 <15780.32780.635036.756155@gargle.gargle.HOWL>
 <2mzntmwu0l.fsf@starship.python.net>
Message-ID: <15781.31446.445161.102321@gargle.gargle.HOWL>

>>>>> "MH" == Michael Hudson <mwh@python.net> writes:

    MH> There's a patch of sf to do this.  It's probably a good idea
    MH> (I forget to do this fairly often).  Or maybe it could be in
    MH> Modules/main.c:main()...

Do you mean #580869?  It's not really much of a patch, e.g. I can't
tell if Steve actually tried that or what file he put it in.  I've
assigned that PR to myself and will look into it for Python 2.3.  Skip
or Jack can feel free to steal it from me if they want to raise the
urgency on it.

-Barry


From Jack.Jansen@cwi.nl  Thu Oct 10 15:24:21 2002
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Thu, 10 Oct 2002 16:24:21 +0200
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <F843BAEC-DC5B-11D6-AA09-0030655234CE@cwi.nl>

On Thursday, Oct 10, 2002, at 14:05 Europe/Amsterdam, Guido van Rossum 
wrote:
>
> I'm curious if perhaps Apple's loader (which started this) interprets
> -L as meaning both -L and -R in traditional Unix?

For frameworks that is what happens, and while I haven't tried it for 
normal shared libraries the documentation suggest that this is indeed 
the case.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From list-python@ccraig.org  Thu Oct 10 16:19:47 2002
From: list-python@ccraig.org (Christopher A. Craig)
Date: 10 Oct 2002 11:19:47 -0400
Subject: [Python-Dev] Re: PEP239 (Rational Numbers) Reference Implementation and new issues
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHAEMHDLAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHAEMHDLAA.tim.one@comcast.net>
Message-ID: <whhefus4mk.fsf@olorin.ts.gatech.edu>

Tim Peters <tim.one@comcast.net> writes:

> If you haven't yet, you should strive to understand Moshe Zadka's prototype
> rational implementation in Python CVS nondist/sandbox/ (which has all the
> approximation functions you could need).  Unfortunately, the SourceForge
> Python Numerics list:
> 
>     https://sourceforge.net/mailarchive/forum.php?forum_id=2280
> 
> appears to have lost almost all the voluminous discussions that went into
> that, retaining mostly a random sampling of spam.

I have read the module.  I really liked his (your?) trim code and I
used that algorithm in cRat and my rational patch.  I don't think I'd
use any of that for generating decimal (or binary) float
approximations, though.  Trim introduces some error into the
calculation which is then furthered by the float division.  You can
provably generate the closest scaled long of a given size (and thus
hopefully the closest float) with one shift and one long integer
division.

-- 
Christopher A. Craig <list-python@ccraig.org>
"It mearly pleases me to behave in a certain way to what appears to be a
cat" -- The Ruler of the Universe (The Restaurant at the End of the Universe)


From skip@pobox.com  Thu Oct 10 17:09:14 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 11:09:14 -0500
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: <15781.31446.445161.102321@gargle.gargle.HOWL>
References: <7936B11E-DB07-11D6-B250-003065517236@oratrix.com>
 <15779.28468.831725.1313@gargle.gargle.HOWL>
 <2mfzvgnefw.fsf@starship.python.net>
 <15780.32780.635036.756155@gargle.gargle.HOWL>
 <2mzntmwu0l.fsf@starship.python.net>
 <15781.31446.445161.102321@gargle.gargle.HOWL>
Message-ID: <15781.42538.323678.806285@montanaro.dyndns.org>

    BAW> Do you mean #580869?

I gave it a try, but was unable to raise my stack size.  I attached a file
to the patch containing my change to Lib/test/regrtest.py.

Skip


From skip@pobox.com  Thu Oct 10 04:25:18 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 9 Oct 2002 22:25:18 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
References: <15780.57693.291891.342258@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
Message-ID: <15780.62238.554661.203642@montanaro.dyndns.org>

    Tim> Am I out to lunch on the idea that the logic should be in
    Tim> Lib/distutils/unixccompiler.py where the -R is added?

I don't think so.  How would you adjust it to work for SCO?

Skip


From skip@pobox.com  Thu Oct 10 15:28:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 09:28:32 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15781.36496.719015.157394@montanaro.dyndns.org>

    Guido> I'm curious if perhaps Apple's loader (which started this)
    Guido> interprets -L as meaning both -L and -R in traditional Unix?

Damned if I could tell by reading the ld(1) manpage.  The change I checked
in to unixccompiler.py returns -L only because I figured "-L<dir>" would be
innocuous, not because I expected it to have "-R<dir>" semantics.

I'll ask around on the Pythonmac-SIG.

Skip


From skip@pobox.com  Thu Oct 10 04:08:34 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 9 Oct 2002 22:08:34 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210091536.g99FaVa11272@odiug.zope.com>
References: <15779.41332.251210.360418@montanaro.dyndns.org>
 <Pine.UW2.4.44.0210082135290.19141-100000@ou8.int.multitalents.net>
 <15779.46954.672730.965163@gargle.gargle.HOWL>
 <200210091536.g99FaVa11272@odiug.zope.com>
Message-ID: <15780.61234.606750.832257@montanaro.dyndns.org>

    >> Shouldn't distutils be taught what to do with runtime_library_dirs
    >> for SCO and MacOSX?

    Guido> My thoughts exactly.

Given that MacOSX's linker doesn't seem to understand -R at all (no matter
how you deliver it), I adjusted the code in runtime_library_dir_option (in
unixccompiler.py) to return "-L" + dir instead of either of the "-R"
possibilities.  This is probably not as good as it could be, but I couldn't
tease anything from the ld(1) man page which looked to me like an equivalent
to "-R" or LD_RUN_PATH.  Perhaps someone with more experience with MacOSX or
link editors in general would like to take a peek at the problem.  If so,
let me know and I'll be happy to shoot you a copy of the ld(1) man page.

Skip


From skip@pobox.com  Thu Oct 10 04:38:02 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 9 Oct 2002 22:38:02 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15780.59262.401827.568672@gargle.gargle.HOWL>
References: <15780.57693.291891.342258@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <15780.59262.401827.568672@gargle.gargle.HOWL>
Message-ID: <15780.63002.206139.926109@montanaro.dyndns.org>

    TR> Am I out to lunch on the idea that the logic should be in
    TR> Lib/distutils/unixccompiler.py where the -R is added?

    BAW> That's what Skip checked in but only for Python 2.3 it seems.  I'm
    BAW> a little confused now about what should be ported from 2.2.2 to
    BAW> 2.3, but maybe I'm still foggied headed from my cold.

I thought this was only a 2.3 thing.  I just tried on the release22_maint
CVS branch.  The bsddb module compiled and linked fine, but I don't know
why.  Oh, wait a minute...  The release22_maint branch doesn't have the
bsddb bells and whistles I added to setup.py a couple months ago.
Consequently, it's picking up the 1.85 version of the include file and
library (which, I will remind people has known breakage).

Should I backport those changes?  I think what's in 2.3's setup.py is much
more robust, but if people have been getting Berkeley DB 1.85 libraries in
2.2.1 and don't get a clean compile in 2.2.2 without editing setup.py
because 1.85 is all they have available, that might be construed as "more
than a bugfix".

Let me know what you want to do in this regard Guido.

Skip


From skip@pobox.com  Thu Oct 10 17:24:48 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 11:24:48 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <m3u1ju3gxh.fsf@mira.informatik.hu-berlin.de>
References: <15780.57693.291891.342258@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <15780.59262.401827.568672@gargle.gargle.HOWL>
 <m3u1ju3gxh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15781.43472.667036.609668@montanaro.dyndns.org>

    Martin> It seems that Skip chose the blacklist approach (aka the
    Martin> I-care-only-about-MacOS approach). Skip's approach is probably
    Martin> overkill, since there will be already -L options for all those
    Martin> directories. It appears that OS X records the full path of the
    Martin> library in the binary, so the notion of a runtime path is
    Martin> irrelevant.

I didn't want to return "" because that would require me to filter it out of
the argument list elsewhere.  "-L" + dir seemed like the most innocuous
change.

Skip



From guido@python.org  Thu Oct 10 17:34:58 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 12:34:58 -0400
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Your message of "Thu, 10 Oct 2002 09:28:32 CDT."
 <15781.36496.719015.157394@montanaro.dyndns.org>
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net> <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
 <15781.36496.719015.157394@montanaro.dyndns.org>
Message-ID: <200210101634.g9AGYwn14473@odiug.zope.com>

>     Guido> I'm curious if perhaps Apple's loader (which started this)
>     Guido> interprets -L as meaning both -L and -R in traditional Unix?
> 
> Damned if I could tell by reading the ld(1) manpage.  The change I
> checked in to unixccompiler.py returns -L only because I figured
> "-L<dir>" would be innocuous, not because I expected it to have
> "-R<dir>" semantics.

Why did you check it in to Python 2.3 but not to 2.2.2?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Oct 10 17:36:32 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 12:36:32 -0400
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: Your message of "Wed, 09 Oct 2002 22:38:02 CDT."
 <15780.63002.206139.926109@montanaro.dyndns.org>
References: <15780.57693.291891.342258@gargle.gargle.HOWL> <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net> <15780.59262.401827.568672@gargle.gargle.HOWL>
 <15780.63002.206139.926109@montanaro.dyndns.org>
Message-ID: <200210101636.g9AGaWM14486@odiug.zope.com>

> I thought this was only a 2.3 thing.

But this thread started with you reporting a 2.2.2 problem trying to
compile SSL!

> I just tried on the release22_maint CVS branch.  The bsddb module
> compiled and linked fine, but I don't know why.  Oh, wait a
> minute...  The release22_maint branch doesn't have the bsddb bells
> and whistles I added to setup.py a couple months ago.  Consequently,
> it's picking up the 1.85 version of the include file and library
> (which, I will remind people has known breakage).
> 
> Should I backport those changes?  I think what's in 2.3's setup.py
> is much more robust, but if people have been getting Berkeley DB
> 1.85 libraries in 2.2.1 and don't get a clean compile in 2.2.2
> without editing setup.py because 1.85 is all they have available,
> that might be construed as "more than a bugfix".
> 
> Let me know what you want to do in this regard Guido.

I think it's too precarious to fix in 2.2.2.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Oct 10 18:29:46 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 12:29:46 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210101634.g9AGYwn14473@odiug.zope.com>
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
 <15781.36496.719015.157394@montanaro.dyndns.org>
 <200210101634.g9AGYwn14473@odiug.zope.com>
Message-ID: <15781.47370.595649.754498@montanaro.dyndns.org>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    Guido> I'm curious if perhaps Apple's loader (which started this)
    Guido> interprets -L as meaning both -L and -R in traditional Unix?
    >> 
    >> Damned if I could tell by reading the ld(1) manpage.  The change I
    >> checked in to unixccompiler.py returns -L only because I figured
    >> "-L<dir>" would be innocuous, not because I expected it to have
    >> "-R<dir>" semantics.

    Guido> Why did you check it in to Python 2.3 but not to 2.2.2?

Because I couldn't build bsddb on 2.3 without it.

Skip



From skip@pobox.com  Thu Oct 10 18:30:58 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 12:30:58 -0500
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <200210101636.g9AGaWM14486@odiug.zope.com>
References: <15780.57693.291891.342258@gargle.gargle.HOWL>
 <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <15780.59262.401827.568672@gargle.gargle.HOWL>
 <15780.63002.206139.926109@montanaro.dyndns.org>
 <200210101636.g9AGaWM14486@odiug.zope.com>
Message-ID: <15781.47442.968831.672932@montanaro.dyndns.org>

    >> I thought this was only a 2.3 thing.

    Guido> But this thread started with you reporting a 2.2.2 problem trying
    Guido> to compile SSL!

Nope, it was bsddb on 2.3.  Someone else was talking about SSL, which I
could generally care less about.  I use bsddb all the time, however.

Skip


From sdm7g@mac.com  Thu Oct 10 18:43:11 2002
From: sdm7g@mac.com (Steven Majewski)
Date: Thu, 10 Oct 2002 13:43:11 -0400
Subject: [Python-Dev] resource.setrlimit on OSX [was:Tests failing in MacPython 2.2.2]
In-Reply-To: <15781.42538.323678.806285@montanaro.dyndns.org>
Message-ID: <BEBEAEAA-DC77-11D6-9E46-003065A7CFBA@mac.com>

On Thursday, October 10, 2002, at 12:09  PM, Skip Montanaro wrote:

>
>     BAW> Do you mean #580869?
>
> I gave it a try, but was unable to raise my stack size.  I attached a file
> to the patch containing my change to Lib/test/regrtest.py.
>
> Skip
>

Skip -- I saw the sourceforge notice and replied there before I saw this 
thread.

Yes, that fix still works for me and No, I'm not running it as root, but I 
am
still running 10.1.5, so if it's not working for you, maybe it's a 10.1 vs 
10.2
problem.

-- Steve



From martin@v.loewis.de  Thu Oct 10 19:12:10 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Oct 2002 20:12:10 +0200
Subject: [Python-Dev] Where is this flag coming from?
In-Reply-To: <15781.36496.719015.157394@montanaro.dyndns.org>
References: <Pine.UW2.4.44.0210091920460.14949-100000@ou8.int.multitalents.net>
 <200210101205.g9AC5Qm06716@pcp02138704pcs.reston01.va.comcast.net>
 <15781.36496.719015.157394@montanaro.dyndns.org>
Message-ID: <m3ptui6u4l.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> Damned if I could tell by reading the ld(1) manpage.  The change I checked
> in to unixccompiler.py returns -L only because I figured "-L<dir>" would be
> innocuous, not because I expected it to have "-R<dir>" semantics.

Thanks to 'otool -L', you'll see that the complete path of the library
is embedded into the executable.

Regards,
Martin



From martin@v.loewis.de  Thu Oct 10 19:25:21 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Oct 2002 20:25:21 +0200
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib os.py,1.62,1.63 dospath.py,1.30,NONE
In-Reply-To: <2mvg4aeje5.fsf@starship.python.net>
References: <E17zKYW-0006MI-00@usw-pr-cvs1.sourceforge.net>
 <2mvg4aeje5.fsf@starship.python.net>
Message-ID: <m3k7kq6tim.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> I think you should remove test_dospath & friends, too?

Done for test_dospath, and documentation. What other friends did you
have in mind?

Regards,
Martin


From mwh@python.net  Thu Oct 10 19:26:55 2002
From: mwh@python.net (Michael Hudson)
Date: Thu, 10 Oct 2002 19:26:55 +0100 (BST)
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib
 os.py,1.62,1.63 dospath.py,1.30,NONE
In-Reply-To: <m3k7kq6tim.fsf@mira.informatik.hu-berlin.de>
Message-ID: <Pine.LNX.4.44.0210101926240.8795-100000@starship.python.net>

On 10 Oct 2002, Martin v. Loewis wrote:

> Michael Hudson <mwh@python.net> writes:
> 
> > I think you should remove test_dospath & friends, too?
> 
> Done for test_dospath, and documentation. What other friends did you
> have in mind?

I was mainly just being vague.  I thought there was a test output file, 
but there wasn't/isn't.

Cheers,
M.



From martin@v.loewis.de  Thu Oct 10 19:29:27 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Oct 2002 20:29:27 +0200
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
References: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de>
 <200210092107.g99L7XA26891@odiug.zope.com>
 <20021010114451.7791.ISHIMOTO@gembook.org>
 <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3fzve6tbs.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > I realized that string_repr's depending on locale can be a problem
> > maker for many unexpected situations. What I wanted in this patch is
> > just to see _real_ string even in lists or dictionaries.
> > I and CJKV users may feel happy even without string_repr locale patch.
> 
> I'm not sure I follow.  What is the alternative that you propose?

I think Hye-Shik Chang would be happy if just tp_print acts according
to the locale, and "most" containers support tp_print. Given that
list, tuple, and dict have tp_print, no further action might be
needed.

I'm now convinced that having repr locale-specific is unacceptable. So
I would back something out. However, I still sympathize with the
original rationale, and I don't see a problem with tp_print differing
from tp_repr.

Regards,
Martin



From guido@python.org  Thu Oct 10 19:52:02 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 14:52:02 -0400
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: Your message of "Thu, 10 Oct 2002 20:29:27 +0200."
 <m3fzve6tbs.fsf@mira.informatik.hu-berlin.de>
References: <m3zntns4xk.fsf@mira.informatik.hu-berlin.de> <200210092107.g99L7XA26891@odiug.zope.com> <20021010114451.7791.ISHIMOTO@gembook.org> <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
 <m3fzve6tbs.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210101852.g9AIq2k14887@odiug.zope.com>

> > > I realized that string_repr's depending on locale can be a problem
> > > maker for many unexpected situations. What I wanted in this patch is
> > > just to see _real_ string even in lists or dictionaries.
> > > I and CJKV users may feel happy even without string_repr locale patch.
> > 
> > I'm not sure I follow.  What is the alternative that you propose?
> 
> I think Hye-Shik Chang would be happy if just tp_print acts according
> to the locale, and "most" containers support tp_print. Given that
> list, tuple, and dict have tp_print, no further action might be
> needed.

That should not be done.  tp_print only gets invoked when sys.stdout
is a real file; otherwise str() or repr() gets invoked.  This should
be transparent; tp_print should be an optimization only, not a way to
get different semantics.  It has already been mentioned that tp_print
is not invoked when using print from IDLE.

> I'm now convinced that having repr locale-specific is unacceptable. So
> I would back something out. However, I still sympathize with the
> original rationale, and I don't see a problem with tp_print differing
> from tp_repr.

I do, see above.  Feel free to back out the multibyte-string patches.
We have learned something.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Thu Oct 10 19:49:53 2002
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 10 Oct 2002 14:49:53 -0400
Subject: [Python-Dev] resource.setrlimit on OSX [was:Tests failing in MacPython 2.2.2]
References: <15781.42538.323678.806285@montanaro.dyndns.org>
 <BEBEAEAA-DC77-11D6-9E46-003065A7CFBA@mac.com>
Message-ID: <15781.52177.932856.1255@gargle.gargle.HOWL>

>>>>> "SM" == Steven Majewski <sdm7g@mac.com> writes:

    SM> Yes, that fix still works for me and No, I'm not running it as
    SM> root, but I am still running 10.1.5, so if it's not working
    SM> for you, maybe it's a 10.1 vs 10.2 problem.

Note that SourceForge's compile farm has some vintage 10.1 OSX server
available.  I've got 10.2 at home.  I'll try it on both.

-Barry


From skip@pobox.com  Thu Oct 10 20:27:27 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 10 Oct 2002 14:27:27 -0500
Subject: [Python-Dev] Re: resource.setrlimit on OSX [was:Tests failing in MacPython 2.2.2]
In-Reply-To: <BEBEAEAA-DC77-11D6-9E46-003065A7CFBA@mac.com>
References: <15781.42538.323678.806285@montanaro.dyndns.org>
 <BEBEAEAA-DC77-11D6-9E46-003065A7CFBA@mac.com>
Message-ID: <15781.54431.709070.201671@montanaro.dyndns.org>

    Steve> Yes, that fix still works for me and No, I'm not running it as
    Steve> root, but I am still running 10.1.5, so if it's not working for
    Steve> you, maybe it's a 10.1 vs 10.2 problem.

Alas, from what I've seen on the pythonmac-sig, this seems to be a common
theme...

Skip




From richie@entrian.com  Thu Oct 10 21:13:33 2002
From: richie@entrian.com (Richie Hindle)
Date: Thu, 10 Oct 2002 21:13:33 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: <200210091555.g99FtD011873@odiug.zope.com>
References: <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be> <200210091555.g99FtD011873@odiug.zope.com>
Message-ID: <vqmbqu4ams8nle09t7dga612pvi78l526v@4ax.com>

Hi,

[Guido]
> Python should not include features (like writable code object
> attributes) that can cause crashes when used inexpertly.

If I do implement this, I'll do it in such a way as to make crashes
impossible (my first suggestion said that crashes would be possible, but I
realise that was a mistake).  It'll verify that the new value of f_lasti
is a valid bytecode position, and it'll either fix up the stack/blockstack
or refuse to move the position in ways that would affect them.  (Armin, I
guess that won't restrict you because you're doing this from C code?)

[Armin]
> For safety I'd consider writing the
> frame-object-modifying code in a C extension module, carefully
> documented as "don't use this".

[Guido]
> That's a reasonable solution: f_lasti should be read-only from Python
> code, but you can write an extension that can write it.

I'd like to see pure Python debuggers (including pdb) have this feature.
If it's safe to use, might it still be considered for inclusion?  (I can't
guarantee I'll have the time to do it, but I'll try.)

I'm also coming to realise that it's not f_lasti I really want to change,
but f_lineno.  The line number is the 'unit' that debuggers use - a
debugger would set f_lineno, and f_lasti would be calculated and changed
to the appropriate value.  I don't think that changes any of the arguments
either way, except maybe to distance the idea a little from what Armin is
doing.

-- 
Richie Hindle
richie@entrian.com



From guido@python.org  Thu Oct 10 21:32:01 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 16:32:01 -0400
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: Your message of "Thu, 10 Oct 2002 21:13:33 BST."
 <vqmbqu4ams8nle09t7dga612pvi78l526v@4ax.com>
References: <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be> <200210091555.g99FtD011873@odiug.zope.com>
 <vqmbqu4ams8nle09t7dga612pvi78l526v@4ax.com>
Message-ID: <200210102032.g9AKW1K15947@odiug.zope.com>

> [Guido]
> > Python should not include features (like writable code object
> > attributes) that can cause crashes when used inexpertly.
> 
> If I do implement this, I'll do it in such a way as to make crashes
> impossible (my first suggestion said that crashes would be possible,
> but I realise that was a mistake).  It'll verify that the new value
> of f_lasti is a valid bytecode position, and it'll either fix up the
> stack/blockstack or refuse to move the position in ways that would
> affect them.  (Armin, I guess that won't restrict you because you're
> doing this from C code?)

Great.

> [Armin]
> > For safety I'd consider writing the
> > frame-object-modifying code in a C extension module, carefully
> > documented as "don't use this".
> 
> [Guido]
> > That's a reasonable solution: f_lasti should be read-only from
> > Python code, but you can write an extension that can write it.
> 
> I'd like to see pure Python debuggers (including pdb) have this
> feature.  If it's safe to use, might it still be considered for
> inclusion?  (I can't guarantee I'll have the time to do it, but I'll
> try.)

Sure, *if* it's safe.

> I'm also coming to realise that it's not f_lasti I really want to
> change, but f_lineno.  The line number is the 'unit' that debuggers
> use - a debugger would set f_lineno, and f_lasti would be calculated
> and changed to the appropriate value.  I don't think that changes
> any of the arguments either way, except maybe to distance the idea a
> little from what Armin is doing.

Sure.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Thu Oct 10 21:41:51 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 10 Oct 2002 22:41:51 +0200
Subject: [Python-Dev] Re: resource.setrlimit on OSX [was:Tests failing in MacPython 2.2.2]
In-Reply-To: <15781.54431.709070.201671@montanaro.dyndns.org>
Message-ID: <B4CA2ED2-DC90-11D6-9207-003065517236@oratrix.com>

On donderdag, oktober 10, 2002, at 09:27 , Skip Montanaro wrote:

>
>     Steve> Yes, that fix still works for me and No, I'm not 
> running it as
>     Steve> root, but I am still running 10.1.5, so if it's not 
> working for
>     Steve> you, maybe it's a 10.1 vs 10.2 problem.
>
> Alas, from what I've seen on the pythonmac-sig, this seems to 
> be a common
> theme...

Don't worry: the recent flurry of discussion is all about 
MacPython, not about unix-Python. Forward compatibility from 
10.1 to 10.2 for unix programs is pretty good, the only problem 
I've seen (runtime-wise, that is) is with programs using curses.

What *is* bothersome, though, is that you cannot easily create a 
program on 10.2 that is backwards compatible to 10.1. Now this 
is probably true on most Unix platforms, but Apple has all this 
wonderful versioning infrastructure for frameworks, and then 
they don't use it themselves:-(
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From guido@python.org  Thu Oct 10 22:04:53 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 17:04:53 -0400
Subject: [Python-Dev] tp_dictoffset calculation in 2.2.2
Message-ID: <200210102105.g9AL4rP16034@odiug.zope.com>

Hi David and Kevin,

I'd like to hear from you if a particular last-minute change to 2.2.2
would affect your code in any way.  If you don't have time to read
this whole message, can you please check if this patch to Python
2.2.2b1 (in CVS on the release22-maint branch) has any bad effects on
your code?

diff -r2.126.4.24 typeobject.c
2090c2090
< 		COPYSLOT(tp_dictoffset);
---
> 		/*COPYSLOT(tp_dictoffset);*/


Background
----------

In Python 2.2, type objects have a tp_dictoffset which tells us where
the instance dict pointer is in the instance object layout.  This
pointer is calculated in three ways.  In precedence:

  1. (Line 1113) For dynamically created types (i.e. created by class
     statements), if the new type has no __slots__, the dominant base
     class has a zero tp_dictoffset, and the dominant base doesn't
     have a custom tp_getattro, a __dict__ will be added to the
     instance layout, and its tp_dictoffset is calculated by taking
     the end of the base class instance struct (or in a more
     complicated way if tp_itemsize is nonzero).

  2. (Line 1941) If the dominant base has a nonzero tp_dictoffset,
     copy the tp_dictoffset from the dominant base.

  3. (Line 2090) The tp_dictoffset of the first non-dominant base
     class that has a nonzero tp_dictoffset is copied.

Jeremy & I just went through a debugging session of some code he's
trying to add to Zope3 where we found that rule #3 is evil.
Simplified, we had the following situation:

  from some_extension import P
  class C(object): pass
  class D(C, P): pass

Type P in has some unique properties: its tp_dictoffset is zero, and
it has a custom tp_setattro.  This means that neither rule #1 fires
(custom tp_setattro) nor rule #2 (the dominant base has no
tp_dictoffset).  So the tp_dictoffset from class C is simply copied
into class D.  This offset is 12, but at that offset lives some
unrelated field in the P instance layout!

I believe that in cases where rule #3 triggers, it is *always* wrong:
when rule #2 hasn't triggered, the dict offset in the dominant base
class is different from the dict offset in the other base classes, but
the instance layout is inherited only from the dominant base.

But I can't prove this to myself with 100% satisfaction, so I'd like
to hear some test results, before I check this in as part of 2.2.2
final.

(The other COPYSLOT calls in the vicinity of the patch are also a bit
suspect, but much less so, since they are function pointers rather
than object offsets.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Thu Oct 10 22:07:26 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 10 Oct 2002 23:07:26 +0200
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: <15779.29749.886457.639973@gargle.gargle.HOWL>
Message-ID: <47ACA202-DC94-11D6-9207-003065517236@oratrix.com>

On woensdag, oktober 9, 2002, at 02:11 , Barry A. Warsaw wrote:

>     JJ> test_email: assertion failed, 2042 != 2. from line 1793 in
>     JJ> test_crlf_separation.  Could this have something to do with
>     JJ> MacPython 2.2's "poor man's universal newlines"? (MacPython
>     JJ> 2.2.X will read both \r and \n as \n for files opened in text
>     JJ> mode, but not \r\n. This quick hack was put in to make 2.2
>     JJ> usable on OSX).
>
> Re-reading this, are you saying that this failure is on MacOS9 only?

Correct. Or, to be hyper-correct: it fails in MacPython 2.2.X on 
OSX as well,
but not in unix-Python 2.2.X on OSX.

> That test passes just fine on OSX 10.2 and the data looks just fine to
> me, so this must be an OS9 failure only.
>
> Maybe the thing to do is to open msg_26.txt in binary mode, but only
> on MacOS9?

Doesn't work, now the assert just above that fails (the payload 
length is 0 in stead of 2), so I think this breaks the boundary 
separation or somesuch. I've added code to simply skip this test 
for MacPython 2.2.X, how do you feel about that?
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From Jack.Jansen@oratrix.com  Thu Oct 10 22:09:12 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 10 Oct 2002 23:09:12 +0200
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
In-Reply-To: <200210090034.g990YTx26342@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <8667DDD1-DC94-11D6-9207-003065517236@oratrix.com>

On woensdag, oktober 9, 2002, at 02:34 , Guido van Rossum wrote:

>> The first one should be easy to fix: backport the revision 1.7
>> fix. Would that cause any problems on other platforms?
>
> No problemo.

Ok, done.
>> test_httplib: mismatch in lines 9-12 of expected output. It
>> wanted two header: Set-Cookie: lines, but got nothing.
>
> Probably a line ending problem.  I notice that those two lines end
> with \r\n!  I don't understand why the test uses \r\n.  I propose to
> change all the \r\n to \n in test_httplib.py and ditto in
> output/test_httplib.

Ok, done. I haven't checked it on unix yet, I'll do that shortly.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From jacobs@penguin.theopalgroup.com  Thu Oct 10 22:43:34 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Thu, 10 Oct 2002 17:43:34 -0400 (EDT)
Subject: [Python-Dev] tp_dictoffset calculation in 2.2.2
In-Reply-To: <200210102105.g9AL4rP16034@odiug.zope.com>
Message-ID: <Pine.LNX.4.44.0210101740170.8479-100000@penguin.theopalgroup.com>

On Thu, 10 Oct 2002, Guido van Rossum wrote:
> I'd like to hear from you if a particular last-minute change to 2.2.2
> would affect your code in any way.  If you don't have time to read
> this whole message, can you please check if this patch to Python
> 2.2.2b1 (in CVS on the release22-maint branch) has any bad effects on
> your code?
> 
> diff -r2.126.4.24 typeobject.c
> 2090c2090
> < 		COPYSLOT(tp_dictoffset);
> ---
> > 		/*COPYSLOT(tp_dictoffset);*/

This doesn't seem to interact with any of my extension classes, or at least
nothing in our test suite.  I'll look into this more deeply over the next
few days, though I agree that case #3 is almost always certain to do the
wrong thing.

Thanks for the heads-up,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From dave@boost-consulting.com  Thu Oct 10 22:44:41 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 10 Oct 2002 17:44:41 -0400
Subject: [Python-Dev] Re: tp_dictoffset calculation in 2.2.2
In-Reply-To: <200210102105.g9AL4rP16034@odiug.zope.com>
References: <200210102105.g9AL4rP16034@odiug.zope.com>
Message-ID: <u4rbu2cl2.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

> Hi David and Kevin,
> 
> I'd like to hear from you if a particular last-minute change to 2.2.2
> would affect your code in any way.  If you don't have time to read
> this whole message, can you please check if this patch to Python
> 2.2.2b1 (in CVS on the release22-maint branch) has any bad effects on
> your code?

I'll try running my test suite against it now.

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From barry@python.org  Thu Oct 10 22:55:11 2002
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 10 Oct 2002 17:55:11 -0400
Subject: [Python-Dev] Tests failing in MacPython 2.2.2
References: <15779.29749.886457.639973@gargle.gargle.HOWL>
 <47ACA202-DC94-11D6-9207-003065517236@oratrix.com>
Message-ID: <15781.63295.453362.50932@gargle.gargle.HOWL>

>>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:

    >> Re-reading this, are you saying that this failure is on MacOS9
    >> only?

    JJ> Correct. Or, to be hyper-correct: it fails in MacPython 2.2.X
    JJ> on OSX as well, but not in unix-Python 2.2.X on OSX.

Ok, cool.  I'm of course only testing unix-Python on OSX.

    >> That test passes just fine on OSX 10.2 and the data looks just
    >> fine to me, so this must be an OS9 failure only.
    >> Maybe the thing to do is to open msg_26.txt in binary mode, but
    >> only on MacOS9?

    JJ> Doesn't work, now the assert just above that fails (the
    JJ> payload length is 0 in stead of 2), so I think this breaks the
    JJ> boundary separation or somesuch. I've added code to simply
    JJ> skip this test for MacPython 2.2.X, how do you feel about
    JJ> that?

I'm happy with that for Python 2.2.2, since I'm convinced it's just a
problem with the test suite.  When we have more breathing room
(i.e. Python 2.3) we can see if we can come up with something more
useful.

-Barry


From dave@boost-consulting.com  Thu Oct 10 23:08:54 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 10 Oct 2002 18:08:54 -0400
Subject: [Python-Dev] Re: tp_dictoffset calculation in 2.2.2
In-Reply-To: <200210102105.g9AL4rP16034@odiug.zope.com>
References: <200210102105.g9AL4rP16034@odiug.zope.com>
Message-ID: <uk7kq0ww9.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

All of my tests passed. Thanks for asking, though -- we're just about
to release Boost.Python v2 (like, tomorrow!)

-Dave
-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From neal@metaslash.com  Thu Oct 10 23:59:54 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 10 Oct 2002 18:59:54 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
Message-ID: <3DA6066A.F349116F@metaslash.com>

>From SF #607253, header file problems
( http://python.org/sf/607253 )
is there any reason why these files:
    graminit.h
    patchlevel.h 
    pyconfig.h (aka pyconfig.h.in)

don't protect against being included multiple times?

ie, they don't have:

    #ifndef Py_FILE_H
    #define Py_FILE_H

    /* ... */

    #endif /* !Py_FILE_H */

Neal


From ishimoto@gembook.org  Fri Oct 11 00:06:33 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Fri, 11 Oct 2002 08:06:33 +0900
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
References: <20021010114451.7791.ISHIMOTO@gembook.org> <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021011075000.E88A.ISHIMOTO@gembook.org>

Hello,

On Thu, 10 Oct 2002 08:59:26 -0400
Guido van Rossum <guido@python.org> wrote:

> [Guido]
> > > Well, if you *want* to see the hex codes for all non-ASCII characters,
> > > repr() used to be your friend.  No more.  If you *want* to see the
> > > printable characters, you could always use print.
> 
> [Atsuo Ishimoto]
> > I'm happy with this.
> 
> "This" was ambiguous.  Are you happy with what's in current CVS, or
> with the old repr()?

Sorry. I meant I'm happy with current CVS.

> I tried this, and I could not find any problems with the resulting
> pickle.  The pickle looks like this:
> 
> "S'\\x83\\\\'\np0\n."
> 
> I couldn't get this to fail loading in Python 2.1, 2.2 or 2.3 (CVS);
> I tried both pickle and cPickle.

May be you tried with ascii or latin-1 locale. But with Japanese
ShiftJIS encoding, you'll see the traceback on unpickling. 
I tried CVS version of python on Japanese edition of Win2000, I got:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 1071, in loads
    return Unpickler(file).load()
  File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 675, in load
    dispatch[key](self)
  File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 754, in load_string
    self.append(rep.decode("string-escape"))
ValueError: Trailing \ in string

--------------------------
Atsuo Ishimoto
ishimoto@gembook.org
Homepage:http://www.gembook.jp




From guido@python.org  Fri Oct 11 00:58:36 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 19:58:36 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: Your message of "Thu, 10 Oct 2002 18:59:54 EDT."
 <3DA6066A.F349116F@metaslash.com>
References: <3DA6066A.F349116F@metaslash.com>
Message-ID: <200210102358.g9ANwa811524@pcp02138704pcs.reston01.va.comcast.net>

> From SF #607253, header file problems
> ( http://python.org/sf/607253 )
> is there any reason why these files:
>     graminit.h
>     patchlevel.h 
>     pyconfig.h (aka pyconfig.h.in)
> 
> don't protect against being included multiple times?
> 
> ie, they don't have:
> 
>     #ifndef Py_FILE_H
>     #define Py_FILE_H
> 
>     /* ... */
> 
>     #endif /* !Py_FILE_H */

Could've been an oversight.  I notice that graminit.h and pyconfig.h
are generated files.  patchlevel.h was originally a one-liner.

Is there anything that is gained by adding those?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct 11 01:04:05 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 10 Oct 2002 20:04:05 -0400
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: Your message of "Fri, 11 Oct 2002 08:06:33 +0900."
 <20021011075000.E88A.ISHIMOTO@gembook.org>
References: <20021010114451.7791.ISHIMOTO@gembook.org> <200210101259.g9ACxVW07997@pcp02138704pcs.reston01.va.comcast.net>
 <20021011075000.E88A.ISHIMOTO@gembook.org>
Message-ID: <200210110004.g9B045011881@pcp02138704pcs.reston01.va.comcast.net>

> > I tried this, and I could not find any problems with the resulting
> > pickle.  The pickle looks like this:
> > 
> > "S'\\x83\\\\'\np0\n."
> > 
> > I couldn't get this to fail loading in Python 2.1, 2.2 or 2.3 (CVS);
> > I tried both pickle and cPickle.
> 
> May be you tried with ascii or latin-1 locale. But with Japanese
> ShiftJIS encoding, you'll see the traceback on unpickling. 
> 
> I tried CVS version of python on Japanese edition of Win2000, I got:
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 1071, in loads
>     return Unpickler(file).load()
>   File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 675, in load
>     dispatch[key](self)
>   File "C:\cygwin\home\ishimoto\wk\python\dist\src\lib\pickle.py", line 754, in load_string
>     self.append(rep.decode("string-escape"))
> ValueError: Trailing \ in string

Oops.  It looks like this is the nail in the coffin of the
locale-specific repr(): a multibyte "character" may end in a backslash
that is not escaped.

Martin, I'm afraid there's nothing we can do except revert your
multibyte character checkins...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pinard@iro.umontreal.ca  Fri Oct 11 04:28:02 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Thu, 10 Oct 2002 23:28:02 -0400
Subject: [Python-Dev] Re: Multiple Inclusion of Header Files
In-Reply-To: <200210102358.g9ANwa811524@pcp02138704pcs.reston01.va.comcast.net> (Guido
 van Rossum's message of "Thu, 10 Oct 2002 19:58:36 -0400")
References: <3DA6066A.F349116F@metaslash.com>
 <200210102358.g9ANwa811524@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <oqelax64e5.fsf@titan.progiciels-bpi.ca>

[Guido van Rossum]

>> From SF #607253, header file problems
>> ( http://python.org/sf/607253 )
>> is there any reason why these files:
>>     graminit.h
>>     patchlevel.h 
>>     pyconfig.h (aka pyconfig.h.in)
>> 
>> don't protect against being included multiple times?
>> 
>> ie, they don't have:
>> 
>>     #ifndef Py_FILE_H
>>     #define Py_FILE_H
>> 
>>     /* ... */
>> 
>>     #endif /* !Py_FILE_H */
>
> Could've been an oversight.  I notice that graminit.h and pyconfig.h
> are generated files.  patchlevel.h was originally a one-liner.
>
> Is there anything that is gained by adding those?

I never have been comfortable with the trend of recent years about include
files protecting themselves against multiple inclusion, which progressively
replaced the previous trend about including more carefully than blindly.

Granted, the wild variety of organisations for include files in various
operating systems is a problem in itself, writing portably in that almost
intractable mess is undoubtedly a challenge.  Protection against multiple
inclusion has been welcome in that area.

However, this habit of protection is spreading (I might even have given into
it myself) and I sometimes wonder if we do this is by mere mimetism, or for
any good reason.  We are loosing possibly useful clues when programmers
include uselessly, or design poor hierarchies for their include files.

It now goes a bit far.  I was surprised to discover that even Bison adds
such protection to the `.h' files it generates, I do not believe there ever
was a problem.  Python does not exist in such a multiplicity of hierarchies
that one is hopeless to ever include properly.  So, we could ask ourselves
if auto-protection is really meaningful for Python header files...

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From martin@v.loewis.de  Fri Oct 11 06:42:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 11 Oct 2002 07:42:11 +0200
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <3DA6066A.F349116F@metaslash.com>
References: <3DA6066A.F349116F@metaslash.com>
Message-ID: <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> is there any reason why these files:
>     graminit.h
>     patchlevel.h 
>     pyconfig.h (aka pyconfig.h.in)
> 
> don't protect against being included multiple times?

I think they cannot possibly be included multiple times in a correct
application, since a correct application does not directly include
them, anyway.

Regards,
Martin



From tismer@tismer.com  Fri Oct 11 06:46:04 2002
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 10 Oct 2002 22:46:04 -0700
Subject: [Python-Dev] Multiple Inclusion of Header Files
References: <3DA6066A.F349116F@metaslash.com> <200210102358.g9ANwa811524@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DA6659C.7040008@tismer.com>

Guido van Rossum wrote:
...

> Could've been an oversight.  I notice that graminit.h and pyconfig.h
> are generated files.  patchlevel.h was originally a one-liner.
> 
> Is there anything that is gained by adding those?

Don't think so, since these are included py Python.h
once, and that is protected.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From ishimoto@gembook.org  Fri Oct 11 07:17:11 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Fri, 11 Oct 2002 15:17:11 +0900
Subject: [Python-Dev] Re: Multibyte repr()
In-Reply-To: <200210110004.g9B045011881@pcp02138704pcs.reston01.va.comcast.net>
References: <20021011075000.E88A.ISHIMOTO@gembook.org> <200210110004.g9B045011881@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021011135552.474D.ISHIMOTO@gembook.org>

Hello,

On Thu, 10 Oct 2002 20:04:05 -0400
Guido van Rossum <guido@python.org> wrote:

> Oops.  It looks like this is the nail in the coffin of the
> locale-specific repr(): a multibyte "character" may end in a backslash
> that is not escaped.
> 
> Martin, I'm afraid there's nothing we can do except revert your
> multibyte character checkins...
> 

I with to multibyte character checkins remain. Hex coded strings are
realy nasty for debugging.

As for illegal-pickle issue, it is easy to fix this problem. We just
need to provide a simple function that converts string to canonical
Python literal representaion, as old repr() does. 

Regards,
--------------------------
Atsuo Ishimoto
ishimoto@gembook.org
Homepage:http://www.gembook.jp



From Paul.Moore@atosorigin.com  Fri Oct 11 10:38:29 2002
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Fri, 11 Oct 2002 10:38:29 +0100
Subject: [Python-Dev] Why does the Windows installer use the shortname form of the install directory?
Message-ID: <16E1010E4581B049ABC51D4975CEDB8861990E@UKDCX001.uk.int.atosorigin.com>

I just installed Python 2.2.2b1 (replacing ActiveState Python) and I =
notice that the installer uses the shortname form of the Python install =
directory. This appears in sys.path, in file associations (and hence in =
sys.argv[0]), and in lots of other places in the registry, not all of =
which I follow the reasons for.

My Python install directory was C:\Applications\Python - not the =
default, but also not containing spaces (which I guess could be a =
problem for some scripts). So the short form should not be necessary. =
I've fixed the problem manually by hacking the registry, but it's not =
something I feel comfortable with...

Would it be possible for the installer to use the long filename form (at =
least in cases where the install path does not contain spaces)?

Sorry I didn't report this earlier, but I've been using the ActiveState =
distribution until now, which doesn't have this problem.

Paul.


From mwh@python.net  Fri Oct 11 10:44:45 2002
From: mwh@python.net (Michael Hudson)
Date: 11 Oct 2002 10:44:45 +0100
Subject: [Python-Dev] Set-next-statement in Python debuggers
In-Reply-To: Richie Hindle's message of "Thu, 10 Oct 2002 21:13:33 +0100"
References: <Pine.OSF.3.96.1021009144614.1601398B-100000@aster3.ulb.ac.be> <200210091555.g99FtD011873@odiug.zope.com> <vqmbqu4ams8nle09t7dga612pvi78l526v@4ax.com>
Message-ID: <2madllxqb6.fsf@starship.python.net>

Richie Hindle <richie@entrian.com> writes:

> Hi,
> 
> [Guido]
> > Python should not include features (like writable code object
> > attributes) that can cause crashes when used inexpertly.
> 
> If I do implement this, I'll do it in such a way as to make crashes
> impossible (my first suggestion said that crashes would be possible, but I
> realise that was a mistake).  It'll verify that the new value of f_lasti
> is a valid bytecode position, and it'll either fix up the stack/blockstack
> or refuse to move the position in ways that would affect them.  (Armin, I
> guess that won't restrict you because you're doing this from C code?)

You can still cause crashes though: move f_lasti to point to a
STORE_NAME (say) without first executing the corresponding LOAD_CONST
(say).

Or just execute a LOAD_CONST over and over again until you fall off
the end of the stack.

> [Armin]
> > For safety I'd consider writing the
> > frame-object-modifying code in a C extension module, carefully
> > documented as "don't use this".
> 
> [Guido]
> > That's a reasonable solution: f_lasti should be read-only from Python
> > code, but you can write an extension that can write it.
> 
> I'd like to see pure Python debuggers (including pdb) have this feature.
> If it's safe to use, might it still be considered for inclusion?  (I can't
> guarantee I'll have the time to do it, but I'll try.)
> 
> I'm also coming to realise that it's not f_lasti I really want to change,
> but f_lineno.  The line number is the 'unit' that debuggers use - a
> debugger would set f_lineno, and f_lasti would be calculated and changed
> to the appropriate value.  I don't think that changes any of the arguments
> either way, except maybe to distance the idea a little from what Armin is
> doing.

This is a good point!  The scenarios I sketched above would be rather
harder (impossible?) to contrive in this situation.

Cheers,
M.

-- 
112. Computer Science is embarrassed by the computer.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From mwh@python.net  Fri Oct 11 10:57:29 2002
From: mwh@python.net (Michael Hudson)
Date: 11 Oct 2002 10:57:29 +0100
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects stringobject.c,2.193,2.194 unicodeobject.c,2.171,2.172
In-Reply-To: gvanrossum@users.sourceforge.net's message of "Thu, 10 Oct 2002 17:43:50 -0700"
References: <E17znuA-0005hK-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2m7kgpxppy.fsf@starship.python.net>

gvanrossum@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Objects
> In directory usw-pr-cvs1:/tmp/cvs-serv19831
> 
> Modified Files:
> 	stringobject.c unicodeobject.c 
> Log Message:
> Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
> '%2147483647d' % -123 segfaults.  This was because an integer overflow
> in a comparison caused the string resize to be skipped.  After fixing
> the overflow, this could call _PyString_Resize() with a negative size,
> so I (1) test for that and raise MemoryError instead; (2) also added a
> test for negative newsize to _PyString_Resize(), raising SystemError
> as for all bad arguments.
> 
> An identical bug existed in unicodeobject.c, of course.
> 
> Will backport to 2.2.2.

No test case? For shame!

I'll do one if you don't beat me to it.

Cheers,
M.

-- 
  And then the character-only displays went away (leading to
  increasingly silly graphical effects and finally to ads on
  web pages).                      -- John W. Baxter, comp.lang.python


From guido@python.org  Fri Oct 11 12:48:52 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 11 Oct 2002 07:48:52 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: Your message of "Fri, 11 Oct 2002 07:42:11 +0200."
 <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>
References: <3DA6066A.F349116F@metaslash.com>
 <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>

> > is there any reason why these files:
> >     graminit.h
> >     patchlevel.h 
> >     pyconfig.h (aka pyconfig.h.in)
> > 
> > don't protect against being included multiple times?
> 
> I think they cannot possibly be included multiple times in a correct
> application, since a correct application does not directly include
> them, anyway.

But that's true of any .h file that's included by Python.h, isn't it?
And there's only a handful of .h files that Python.h doesn't include...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Oct 11 14:09:16 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 11 Oct 2002 09:09:16 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>
References: <3DA6066A.F349116F@metaslash.com> <m3wuopmszw.fsf@mira.informatik.hu-berlin.de> <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021011130916.GL30812@epoch.metaslash.com>

On Fri, Oct 11, 2002 at 07:48:52AM -0400, Guido van Rossum wrote:
> > > is there any reason why these files:
> > >     graminit.h
> > >     patchlevel.h 
> > >     pyconfig.h (aka pyconfig.h.in)
> > > 
> > > don't protect against being included multiple times?
> > 
> > I think they cannot possibly be included multiple times in a correct
> > application, since a correct application does not directly include
> > them, anyway.
> 
> But that's true of any .h file that's included by Python.h, isn't it?
> And there's only a handful of .h files that Python.h doesn't include...

I don't really think there is much to be gained by adding protection.
It's unlikely the files would be included directly by an application.
Even if the files are included multiple times, the worst that should
happen is a redefined warning.

Neal


From dave@boost-consulting.com  Fri Oct 11 14:26:42 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 11 Oct 2002 09:26:42 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <20021011130916.GL30812@epoch.metaslash.com>
References: <3DA6066A.F349116F@metaslash.com>
 <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>
 <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>
 <20021011130916.GL30812@epoch.metaslash.com>
Message-ID: <uptuhcdil.fsf@boost-consulting.com>

Neal Norwitz <neal@metaslash.com> writes:

> On Fri, Oct 11, 2002 at 07:48:52AM -0400, Guido van Rossum wrote:
> > > > is there any reason why these files:
> > > >     graminit.h
> > > >     patchlevel.h 
> > > >     pyconfig.h (aka pyconfig.h.in)
> > > > 
> > > > don't protect against being included multiple times?
> > > 
> > > I think they cannot possibly be included multiple times in a correct
> > > application, since a correct application does not directly include
> > > them, anyway.
> > 
> > But that's true of any .h file that's included by Python.h, isn't it?
> > And there's only a handful of .h files that Python.h doesn't include...
> 
> I don't really think there is much to be gained by adding protection.
> It's unlikely the files would be included directly by an application.
> Even if the files are included multiple times, the worst that should
> happen is a redefined warning.

Ahem. Don't forget that C++ applications may need to read these
files. The rules about redefinition are different in C++ IIRC.

So, if these files /really/ don't need to be #included directy by an
application, that's fine...

However, I note that Boost.Python has to #include <patchlevel.h> because
we have to apply workarounds before #including <Python.h> for certain
versions of Python, so I wouldn't say that it's very unrealistic to think
applications might need these files separately.

-- 
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From guido@python.org  Fri Oct 11 14:40:53 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 11 Oct 2002 09:40:53 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: Your message of "Fri, 11 Oct 2002 09:26:42 EDT."
 <uptuhcdil.fsf@boost-consulting.com>
References: <3DA6066A.F349116F@metaslash.com> <m3wuopmszw.fsf@mira.informatik.hu-berlin.de> <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net> <20021011130916.GL30812@epoch.metaslash.com>
 <uptuhcdil.fsf@boost-consulting.com>
Message-ID: <200210111340.g9BDerK23702@pcp02138704pcs.reston01.va.comcast.net>

> However, I note that Boost.Python has to #include <patchlevel.h> because
> we have to apply workarounds before #including <Python.h> for certain
> versions of Python, so I wouldn't say that it's very unrealistic to think
> applications might need these files separately.

But this works fine without the protection; probably because
patchlevel.h contains no C code, only preprocessor stuff.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dave@boost-consulting.com  Fri Oct 11 14:54:49 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 11 Oct 2002 09:54:49 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <200210111340.g9BDerK23702@pcp02138704pcs.reston01.va.comcast.net>
References: <3DA6066A.F349116F@metaslash.com>
 <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>
 <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>
 <20021011130916.GL30812@epoch.metaslash.com>
 <uptuhcdil.fsf@boost-consulting.com>
 <200210111340.g9BDerK23702@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <uheftcc7q.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

> > However, I note that Boost.Python has to #include <patchlevel.h> because
> > we have to apply workarounds before #including <Python.h> for certain
> > versions of Python, so I wouldn't say that it's very unrealistic to thi=
nk
> > applications might need these files separately.
>=20
> But this works fine without the protection; probably because
> patchlevel.h contains no C code, only preprocessor stuff.

Certainly (though PP stuff can cause problems too). I'm just saying
that I if you intentionally avoid #include guards on the basis that
it's going to help you find your own bugs/inefficiencies, as Fran=E7ois
seems to be suggesting, you may be unneccessarily limiting
interoperability, which I think is Python's higher goal.

--=20
           David Abrahams * Boost Consulting
dave@boost-consulting.com * http://www.boost-consulting.com



From tim.one@comcast.net  Fri Oct 11 16:13:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 11 Oct 2002 11:13:38 -0400
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <uheftcc7q.fsf@boost-consulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEOKBKAB.tim.one@comcast.net>

Guards in .h files don't hurt, and can help.  They're trivial to put in or
take out.  I vote we ignore "the issue".

leading-by-example<wink>-ly y'rs  - tim



From tim.one@comcast.net  Fri Oct 11 16:47:30 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 11 Oct 2002 11:47:30 -0400
Subject: [Python-Dev] Why does the Windows installer use the shortname form
 of the install directory?
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8861990E@UKDCX001.uk.int.atosorigin.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOMBKAB.tim.one@comcast.net>

[Moore, Paul]
> I just installed Python 2.2.2b1 (replacing ActiveState Python)
> and I notice that the installer uses the shortname form of the
> Python install directory. This appears in sys.path, in file
> associations (and hence in sys.argv[0]), and in lots of other
> places in the registry, not all of which I follow the reasons for.
>
> My Python install directory was C:\Applications\Python - not the
> default, but also not containing spaces (which I guess could be a
> problem for some scripts). So the short form should not be
> necessary. I've fixed the problem manually by hacking the
> registry, but it's not something I feel comfortable with...

I must have missed what "the problem" is.  What is the problem?  Path
components generally have two distinct names on Windows.  That's how the OS
works -- we didn't invent this madness <wink>.

> Would it be possible for the installer to use the long filename
> form

Yes, but I won't make that change, because we *used* to do it that way and
it confused newbies endlessly, mostly because of embedded spaces.  By
default, the installer suggests an 8.3 name so that nobody can get confused.
Pick a non-8.3 name instead, and two distinct names are a fact of Windows
Life you have to live with.

> (at least in cases where the install path does not contain spaces)?

I don't know that the WISE scripting language is capable of making this
distinction, and in the absence of an actual problem I'm not going to spend
time digging into it.  It's certainly too late to make this kind of change
in the 2.2 line, so if a volunteer appears who cares enough to address it,
it would have to wait for 2.3.



From Paul.Moore@atosorigin.com  Fri Oct 11 16:59:00 2002
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Fri, 11 Oct 2002 16:59:00 +0100
Subject: [Python-Dev] Why does the Windows installer use the shortname formof the install directory?
Message-ID: <16E1010E4581B049ABC51D4975CEDB885E2D0C@UKDCX001.uk.int.atosorigin.com>

From: Tim Peters [mailto:tim.one@comcast.net]

> [Moore, Paul]
> > I just installed Python 2.2.2b1 (replacing ActiveState Python)
> > and I notice that the installer uses the shortname form of the
> > Python install directory. This appears in sys.path, in file
> > associations (and hence in sys.argv[0]), and in lots of other
> > places in the registry, not all of which I follow the reasons for.
> >
> > My Python install directory was C:\Applications\Python - not the
> > default, but also not containing spaces (which I guess could be a
> > problem for some scripts). So the short form should not be
> > necessary. I've fixed the problem manually by hacking the
> > registry, but it's not something I feel comfortable with...
>=20
> I must have missed what "the problem" is.  What is the problem?  Path
> components generally have two distinct names on Windows.  That's how
> the OS works -- we didn't invent this madness <wink>.

"problem" is probably a bit strong. It is user-visible in a few places =
(notably sys.path) and is arguably difficult to read in those places. =
But it's an issue of aesthetics rather than anything else.

OK, so if I have a sense of aesthetics, I probably shouldn't be using =
Windows :-)

Agreed, it's mainly an OS issue. If Windows didn't do the long/short =
pathname idiocy, and if the practice of using pathnames with spaces had =
never been promoted, we wouldn't be having this discussion. (Or if Unix =
people had got into the habit of actually using the freedom thay have to =
use pathnames containing all sorts of strange characters, we may be even =
worse off :-))

>=20
> > Would it be possible for the installer to use the long filename
> > form
>=20
> Yes, but I won't make that change, because we *used* to do it that way =
and
> it confused newbies endlessly, mostly because of embedded spaces.  By
> default, the installer suggests an 8.3 name so that nobody can get =
confused.
> Pick a non-8.3 name instead, and two distinct names are a fact of =
Windows
> Life you have to live with.

OK. If the other way has been shown to cause real problems, then that's =
a reason not to do that. Having used ActiveState's distribution until =
now (which uses the long form, and advises you not to use paths with =
spaces - I've no idea how it copes if you ignore its advice, though...) =
I didn't realise that you'd been through this.

> > (at least in cases where the install path does not contain spaces)?
>=20
> I don't know that the WISE scripting language is capable of making =
this
> distinction, and in the absence of an actual problem I'm not going to =
spend
> time digging into it.  It's certainly too late to make this kind of =
change
> in the 2.2 line, so if a volunteer appears who cares enough to address =
it,
> it would have to wait for 2.3.

That's fair enough. And it's not major enough that I'm pushing for =
anyone to waste time on it. I just thought I'd mention it in case it was =
a simple change to a checkbox, or something. I'm always optimistic on a =
Friday - I save the cynicism for Mondays... :-)

Thanks for taking the time to comment.

Paul.


From tim.one@comcast.net  Fri Oct 11 17:31:36 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 11 Oct 2002 12:31:36 -0400
Subject: [Python-Dev] Why does the Windows installer use the shortname
 formof the install directory?
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB885E2D0C@UKDCX001.uk.int.atosorigin.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net>

[Moore, Paul]
> ...
> That's fair enough. And it's not major enough that I'm pushing
> for anyone to waste time on it. I just thought I'd mention it in
> case it was a simple change to a checkbox, or something.

It *almost* is a checkbox!  In WISE's idea of "a scripting language", the
relevant line looks like this:

   Set Variable MAINDIR to %MAINDIR%

There's not a visible clue that this isn't a no-op.  If you double-click on
the line in the WISE GUI, though, a dialog box pops up, with a dropdown list
labeled "Operation", one of whose entries is

    Convert to short filename

There isn't another variant that adds "but only if it contains a space".

Beyond that, I'm stuck -- since Adobe graciously updated my Acrobat Reader,
every attempt to view the WISE reference docs (pdf format) hard-freezes
(power cycle required) my box.  I *like* your suggestion to skip this unless
there's an embedded space, I just seem unable to do anything about it now
(but it would still be for 2.3 -- mucking with 2.2 is too dangerous).



From thomas.heller@ion-tof.com  Fri Oct 11 17:47:59 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 11 Oct 2002 18:47:59 +0200
Subject: [Python-Dev] Why does the Windows installer use the shortname formof the install directory?
References: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net>
Message-ID: <000901c27145$f51e9720$e000a8c0@thomasnotebook>

> [Moore, Paul]
> > ...
> > That's fair enough. And it's not major enough that I'm pushing
> > for anyone to waste time on it. I just thought I'd mention it in
> > case it was a simple change to a checkbox, or something.
[Tim]
> 
> It *almost* is a checkbox!  In WISE's idea of "a scripting language", the
> relevant line looks like this:
> 
>    Set Variable MAINDIR to %MAINDIR%
> 
> There's not a visible clue that this isn't a no-op.  If you double-click on
> the line in the WISE GUI, though, a dialog box pops up, with a dropdown list
> labeled "Operation", one of whose entries is
> 
>     Convert to short filename
> 
> There isn't another variant that adds "but only if it contains a space".
> 
> Beyond that, I'm stuck -- since Adobe graciously updated my Acrobat Reader,
> every attempt to view the WISE reference docs (pdf format) hard-freezes
> (power cycle required) my box.  I *like* your suggestion to skip this unless
> there's an embedded space, I just seem unable to do anything about it now
> (but it would still be for 2.3 -- mucking with 2.2 is too dangerous).
> 
I did a quick check: the online pdf file does say exactly the same
as the online help (which is still in windows help format).
It pops up when youu press F1 in this dialog.
If even this does not work on win 98 <wink>, I can post
the text here.

Thomas



From tim.one@comcast.net  Fri Oct 11 18:26:08 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 11 Oct 2002 13:26:08 -0400
Subject: [Python-Dev] Why does the Windows installer use the shortname
 formof the install directory?
In-Reply-To: <000901c27145$f51e9720$e000a8c0@thomasnotebook>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPHBKAB.tim.one@comcast.net>

[Tim]
> ...
> I *like* your suggestion to skip this unless there's an embedded
> space, I just seem unable to do anything about it now (but it would
> still be for 2.3 -- mucking with 2.2 is too dangerous).

[Thomas Heller]
> I did a quick check: the online pdf file does say exactly the same
> as the online help (which is still in windows help format).
> It pops up when youu press F1 in this dialog.  If even this does not
> work on win 98 <wink>, I can post the text here.

Better, if this interests you, how about you submit a (tested) patch that
skips the convert-to-short business unless there's an embedded space?  This
isn't going to become a priority for me -- it's a "doctor! doctor!" thing.
That is, the installer suggests a path that isn't going to confuse anyone,
no way, no how.  When someone insists on making life more difficult, my
capacity for empathic outrage is limited <wink>; top-level directories
haven't been a limited critical resource since FAT faded away.



From thomas.heller@ion-tof.com  Fri Oct 11 18:31:59 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 11 Oct 2002 19:31:59 +0200
Subject: [Python-Dev] Why does the Windows installer use the shortname formof the install directory?
References: <LNBBLJKPBEHFEDALKOLCEEPHBKAB.tim.one@comcast.net>
Message-ID: <001701c2714c$1aa661c0$e000a8c0@thomasnotebook>

> [Tim]
> > ...
> > I *like* your suggestion to skip this unless there's an embedded
> > space, I just seem unable to do anything about it now (but it would
> > still be for 2.3 -- mucking with 2.2 is too dangerous).
> 
> [Thomas Heller]
> > I did a quick check: the online pdf file does say exactly the same
> > as the online help (which is still in windows help format).
> > It pops up when youu press F1 in this dialog.  If even this does not
> > work on win 98 <wink>, I can post the text here.
> 
> Better, if this interests you, how about you submit a (tested) patch that
> skips the convert-to-short business unless there's an embedded space?  This
> isn't going to become a priority for me -- it's a "doctor! doctor!" thing.
> That is, the installer suggests a path that isn't going to confuse anyone,
> no way, no how.  When someone insists on making life more difficult, my
> capacity for empathic outrage is limited <wink>; top-level directories
> haven't been a limited critical resource since FAT faded away.
> 
No, it doesn't interest me. I'm not going to install python (which
I often use from the command line) ever into a deep subpath, or one
containing spaces. Even my python 1.5.2 lives in c:\Python1.5.

I just wanted to remind you on the online (F1) help, which is
real good IMO in wise.

Thomas




From martin@v.loewis.de  Fri Oct 11 18:35:52 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 11 Oct 2002 19:35:52 +0200
Subject: [Python-Dev] Multiple Inclusion of Header Files
In-Reply-To: <uptuhcdil.fsf@boost-consulting.com>
References: <3DA6066A.F349116F@metaslash.com>
 <m3wuopmszw.fsf@mira.informatik.hu-berlin.de>
 <200210111148.g9BBmqu22885@pcp02138704pcs.reston01.va.comcast.net>
 <20021011130916.GL30812@epoch.metaslash.com>
 <uptuhcdil.fsf@boost-consulting.com>
Message-ID: <m365w87u9z.fsf@mira.informatik.hu-berlin.de>

David Abrahams <dave@boost-consulting.com> writes:

> However, I note that Boost.Python has to #include <patchlevel.h> because
> we have to apply workarounds before #including <Python.h> for certain
> versions of Python, so I wouldn't say that it's very unrealistic to think
> applications might need these files separately.

If people do such things, I think we'ld like to hear about that
(atleast if it causes problems). Most likely, people are misguided
when they include other Python headers but Python.h (or include
Python.h after including system headers); so if there is a problem, it
is likely fixed by some other means.

If we commit to supporting things we don't know, we might get into
troubles some day, having either to break those applications some day,
or having to maintain ugly code just because we have to continue to
support applications should not have used the feature in the first
place.

Regards,
Martin


From kajiyama@grad.sccs.chukyo-u.ac.jp  Sat Oct 12 08:24:19 2002
From: kajiyama@grad.sccs.chukyo-u.ac.jp (Tamito KAJIYAMA)
Date: Sat, 12 Oct 2002 16:24:19 +0900
Subject: [Python-Dev] Re: Multibyte repr()
Message-ID: <200210120724.g9C7OJQ29295@grad.sccs.chukyo-u.ac.jp>

Hi,

I know it's too late to post this, but I'd like to express my
opinion on locale-dependent repr() from a CJKV user's viewpoint.

Guido <guido@python.org> writes:
|
| I don't think this particular issue (pickling) is a problem.  But I
| *do* continue to worry that making repr() depend on the locale may be
| a bigger problem than what it attempts to solve.

I think locale-dependent repr() is a bad idea.

The most important (IMHO) aspect of repr() is that returned
values of repr() can be used as fragments of Python programs.
If repr() were locale-dependent, the returned values could not
be used as fragments in some cases.  In Japanese applications,
Shift_JIS string literals would be a major problem in this
sense.

I think one of the virtues of Python is that Python has no
language feature that is (automagically) affected by locale
settings.  Applications with locale-specific features would be
useful, but I wish Python be as locale-independent as possible.

Regards,

-- 
KAJIYAMA, Tamito <kajiyama@grad.sccs.chukyo-u.ac.jp>


From tismer@tismer.com  Sat Oct 12 12:32:30 2002
From: tismer@tismer.com (Christian Tismer)
Date: Sat, 12 Oct 2002 04:32:30 -0700
Subject: [Python-Dev] Why does the Windows installer use the shortname
 formof the install directory?
References: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net>
Message-ID: <3DA8084E.3010509@tismer.com>

Tim Peters wrote:
> [Moore, Paul]
> 
>>...
>>That's fair enough. And it's not major enough that I'm pushing
>>for anyone to waste time on it. I just thought I'd mention it in
>>case it was a simple change to a checkbox, or something.
> 
> 
> It *almost* is a checkbox!  In WISE's idea of "a scripting language", the
> relevant line looks like this:
> 
>    Set Variable MAINDIR to %MAINDIR%
> 
> There's not a visible clue that this isn't a no-op.  If you double-click on
> the line in the WISE GUI, though, a dialog box pops up, with a dropdown list
> labeled "Operation", one of whose entries is
> 
>     Convert to short filename
> 
> There isn't another variant that adds "but only if it contains a space".

The F1 key gives you a regular windows help file, as I
remember.
And you can surround the
    Convert to short filename
thing with an if contruct that checks for a space, something like
    If MAINDIR Contains " " then
       # here the Convert thingie
    End

ciao - chris

p.s.: The whol script is in ASCII. I used to generate scripts
from WISE templates which I've written once.
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From arigo@ulb.ac.be  Sat Oct 12 18:45:02 2002
From: arigo@ulb.ac.be (Armin Rigo)
Date: Sat, 12 Oct 2002 19:45:02 +0200
Subject: [Python-Dev] Memory size overflows
Message-ID: <3DA85F9E.339DAC1F@ulb.ac.be>

Hello everybody,

All around the C code there are potential problems with objects of very
large sizes (http://www.python.org/sf/618623).  The problem is that to
allocate a variable-sized object of type 't' with 'n' elements we
compute 'n*t->tp_itemsize', which can overflow even if 'n' is a
perfectly legal value.  If the truncated result is small, the subsequent
malloc() suceeds, and we run into a segfault by trying to access more
memory than reserved.  The same problem exists at other places -- more
or less everywhere we add or multiply something to a number that could
be user-supplied.  For example, Guido just fixed '%2147483647d'%-123.  A
rather artificial example, I agree, but a hole anyway.

To fix this I suggest introducing a few new macros in pymem.h that
compute things about sizes with overflow checking.  I can see a couple
of approaches based on special values that mean "overflow":

1) there is just one special "overflow" value, e.g.
((size_t)-1), that is returned and propagated by the macros
when an overflow is detected.  This might be error-prone
because if we forget once to use the macros to add a few
bytes to the size, this special value will wrap down to a
small legal value -- and segfault.

2) same as above, but with a whole range of overflow
values.  For example, just assume (or decide) that no malloc
of more than half the maximum number that fits into a size_t
can succeed.  We don't need any macro to add a (resonable)
constant to a size.  We need a macro for multiplication that
-- upon overflow -- returns the first number of the "overflow"
range.  The Add macro is still needed to sum *two* potentially
large numbers.

3) we compute all sizes with signed integers (int or long),
as is currently (erroneously?) done at many places.  Any
negative integer is regarded as "overflow", but the
multiplication macro returns the largest negative integer in
case of overflow, so that as above no addition macro is
needed for the simple cases.

This will require a "multiplication hunt party" :-)

Also, approaches 2 and 3 require fixes to ensure that
'malloc(any-overflow-size)' always fail, for any of the several
implementations of malloc found in the code.  Even with
approach 1, I would not trust the platform malloc to correctly
fail on malloc(-1) -- I guess it might "round up" the value to
0 before it proceed...


Armin



From tim.one@comcast.net  Sun Oct 13 01:06:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 12 Oct 2002 20:06:37 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <3DA85F9E.339DAC1F@ulb.ac.be>
Message-ID: <LNBBLJKPBEHFEDALKOLCEECPBLAB.tim.one@comcast.net>

[Armin Rigo]
> All around the C code there are potential problems with objects of very
> large sizes (http://www.python.org/sf/618623).  The problem is that to
> allocate a variable-sized object of type 't' with 'n' elements we
> compute 'n*t->tp_itemsize', which can overflow even if 'n' is a
> perfectly legal value.

I confess I always ignore these until one pops up in real life.  Checking
slows the code, and that causes me pain <0.5 wink>.

Note that the multiplication isn't the only problem:  we usually go on to
add in the size of the object header, and that can overflow too.  This leads
to heartbreaking code such as string_repeat's:

	nbytes = size * sizeof(char);
	if (nbytes / sizeof(char) != (size_t)size ||
	    nbytes + sizeof(PyStringObject) <= nbytes) {
		PyErr_SetString(PyExc_OverflowError,
			"repeated string is too long");
		return NULL;
	}

Making it worse, while malloc takes an unsigned arg, Python usually uses a
signed type to hold these things.  When I fixed the code above (it did pop
up in real life there), I made sure nbytes was declared size_t, so that the
compiler could do the division-by-constant with a simple shift (well, OK,
sizeof(char) is necessarily 1U, so the division is pointless there anyway;
in other contexts it's not, and signed/power_of_2 is more expensive than
unsigned/power_of_2 because most C compilers don't allow the former to
compute the natural floor; note that anything/non_constant can be incredibly
expensive!).

> If the truncated result is small, the subsequent malloc() suceeds, and
> we run into a segfault by trying to access more memory than reserved.
> The same problem exists at other places -- more or less everywhere we
> add or multiply something to a number that could be user-supplied.

Yup -- and that, in a nutshell, is why I usually ignore these <wink>.

Note that it's not enough even to ensure malloc gets a sensible argument:
on at least two flavors of Unix, malloc() returning a non-NULL pointer
doesn't guarantee you can access the memory without segfaulting.  In the
end, it's a game that can't be won.  It may be nicer to do a little better,
but there are real costs too.

> ...
> To fix this I suggest introducing a few new macros in pymem.h that
> compute things about sizes with overflow checking.  I can see a couple
> of approaches based on special values that mean "overflow":

If we're going to this, I suggest an ubiquitous trick from Zope's C code:

    DO_SOMETHING_OR(RESULT_LVALUE, INPUT1, ..., ON_ERROR_BLOCK);

That is, the result goes into RESULT_LVALUE, unless there's an overflow; in
the latter case, ON_ERROR_BLOCK is executed.  This is usually something like
"return NULL;" or "goto Overflow;".  This has the nice property of pushing
the error-case code out of the normal flow.  A goto is especially nice,
because the label provides a handy place to set a debugger breakpoint
(always a practical problem when code hides in macro expansions).

> ...
> 3) we compute all sizes with signed integers (int or long),
> as is currently (erroneously?) done at many places.  Any
> negative integer is regarded as "overflow", but the
> multiplication macro returns the largest negative integer in
> case of overflow, so that as above no addition macro is
> needed for the simple cases.

There's really never been anything special about "the sign bit", and some
32-bit boxes can allocate 1UL<<31 bytes.  This becomes more of a problem as
Python gets applied to larger problems, and 2 gigabytes is no longer an
insane amount of address space.

> ...
> Also, approaches 2 and 3 require fixes to ensure that
> 'malloc(any-overflow-size)' always fail, for any of the several
> implementations of malloc found in the code.

pymalloc is already suitably paranoid, even to the extent that the debug
pymalloc makes sure adding 16 byes for its padding doesn't overflow.

> Even with approach 1, I would not trust the platform malloc to
> correctly fail on malloc(-1) -- I guess it might "round up" the
> value to 0 before it proceed...

malloc() can't get an argument of -1:  it takes a size_t argument, and
size_t is guaranteed to be unsigned.  What *does* happen is that some
platform mallocs don't do what the debug pymalloc does, meaning that they
don't check that adding in their own overhead doesn't overflow.  So, e.g.,
if they need a 4-byte header for bookkeeping, they'll add 4 to the argument
and not even notice if that "wraps around" to 0, 1, 2, or 3.  Disaster
ensues.  But it's not really Python's job to fix broken platform libraries.

lukewarmly y'rs  - tim



From tismer@tismer.com  Sun Oct 13 06:37:43 2002
From: tismer@tismer.com (Christian Tismer)
Date: Sat, 12 Oct 2002 22:37:43 -0700
Subject: [Python-Dev] Memory size overflows
References: <LNBBLJKPBEHFEDALKOLCEECPBLAB.tim.one@comcast.net>
Message-ID: <3DA906A7.3030408@tismer.com>

Tim Peters wrote:
> [Armin Rigo]
> 
>>All around the C code there are potential problems with objects of very
>>large sizes (http://www.python.org/sf/618623).  The problem is that to
>>allocate a variable-sized object of type 't' with 'n' elements we
>>compute 'n*t->tp_itemsize', which can overflow even if 'n' is a
>>perfectly legal value.
> 
> 
> I confess I always ignore these until one pops up in real life.  Checking
> slows the code, and that causes me pain <0.5 wink>.
> 
> Note that the multiplication isn't the only problem:  we usually go on to
> add in the size of the object header, and that can overflow too.  This leads
> to heartbreaking code such as string_repeat's:

(sigh)

[snipped all the rest of good stuff]

May I throw in a "weird" idea, again?

I'm thinking since quite some time that Python should
really abandon 32 bit arithmetic once and forwver and
switch to use 64 bits, everywhere.
It would not be the first scripting language which
does so (wasn't it Ruby?), and it would benefit from that
quite much, I think.

I believe that 32 bit technology is outworn like 16 bits
have before. When? Well, I'd say 16 years ago, exactly,
if we assume an increase of memory size of 1 bit per year,
which seems realistic.

There are already a couple of 64 bit machines running Python.
These would not see any slow-down by switching to 64 bits.
New upcoming processors a likely to be 64 bitters pretty soon,
and I hope to see such a nice machine like Knuth's MMIX
in the next five years or earlier.
Most probably, 64 bits will then be state of the art,
and by switching to it right now, we are solving a lot
of problems for the next 32 years ahead.
This is beyond the careers of most of the "old" guys
around, giving us a chance to solve some problem
"once and forever".
The current small speed loss will vanish from alone :)

It-might-sound-funny-but-it-*is*-my-opinion - ly y'rs chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From drifty@bigfoot.com  Sun Oct 13 09:03:08 2002
From: drifty@bigfoot.com (Brett Cannon)
Date: Sun, 13 Oct 2002 01:03:08 -0700 (PDT)
Subject: [Python-Dev] Python-dev Summary for 2002-10-01 through 2002-10-13
Message-ID: <Pine.SOL.4.44.0210130100400.6334-100000@death.OCF.Berkeley.EDU>

I realize this is a little early, but I am going on vacation on Monday for
two weeks (Raymond Hettinger has agreed to do the summary while I am
gone), so I wanted to get this out the door before I leave.

Usual 24 hour correction period for the list to make my writing skills and
comprehension skills seem inferior is in effect.  =)

--------------------------------------------------------------------

This is a summary of traffic on the `python-dev mailing list`_ between
October 1, 2002 and October 13, 2002 (inclusive).  It is intended to
inform the wider Python community of on-going developments on the list
that might interest the wider Python community.  To comment on anything
mentioned here, just post to python-list@python.org or comp.lang.python in
the usual way; give your posting a meaningful subject line, and if it's
about a PEP, include the PEP number (e.g. Subject: PEP 201 - Lockstep
iteration). All python-dev members are interested in seeing ideas
discussed by the community, so don't hesitate to take a stance on a PEP
(or anything else for that matter) if you have an opinion.  And if all of
this really interests you then get involved and join Python-dev!

This is the fourth summary written by Brett Cannon (with a partially fried
brain thanks to the GRE).

Summaries by me (2002-09-15 to ... when I burn out) are archived at:
    http://www.ocf.berkeley.edu/~bac/python-dev/summaries/index.php
You can find summaries by Michael Hudson (2002-02-01 to 2001-07-05) at:
    http://starship.python.net/crew/mwh/summaries/index.html
Summaries by A.M. Kuchling (2000-12-01 to 2001-01-31) are at:
    http://www.amk.ca/python/dev/

Please note that this summary is written using reStructuredText_ which can
be found at http://docutils.sourceforge.net/rst.html .  Any unfamiliar
punctuation is probably markup for reST; you can safely ignore it
(although I suggest learning reST; its nice and is accepted for PEP
markup).  Also, because of the wonders of reformatting thanks to whatever
you are using to read this, I cannot guarantee you will be able to run
this text through DocUtils as-is.  If you want to do that, get the
original text from the archive.

.. _python-dev mailing list:
http://mail.python.org/mailman/listinfo/python-dev
.. _reST:
.. _reStructuredText: http://docutils.sf.net/

======================
Summary Announcements
======================

This is a new section to the summary that I have decided to introduce.  It
is mainly going to serve to make any general announcements or comments on
this summary and this summary alone.  All universal comments will stay at
the top of the summaries.

Just to let everyone know, I am taking off for two weeks on vacation
starting 2002-10-14 and I will not return until 2002-10-30.  Now, before
you all start sobbing over the loss of one of my great summaries, you
should know that Raymond Hettinger has graciously taken up the job of temp
summarizer for me and will do the summary while I am gone.

Michael Hudson has made the suggestion that I inject more of my
personality into the summary so as to liven it up a little.  I am
personally quite happy to do this.  But the real question is do you, fine
reader, mind the idea?  If I don't hear from throngs of people going "your
sarcastic tone takes away from the wonderfully drawl summaries and that is
a bad thing", then I will just go ahead and write with personality.  Just
don't complain later.  =)

2.2.2b1 has been occupying Python-dev during this summary period, and so
this summary is shorter than usual.  I left out a bunch of threads that
were discussing bugfixes that I either didn't find interesting or didn't
think the rest of the world would care about.

Now it's time for my personal favor of the month.  I am going to start
applying to grad school for computer (science | programming) when I get
back from vacation.  If anyone knows of Python-friendly schools out there,
let me know.  Heck, I am even willing to leave America to go to school as
long as the classes are in English.  So if you know of any, please let me
know!

And now on to the summary.

=====================================
`Python 2.2.2 beta release on Monday!`__
=====================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029204.html

splintered thread:

	RELEASED: Python 2.2.2b1 :
http://mail.python.org/pipermail/python-dev/2002-October/029337.html

Python 2.2.2.b1 was released on Monday, October 7.  This is the reason (or
perhaps excuse is a better description) for the lighter summary this week.
A good amount of the traffic on Python-dev was about bugfixing 2.2.2.b1.
Most of this probably would not interest the average Python user, and thus
I didn't summarize a bunch of threads.  Taking the GRE also didn't help
with my free time and thus has caused me to cut down on the summary since
I am having to go through a huge backlog to get this out the door.

=====================================
`Dropping support for Tcl 8.0 and 8.1`__
=====================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029024.html

Martin v. Loewis asked if it would be okay to drop support for Tcl_ 8.0
and 8.1 since `_tkinter.c`_ has special code in there just for those
outdated versions.  Guido ok'ed it, so if you are using those still using
a version of Tk from way back when, it's time to upgrade.

.. __tkinter.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/_tkinter.c

=======================================
`*very* revised proposal for interfaces`__
=======================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029042.html

Previously John Williams came up with a proprosal for implementing the
stuff from `PEP 245`_ (Python Interface Syntax) and `PEP 246`_ (Object
Adaptation).  From my understanding of what John has done, it appears he
has written an interface system in pure Python.  If you want backstory to
this long and involved discussion to get an interface system into Python
read the Python-dev Summaries for 2002-08-16 to 2002-09-01 and 2002-09-01
to 2002-09-15 .

Gerald Williams, Michael Chermside, and Esteban Castro all commented on
the implementation and made various suggestions.

John said that he was not done yet implementing this.  But if you were
interested in the whole previous discussion on interfaces you could
consider looking at what John has done.  As for it going into the
language, I suspect that will have to wait until John is done and has
convinced the PEP writers and Python-dev that his implementation fits the
bill.  Stay tuned.

If you want some more background info on interfaces, read the previously
mentioned summaries.  As for object adaptation, read on c.l.py and on
Python-dev anything by Alex Martelli on the subject.  He has become the
main proponent of object adaptation and has written several very extensive
essays on the subject.

.. _PEP 245: http://www.python.org/peps/pep-0245.html
.. _PEP 246: http://www.python.org/peps/pep-0246.html

=================
`perplexed by mro`__
=================
__ http://mail.python.org/pipermail/python-dev/2002-October/029035.html

Splinter threads:

	Re: my proposals about mros (was: perplexed by mro) :
http://mail.python.org/pipermail/python-dev/2002-October/029167.html
	C3 implementation :
http://mail.python.org/pipermail/python-dev/2002-October/029230.html

Samuele Pedroni said he was "trying to wrap [his] head around the mro
computation in 2.2".  Apparently there is the algorithm mentioned at
http://www.python.org/2.2.1/descrintro.html (dubbed the naive algorithm)
and then the one implemented in typeobject.c_ (called the 2.2 algorithm).
Samuele discovered some inconsistencies with the implemented algorithm
that he desired some explanation about.

Guido responded, thankful that someone was giving this a look because his
"intuition about the equivalence between algorithms turned out to be
wrong".  Guido stated that he thought that he wrote the algorithm from the
book "Putting Metaclasses To Work" correctly sans raising an error when
major conflicts occur in the ordering.  In a later email Guido explained
that the naive algorithm came about by his attempt to simplify the
explanation of the 2.2 algorithm.  Guido pretty much wrote the algorithm
from the aforementioned book.  Now the algorithm is not simple, so Guido
did his best to simplify the explanation.  Unknowningly, though, he came
up with a variant on the algorithm in his explanation.

Greg Ewing pointed out that he thought the naive algorithm was nicer since
it seemed to work more intuitively and was easier to explain (and remember
kids, these are basic tenants in Python programming).  Guido ended up
stating that "If Samuele agrees that the naive algorithm works better,
[Guido will] try to make it so in 2.3".  Well, Samuele said that the "2.2
mro is the worst of our options".

There was a problem, though, with the naive algorithm; it is not monotonic
as pointed out by Samuele.  This led him to put out two options:

1. Use the naive algorithm, which had the drawback of being monotonic.
Samuele also believed that it didn't produce "the most natural results".

2. Adopt C3_ as described at
http://www.webcom.com/haahr/dylan/linearization-oopsla96.html and
apparently used by Dylan_ and Goo_ .  This algorithm is monotonic and
Samuele says is more intuitive.

Guido got around to reading the C3_ paper and agreed that "we should adopt
C3".  He thought that the 2.2 algorithm was like the C*[LOOPS] algorithm
mentioned in the paper.  Samuele then wrote a C implementation of the
algorithm.  Guido said he would get to the patch after 2.2.2b1 got out the
door.

.. _C3: http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
.. _Dylan: http://monday.sourceforge.net/wiki/
.. _Goo: http://www.ai.mit.edu/~jrb/goo/manual.43/goomanual_55.html

.._typeobject.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/typeobject.c

===========================
`Psyco requests and patches`__
===========================
__ http://mail.python.org/pipermail/python-dev/2002-October/029038.html

Armin Rigo of psyco_ an email to Python-dev with some thoughts on psyco
and some requests.  After mentioning how he wished more people realized
psyco is meant to be used by anyone and not speed-hungry coders (it is a
cool app and if you have any interest in compilers you should take a look)
and that psyco could get some more advertising he mentioned three patches
that he wrote (patches 617309_ , 617311_ , and 617312_ ) against 2.2.2
that he would like to see be accepted so as to ease maintenance of psyco.

Armin also mentioned how he would like to move psyco forward.  He pointed
out he would like to eventually write it all in Python.  This would
require tracking changes in the interpreter that psyco dealt with.  He
would like to keep this in mind when Python 3 discussions kick up (and
don't ask when that will happen; not for a VERY long time).

In regards to Armin's patches, Martin v. Loewis thought they broke binary
compatibility (big no-no between micro releases), but Armin claimed it
didn't.

After glancing over the patches it seems they have all been applied
against 2.2.2 and are being actively worked on by Armin for application to
2.3.

.. _617309: http://www.python.org/sf/617309
.. _617311: http://www.python.org/sf/617311
.. _617312: http://www.python.org/sf/617312

.. _psyco: http://psyco.sf.net/

===================================================================
`PEP239 (Rational Numbers) Reference Implementation and new issues`__
===================================================================
__http://mail.python.org/pipermail/python-dev/2002-October/029065.html

splintered threads:
	Re: PEP239 (Rational Numbers) Reference Implementation and new
issues :
http://mail.python.org/pipermail/python-dev/2002-October/029068.html

Christopher Craig uploaded `patch 617779`_ that implemented `PEP 239`_
(Rational Numbers).  He had some questions for Python-dev, though,
regarding a couple points.  These few points have become the bane of my
summarizing existence; this thread is huge.

One was whether division should return rationals instead of floats.  Since
rationals keep precision in division they are the most accurate way to
perform division.  THey also make the most output sense (e.g. 1/3).
Problem with this is that rational math is slow and this would cause issue
with any code that expected a float.

The next issue was about comparison.  Should a rational compare only when
it is exactly equal to a float or when the float is really close?

Lastly, Christopher wondered if rationals should hash the same as floats.
The answer to the second issue would influence the answer to this issue.

Issue 1. Eric Raymond was for returning a rational.  Francois was kind of
on the fence.  Christian was +1 for returning rationals.  Guido said ABC
did this and that numeric processes thus ended up being slow.

Issue 2. Christian said "Let it grow!  Let the user feel what precision
he's carrying around, and how much they throw away when they reduce down
to a float."

Issue 3.  Eric Raymond suggested a global "fuzz" variable that defines a
"close-enough-for-equality range"; this idea was used by APL.  Andrew
Koenig was against this because you don't always want a fuzzy comparison
and it destroys substitutability: "If a==b, it is not always true that
f(a)==f(b)".  Andrew said he preferred Scheme's numeric model.  To this,
Guido said that "'It works in Scheme' doesn't give me a warm fuzzy feeling
that it's been tried in real life"; Tim later laid the smack down on
Scheme's numeric model and ended it with "There's a reason the NumPy folks
never bug you for Scheme features <wink>".  Christian pointed out that
keeping it as a rat would prevent overflows from ever occuring from long
division.  Tim was staunchly against a fuzz variable.  Raymond suggested a
fuzz comparison function that took in a fuzz value.  Christopher said that
the way it stands now in the implementation is that rationals are coerced
to floats and then compared.  Oren Tirosh suggested a thired boolean,
'Undetermined', that would be raised when the "difference between A and B
is below the error margin".  David Abrahams said that Boost discussed this
and said that the cost of adding ternary logic was not worth it.

Andrew asked how rats could be optimized.  He suggested ditching trailing
zeros.  Tim wondered how much of a save you would get from this.  Raymond
Hettinger suggested having a builtin variable that would specify the
"maximum denominator magnitude".  Christian liked this idea.  Greg didn't
think this would be a good solution because people using rats are going to
want them specifically because they are exact.  So Raymond suggest the
default be unrestricted denominator.

Guido brought up the question of how rats should be represented when
printed.

The syntax for rats came up.  Greg Ewing got the ball rolling by
suggesting the syntax for rat division as ``\\\``.  M.A. Lemburg suggested
just having a constructor like ``rat(2,3)``.  The discussion then had a
gamut of suggested syntax: ``2:3``, ``2r3``, ``{2/3}`` (Guido shot this
down because he wants to leave the option open for possible set notation),
``<2/3>``,``2r/3``, something by Barry using an extended character that
Pine wouldn't display (it was a joke), and finally ``2/3r`` by Guido.
People agreed that this last one suggested by Guido was the best one.  Tim
also pointed out that Scheme has notation to specify whether a number is
exact or not and using the 'r' notation would basically provide the same
functionality.

But regardless of what syntax people preferred, it was overwhelmingly
agreed that choosing the syntax should wait until rationals have been in
the language for a while and it is known how they are used.

If you only read one thing, read Tim's emails since he explains all of
this really well and is the resident math whiz on Python-dev.

.. _patch 617779: http://www.python.org/sf/617779
.. _PEP 239: http://www.python.org/peps/pep-0239.html

==============================================
`Non-ASCII characters in test_pep277.py in 2.3`__
==============================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029247.html

Guido pointed out that test_pep277.py_ uses an encoding cookie which was
not being recognized by his toolchain.  At some point he stated that he is
"still not 100% comfortable with using arbitrary coding cookies in the
Python distribution".

The reason I mention this thread (beyond for the quote above) is that info
on how to get XEmacs to recognize the cookie came out.  Sjoerd Mullender
sent out the link
http://www.xemacs.org/Documentation/packages/html/mule-ucs_2.html which
helped some people.  As for Linux distro-specific problems, M.A. Lemburg
noticed that SuSE puts Mule support in a package named 'mule-ucs-xemacs'
and once he got the package loaded XEmacs worked.

.. _test_pep277.py:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/test/test_pep277.py

==================================================
`Unclear on the way forward with unsigned integers`__
==================================================
__http://mail.python.org/pipermail/python-dev/2002-October/029249.html

Splinter threads:
	PEP 263 in the works :
http://mail.python.org/pipermail/python-dev/2002-October/029289.html

Mark Hammond was "a little confused by the new world order for working
with integers in extension modules".  Mark wanted know how to create
objects that were more like a collection of bits than an integer.  Tim
suggested creating a Python long; that would act more like an unsigned int
in terms of its bits.

The FutureWarning for hexadecimal constants was brought up and it was
pointed out that to deal with those just stick an 'L' at the end.
Remember folks, that in Python 2.3 ``0x80000000L == 2147483648``.

The usefulness of __future__ statements also came up.  Tim wondered how
useful they were.  Thomas Wouters, though, came to __future__'s defense
and explained how it helped him migrate people to newer versions of Python
without being yelled at for breaking their code.

=============================================
`segmentation fault with python2.3a0 from cvs`__
=============================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029286.html

Gersen Kurz was having problems importing a huge file.  The bug was
attributed to cygwin's malloc implementation, so people might want to
watch out for that.

It was also pointed out that a loop with a bunch of items in a dict
created a huge number of references.  It turns out that dicts use dummy
references in its implementation for when something is deleted.  So don't
be alarmed by huge references even after you deleting an immense dict.

======================================
`Snapshot win32all builds of interest?`__
======================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029352.html

As mentioned in the last summary, Mark Hammond wondered if anyone would
care to have access to compiled snapshots of CVS for Windows.  He got
enough of a response to give access at
http://starship.python.net/crew/mhammond .  This is not like the standard
Windows installer, though; "This version installs no shortcuts, does not
compile .pyc files etc - you are pretty much on your own.
Pythonwin\start_pythonwin.pyw is installed to start Pythonwin, but you
must do so manually".  Mark would like to know if you end up using this.

=======================================
`Set-next-statement in Python debuggers`__
=======================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029377.html

Richie Hindle wants to write a pure Python debugger; the problem is that
it would be difficult without certain C-level stuff exposed to Python.
Specifically, he wants frame.f_lasti (to be found in frameobject.c_ ) to
be writable by Python so he "could implement Set-Next-Statement".

Michael Hudson is the first email I have of someone chiming against this.
He thought it would be better to make it a descriptor so as that "you can
do some sanity checking on the values".  Guido then chimed in saying that
if it was writable that would open up a hole for crashing a program.
Guido eventually said making it read-only would be fine.

This led to Armin Rigo pointing out that you can crash the interpreter
already with the new module "or by writing crappy .pyc files".  Guido
acknowledged this, but said he didn't want to add anymore if it could be
helped.  He pointed out that he wants to stick with the idea that a
segfault is Python's fault unless proven otherwise.

.. _frameobject.c:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/frameobject.c

=================
`Multibyte repr()`__
=================
__ http://mail.python.org/pipermail/python-dev/2002-October/029442.html

A patch was applied that allowed repr() to return characters with the high
bit set; repr() used the "multibyte C library for printing string if
available".  This had caused a bug and made Guido wonder if this was a
good thing to do.  For an example::

  >>> u = u'\u1f40'  # Python 2.2
  >>> s = u.encode('utf8')
  >>> s
  '\xe1\xbd\x80'
  >>>



>
>
>

u

=

u
'
\
u
1
f
4
0
'



>
>
>

s

=

u
.
e
n
c
o
d
e
(
'
u
t
f
8
'
)



>
>
>

s



'12
\
x
8
0
'
  # Notice the extended character
  >
>
>

"The latter output is not helpful, because the encoding of s is not the
locale's encoding".

Martin v. Loewis said that he thought author of the patch's intention was
"to get 'proper' output in interactive mode for strings".  Part of the
issue with all of this is the GNU readline calls setlocale()
automatically.  A patch came about to reset it in the extension module
back to its original state.

The issue of whether pickling would break because of this.  Guido tried it
and had no issue.  Atsuo Ishimoto (who brought up the possible problem)
said it broke when using the ShiftJFS locale.

But the worry of having repr() be locale-specific still lingered.  Martin
said he was "convinced that having repr locale-specific is unacceptable".
He said, though, that having the tp_print slot use a locale-aware print
function was fine and to have it differ from tp_repr was fine.  But this
was shot down by Guido; "tp_print only gets invoked when sys.stdout is a
real file; otherwise str() or repr() get invoked".  Apparently tp_print is
a performance optimization and thus should be fully transparent and not be
different in any way to the user.

Guido said that the multibyte-string patch should be backed out.  With the
pickle issue and different semantics for sys.stdout because of tp_print,
Guido said the patch had to be backed out.

As Tamito Kajiyama said, "one of the virtues of Python is that Python has
no language feature that is (automagically) affected by locale settings".

===================================
`tp_dictoffset calculation in 2.2.2`__
===================================
__ http://mail.python.org/pipermail/python-dev/2002-October/029502.html

Guido asked David Abrahams and Kevin Jacobs if a change in how
tp_dictoffset (found in typeobject.c_ ) was calculated would affect them.
To give some background, tp_dictoffset "tells us where the instance dict
pointer is in the instance object layout".  This was a three step process
(all Guido's words):

1. (Line 1113) For dynamically created types (i.e. created by class
statements), if the new type has no __slots__, the dominant base class has
a zero tp_dictoffset, and the dominant base doesn't have a custom
tp_getattro, a __dict__ will be added to the instance layout, and its
tp_dictoffset is calculated by taking the end of the base class instance
struct (or in a more complicated way if tp_itemsize is nonzero).

2. (Line 1941) If the dominant base has a nonzero tp_dictoffset, copy the
tp_dictoffset from the dominant base.

3. (Line 2090) The tp_dictoffset of the first non-dominant base class that
has a nonzero tp_dictoffset is copied.

That last rule had caused Guido and Jeremey Hylton some problems with some
code they were bugfixing.  Guido wanted to just get rid of that rule since
he though "it is *always* wrong".  Both David and Kevin said that nothing
broke for them, so that is now all straightened out.

======================
`Memory size overflows`__
======================
__ http://mail.python.org/pipermail/python-dev/2002-October/029535.html

Armin Rigo pointed out some potential overflow errors "with objects of
very large sizes".  The issue was when the amount of memory needed to
allocate was calculated there was the chance that it would overflow.
Armin suggested adding macros to deal with various issues.

Arrive Tim Peters, creator of pymalloc.  He admitted that he "always
ignore[s] these [errors] until one pops up in real life" since "Checking
slows the code, and that causes [Tim] pain <0.5 wink>".  He pointed out
another possible overflow calculation problem.  But it was a basically a
hopeless battle since malloc() has its own cross-platform issues.

But Tim did say that if it was decided to go down the macro route, then he
wanted something like what Zope does: ``DO_SOMETHING_OR(RESULT_LVALUE,
INPUT1, ..., ON_ERROR_BLOCK);``.  The result goes into RESULT_LVALUE
unless there is a problem, in which case ON_ERROR_BLOCK is run.

Christian Tismer chimed in and said that he thought we should just move
completely over to 64 bit math.  Ruby had done it successfully so it
wasn't like we were taking a blind leap.  It would also save us the hassle
from doing it down the road when 64 bit processors become the norm instead
of the exception.




From oren-py-d@hishome.net  Sun Oct 13 12:21:36 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Sun, 13 Oct 2002 07:21:36 -0400
Subject: [Python-Dev] Why does the Windows installer use the shortname formof the install directory?
In-Reply-To: <3DA8084E.3010509@tismer.com>
References: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net> <3DA8084E.3010509@tismer.com>
Message-ID: <20021013112136.GA77207@hishome.net>

On Sat, Oct 12, 2002 at 04:32:30AM -0700, Christian Tismer wrote:
> >There isn't another variant that adds "but only if it contains a space".
> 
> The F1 key gives you a regular windows help file, as I
> remember.
> And you can surround the
>    Convert to short filename
> thing with an if contruct that checks for a space, something like
>    If MAINDIR Contains " " then
>       # here the Convert thingie
>    End

Translated to WISEspeak:

  Parse String "%MAINDIR%" into BEFORESPACE and AFTERSPACE  [*]
  If AFTERSPACE Not Equal "" then
    Set Variable MAINDIR to %MAINDIR%
  End

[*] Set "Pattern/Position" field to single space (invisible in dialog box)
    Set "Operation" field to "Split value at first occurence of pattern"

	Oren


From skip@manatee.mojam.com  Sun Oct 13 13:00:18 2002
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 13 Oct 2002 07:00:18 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200210131200.g9DC0InD000870@manatee.mojam.com>

Bug/Patch Summary
-----------------

310 open / 2943 total bugs (+15)
96 open / 1727 total patches (-16)

New Bugs
--------

PythonLauncher Dialogs do not work (2002-10-06)
	http://python.org/sf/619245
test_builtin hangs on MacOSX 10.2.1 (2002-10-06)
	http://python.org/sf/619400
Import fails in Idle (2002-10-07)
	http://python.org/sf/619713
imaplib fails with literals in LIST resp (2002-10-07)
	http://python.org/sf/619732
segmentation fault importing huge source (2002-10-07)
	http://python.org/sf/619789
__ipow__ broken for 'object's (2002-10-08)
	http://python.org/sf/620179
inspect and object instances (2002-10-08)
	http://python.org/sf/620190
HTMLParser:endtag events in comments (2002-10-08)
	http://python.org/sf/620243
core setup.py fails on Windows (2002-10-08)
	http://python.org/sf/620364
Max recursion limit with "*?" pattern (2002-10-08)
	http://python.org/sf/620412
distutils mixed stdin/stdout output (2002-10-08)
	http://python.org/sf/620630
websucker relative-URL errors (2002-10-09)
	http://python.org/sf/620705
missing mappings in locale tables (2002-10-09)
	http://python.org/sf/620739
SocketServer/socket allow_reuse_address  (2002-10-09)
	http://python.org/sf/620783
File write examples are inadequate (2002-10-09)
	http://python.org/sf/621057
email: ASCII decoding error on 8bit data (2002-10-10)
	http://python.org/sf/621457
Tix Checklist getselection getstatus bug (2002-10-10)
	http://python.org/sf/621511
sizeof(struct stat) changes (2002-10-10)
	http://python.org/sf/621523
docs do not include spec info from PEPs (2002-10-10)
	http://python.org/sf/621526
Numerous defunct threads left behind (2002-10-10)
	http://python.org/sf/621548
menues in emacs21 in transient mode (2002-10-10)
	http://python.org/sf/621554
httplib HEAD request fails - keepalive (2002-10-11)
	http://python.org/sf/622042

New Patches
-----------

C3 MRO algorithm implementation (2002-10-06)
	http://python.org/sf/619475
tkFileDialog multiple option (2002-10-11)
	http://python.org/sf/621891
dummy_thread.py implementation (2002-10-13)
	http://python.org/sf/622537

Closed Bugs
-----------

header file problems (2002-09-10)
	http://python.org/sf/607253
Numpy doesn't build for Python.framework (2002-09-17)
	http://python.org/sf/610730
broken link in documentation (2002-09-25)
	http://python.org/sf/614821
Connecting to IPv4 addrs from IPv6 socke (2002-10-04)
	http://python.org/sf/618594
list(xrange(sys.maxint/4)) again (2002-10-04)
	http://python.org/sf/618623

Closed Patches
--------------

a move function for shutil.py (2001-08-04)
	http://python.org/sf/448038
--python-build for install (2001-09-05)
	http://python.org/sf/458898
Multibyte string on string::string_print (2001-11-08)
	http://python.org/sf/479898
binary packagers (2002-03-19)
	http://python.org/sf/531901
2.2 patches for BSD/OS 5.0 (2002-03-26)
	http://python.org/sf/535335
Webbrowser.py and konqueror (2002-04-04)
	http://python.org/sf/539360
addition of cmath.arg function (2002-04-16)
	http://python.org/sf/544909
Fix broken smtplib.login() (2002-05-05)
	http://python.org/sf/552605
Mutable object change flag (2002-05-12)
	http://python.org/sf/555251
Implementation of major, minor and makedev (2002-06-14)
	http://python.org/sf/569139
AUTH method LOGIN for smtplib (2002-06-21)
	http://python.org/sf/572031
Adds Galeon support to webbrowser.py (2002-07-24)
	http://python.org/sf/585913
galeon support in webbrowser (2002-07-25)
	http://python.org/sf/586437
Bugfix for urllib2.py (2002-08-25)
	http://python.org/sf/599836
Updated .spec file for 2.2 series. (2002-09-18)
	http://python.org/sf/611191
"Bare" text tag_configure in Tkinter (2002-09-21)
	http://python.org/sf/612602
fix for urllib2.AbstractBasicAuthHandler (2002-09-25)
	http://python.org/sf/614596
textwrap: prefix short lines (2002-09-30)
	http://python.org/sf/616496
Fix test_pep277 on Win9x (2002-10-05)
	http://python.org/sf/619108


From pedronis@bluewin.ch  Sun Oct 13 16:22:19 2002
From: pedronis@bluewin.ch (Samuele Pedroni)
Date: Sun, 13 Oct 2002 17:22:19 +0200
Subject: [Python-Dev] Python-dev Summary for 2002-10-01 through 2002-10-13
References: <Pine.SOL.4.44.0210130100400.6334-100000@death.OCF.Berkeley.EDU>
Message-ID: <008e01c272cc$521e2f80$6d94fea9@newmexico>

From: "Brett Cannon" <bac@OCF.Berkeley.EDU>
> =================
> `perplexed by mro`__
> =================
> __ http://mail.python.org/pipermail/python-dev/2002-October/029035.html
>
> Splinter threads:
>
> Re: my proposals about mros (was: perplexed by mro) :
> http://mail.python.org/pipermail/python-dev/2002-October/029167.html
> C3 implementation :
> http://mail.python.org/pipermail/python-dev/2002-October/029230.html
>
> Samuele Pedroni said he was "trying to wrap [his] head around the mro
> computation in 2.2".  Apparently there is the algorithm mentioned at
> http://www.python.org/2.2.1/descrintro.html (dubbed the naive algorithm)
> and then the one implemented in typeobject.c_ (called the 2.2 algorithm).
> Samuele discovered some inconsistencies with the implemented algorithm
> that he desired some explanation about.
>
> Guido responded, thankful that someone was giving this a look because his
> "intuition about the equivalence between algorithms turned out to be
> wrong".  Guido stated that he thought that he wrote the algorithm from the
> book "Putting Metaclasses To Work" correctly sans raising an error when
> major conflicts occur in the ordering.  In a later email Guido explained
> that the naive algorithm came about by his attempt to simplify the
> explanation of the 2.2 algorithm.  Guido pretty much wrote the algorithm
> from the aforementioned book.  Now the algorithm is not simple, so Guido
> did his best to simplify the explanation.  Unknowningly, though, he came
> up with a variant on the algorithm in his explanation.
>
> Greg Ewing pointed out that he thought the naive algorithm was nicer since
> it seemed to work more intuitively and was easier to explain (and remember
> kids, these are basic tenants in Python programming).  Guido ended up
> stating that "If Samuele agrees that the naive algorithm works better,
> [Guido will] try to make it so in 2.3".  Well, Samuele said that the "2.2
> mro is the worst of our options".
>
> There was a problem, though, with the naive algorithm; it is not monotonic
> as pointed out by Samuele.  This led him to put out two options:
>
> 1. Use the naive algorithm, which had the drawback of being monotonic.

s/being/not being/

> Samuele also believed that it didn't produce "the most natural results".
>
> 2. Adopt C3_ as described at
> http://www.webcom.com/haahr/dylan/linearization-oopsla96.html and
> apparently used by Dylan_ and Goo_ .

only by Goo. Dylan uses stilll a different algorithm with results more similar
to those of the one used by CLOS, but I don't think that's important. They are
all described in the paper. Dylan would have adopted C3, but it was switching
algorithm aftering starting with the CLOS one and having already code around.

> This algorithm is monotonic and
> Samuele says is more intuitive.

more intuitive in its results.

> Guido got around to reading the C3_ paper and agreed that "we should adopt
> C3".

>  He thought that the 2.2 algorithm was like the C*[LOOPS] algorithm
> mentioned in the paper.

there was a typo, it's L*[LOOPS]. Btw, we still don't know exactly, it's
probably a variant, and I don't think it's so relevant.

> Samuele then wrote a C implementation of the
> algorithm.  Guido said he would get to the patch after 2.2.2b1 got out the
> door.
>
> .. _C3: http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
> .. _Dylan: http://monday.sourceforge.net/wiki/

if the reference to Dylan goes away above, this one should also go away.
Otherwise I would use

http://www.gwydiondylan.org/

as reference URL for Dylan or simply comp.lang.dylan .


> .. _Goo: http://www.ai.mit.edu/~jrb/goo/manual.43/goomanual_55.html
>
> .._typeobject.c:
>
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/t
ypeobject.c

regards.



From martin@v.loewis.de  Sun Oct 13 20:48:08 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sun, 13 Oct 2002 21:48:08 +0200
Subject: [Python-Dev] The -U flag
Message-ID: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>

Since Python allows Unicode strings in sys.path, we are making
progress on getting -U to work. When entering interactive mode, the
stumbling block is

_idmap = ''
for i in range(256): _idmap = _idmap + chr(i)
del i

Here, _idmap is initialized with a Unicode string, and the chr(i)
results are promoted to Unicode, which eventually causes a
UnicodeErorr when you get past 127.

The work-around would be to write

_idmap = str('')
for i in range(256): _idmap = _idmap + chr(i)
del i

With that, we can enter interactive mode in python -U.

Is such a change acceptable?

Curious,
Martin

P.S. In case you wonder how the testsuite works for -U: The first
problem is

  File "/usr/src/python/Lib/warnings.py", line 267, in ?
    filterwarnings("ignore", category=OverflowWarning, append=1)
  File "/usr/src/python/Lib/warnings.py", line 131, in filterwarnings
    assert isinstance(message, str), "message must be a string"
AssertionError: message must be a string

I'm not going to ask whether I can use basestring there, yet...


From tim.one@comcast.net  Sun Oct 13 21:07:39 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 13 Oct 2002 16:07:39 -0400
Subject: [Python-Dev] Why does the Windows installer use the shortname
 formof the install directory?
In-Reply-To: <20021013112136.GA77207@hishome.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEENBLAB.tim.one@comcast.net>

[Christian Tismer]
> The F1 key gives you a regular windows help file, as I remember.

I hate "regular Windows help files", but I agree it exists <wink>.

> And you can surround the
>    Convert to short filename
> thing with an if contruct that checks for a space, something like
>    If MAINDIR Contains " " then
>       # here the Convert thingie
>    End

[Oren Tirosh]
> Translated to WISEspeak:
>
>   Parse String "%MAINDIR%" into BEFORESPACE and AFTERSPACE  [*]
>   If AFTERSPACE Not Equal "" then
>     Set Variable MAINDIR to %MAINDIR%
>   End

Christian's was already WISEspeak, but, as before, I've got insufficent
interest to bother with any of this.  If one of you wants to submit a tested
patch, great!



From amk@nyman.amk.ca  Sun Oct 13 22:38:38 2002
From: amk@nyman.amk.ca (A.M. Kuchling)
Date: Sun, 13 Oct 2002 17:38:38 -0400
Subject: [Python-Dev] PEP 218 update questions
Message-ID: <200210132138.g9DLccX06379@nyman.amk.ca>

I've just updated PEP 218 to match the sets.py module that's in CVS.  

The original PEP text had a single Set class, and Set instances froze
after their __hash__ was called so you could use a set as a dictionary
key.  sets.py takes a more straightforward approach, with distinct Set
and ImmutableSet classes.

Two things in the PEP need to be updated to take this change into account.

1)
The long-term proposal in PEP 218 lists a single built-in conversion
function, set(iterable).  This made sense when there was a single Set
class, but now this needs to be updated.  

How would instances of a built-in immutable set type be created?
Would there be a second immutable_set() built-in, or would the set()
function take an additional argument: set(iterable, immutable=True)?

2)
The PEP proposes {1,2,3} as the set notation and {-} for the empty
set.  Would there be different syntax for an immutable and a mutable
set?  

My thought is, "no"; {1,2,3} or {-} is always mutable, and an
immutable set would be written as immutable_set({1,2,3}).  That's up
to python-dev, though, but whatever gets chosen needs to be specified
in the PEP.

--amk


From drifty@bigfoot.com  Sun Oct 13 22:43:27 2002
From: drifty@bigfoot.com (Brett Cannon)
Date: Sun, 13 Oct 2002 14:43:27 -0700 (PDT)
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: <200210132138.g9DLccX06379@nyman.amk.ca>
Message-ID: <Pine.SOL.4.44.0210131436170.3654-100000@death.OCF.Berkeley.EDU>

[A.M. Kuchling]

> How would instances of a built-in immutable set type be created?
> Would there be a second immutable_set() built-in, or would the set()
> function take an additional argument: set(iterable, immutable=True)?
>

I say have a single set() with an argument flag.  Mutable or not, they are
both subsets of BaseSet and thus should be reflected in the constructor
function.

> 2)
> The PEP proposes {1,2,3} as the set notation and {-} for the empty
> set.  Would there be different syntax for an immutable and a mutable
> set?
>

As long as mutable sets can be automatically converted to immutable, I say
keep only one syntax.  If the user wants to explicitly create an immutable
set, call set(mutable_set, immutable=True) or have a very explicit way for
mutable sets to turn themselves into immutable sets.  No need to add more
syntax for something that can automatically be done for the user.

-Brett C.



From aleaxit@yahoo.com  Sun Oct 13 23:05:44 2002
From: aleaxit@yahoo.com (Alex Martelli)
Date: Mon, 14 Oct 2002 00:05:44 +0200
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: <Pine.SOL.4.44.0210131436170.3654-100000@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.44.0210131436170.3654-100000@death.OCF.Berkeley.EDU>
Message-ID: <02101400054404.01541@arthur>

On Sunday 13 October 2002 23:43, Brett Cannon wrote:
> [A.M. Kuchling]
>
> > How would instances of a built-in immutable set type be created?
> > Would there be a second immutable_set() built-in, or would the set()
> > function take an additional argument: set(iterable, immutable=True)?
>
> I say have a single set() with an argument flag.  Mutable or not, they
> are both subsets of BaseSet and thus should be reflected in the
> constructor function.

Why expose a function rather than exposing the type directly?  I think
we should have Set, the type, rather than set, the function, as the
built-in -- so it can be inherited from as well as being constructed.
This, in turn, suggests exposing type ImmutableSet too.

That is, if we want such built-ins at all.  If (e.g.) types such as array
and mmap can stay in modules, why not sets, for now?


> > 2)
> > The PEP proposes {1,2,3} as the set notation and {-} for the empty
> > set.  Would there be different syntax for an immutable and a mutable
> > set?
>
> As long as mutable sets can be automatically converted to immutable, I
> say keep only one syntax.  If the user wants to explicitly create an
> immutable set, call set(mutable_set, immutable=True) or have a very
> explicit way for mutable sets to turn themselves into immutable sets.  No
> need to add more syntax for something that can automatically be done for
> the user.

Either via ImmutableSet, or the method currently private and named
_as_immutable could be made public and named as_immutable or
asImmutable.  I prefer the latter solution because it would give us a
needed protocol -- "return an immutable snapshot of your current
state" -- sets now do it, but it would be nice to have a way for all
mutable types that are interested in cooperating in such "snapshot"
operations to be able to (lists could return tuple(self), dicts could
return 'frozendicts' if and when such beasts appear, etc) in such a
uniform way.  One last possibility, there could instead be a builtin
immutable(x) (or immutable_copy(x)) that knows about some builtin
types (e.g. turns lists into tuples itself, returns already-immutable
types unchanged) and tries to delegate to a new special method
__as_immutable__ -- this would be at least as good as the second
solution above and more in keeping with how several existing Python
builtins already work.


Alex


From arigo@ulb.ac.be  Sun Oct 13 23:23:34 2002
From: arigo@ulb.ac.be (Armin Rigo)
Date: Mon, 14 Oct 2002 00:23:34 +0200
Subject: [Python-Dev] Memory size overflows
References: <LNBBLJKPBEHFEDALKOLCEECPBLAB.tim.one@comcast.net>
Message-ID: <3DA9F266.C87E5A58@ulb.ac.be>

Hello Tim,

Let me quote from stringobject.c the "heartbreaking code such as
string_repeat's":

        nbytes = size * sizeof(char);
        if (nbytes / sizeof(char) != (size_t)size ||
            nbytes + sizeof(PyStringObject) <= nbytes)
                <OverflowError>
        op = (PyStringObject *)
                PyObject_MALLOC(sizeof(PyStringObject) + nbytes);
        if (op == NULL)
                <MemoryError>

With the proper macro, the whole code above can be written as:

        nbytes = Py_MUL_ADD(size, sizeof(char), sizeof(PyStringObject));
        op = (PyStringObject *) PyObject_MALLOC(nbytes);
        if (op == NULL)
                <MemoryError>

which seems much cleaner to me.

For this reason I do not believe it is a good idea here to define a
macro a la Zope C:

    DO_MULADD_OR(nbytes, size, sizeof(char), sizeof(PyStringObject),
goto Overflow);

for two reasons: there is no real need for a special "Overflow" case, as
long as the returned 'nbytes' is guaranteed to be un-malloc-able;
besides, no existing macro (e.g. _PyObject_VAR_SIZE) can support an
extra "goto Overflow" case without breaking C code that depends on it.

There is however a difference between the two above code fragments, in
that the former will raise an OverflowError for "really too large
values" and a MemoryError for "reasonable value but still too large for
me to allocate right now".  Whether this is a correct behavior is
debatable.  I would argue that raising MemoryError in both cases is
fine: after all, "123"*9999999999999999 could be perfectly fine in
another Python implementation with lazy strings.  It is thus only a
memory limit that makes this fail in the current implementation.

Note that if we define MAX_SIZE to ((size_t)-1), the overflow check in
the present stringobject.c could be more efficiently written down as:

        if (size > (MAX_SIZE - sizeof(PyStringObject)) / sizeof(char))
                <OverflowError>

where everything at the right of '>' reduces to a compile-time
constant.  The Py_MUL_ADD macro can be implemented with this idea in
mind, which works best if the 2nd and 3rd arguments are constants.

> note that anything/non_constant can be incredibly expensive!

Yes.  Couldn't a better overflow-checking algorithm be designed for
these cases?  For example, to multiply the non-constants 'a' and 'b', we
can be sure that there is no overflow if both 'a' and 'b' are small
enough (which is the common case anyway).  I believe I can even think
about a single Py_MUL_ADD macro that is as efficient as above with
constant arguments, and otherwise uses this trick.

About using 64-bit computations everywhere (as suggested by Christian):
if the length of objects (e.g. lists) is still constrained to 31 bits,
then it's fine; otherwise, the problem is the same.  Do all supported
64-bit platforms have a 32-bit 'int', and thus a 31-bit limit on object
lengths?

By the way, this makes me think of a good efficient overflow-detection
method for platforms with a "long long" type longer than size_t:

  unsigned long long bytes = ((unsigned long long)a)*((unsigned long
long)b)+c;
  if (bytes > ((size_t)-1))
    <overflow>;

This compiles to very good assembly code on x86.

Mmmh, in summary I would recommend that we define a common
"multiply-and-add" macro, returning a size_t or OVERFLOW_VALUE in case
of overflow, which is implemented in several different ways on different
platforms, as selected by #if's.  The same macro is used for just
"multiply" or just "add" by relying on the compiler's strengh reduction.


Armin



From drifty@bigfoot.com  Mon Oct 14 00:11:12 2002
From: drifty@bigfoot.com (Brett Cannon)
Date: Sun, 13 Oct 2002 16:11:12 -0700 (PDT)
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: <02101400054404.01541@arthur>
Message-ID: <Pine.SOL.4.44.0210131556580.3654-100000@death.OCF.Berkeley.EDU>

[Alex Martelli]

> Why expose a function rather than exposing the type directly?  I think
> we should have Set, the type, rather than set, the function, as the
> built-in -- so it can be inherited from as well as being constructed.

That is what I meant; function is a factory function like dict() and
friends are now.  Sorry if that wasn't clear.

> This, in turn, suggests exposing type ImmutableSet too.
>

I don't see that as an automatic conclusion.  I prefer to view
ImmutableSet as just as a Set with a property turned on.

But tuple and lists do set a precedent for clearly separating a mutable
and immutable version of something.  Probably should stick with that
separation.

> > As long as mutable sets can be automatically converted to immutable, I
> > say keep only one syntax.  If the user wants to explicitly create an
> > immutable set, call set(mutable_set, immutable=True) or have a very
> > explicit way for mutable sets to turn themselves into immutable sets.  No
> > need to add more syntax for something that can automatically be done for
> > the user.
>
> Either via ImmutableSet, or the method currently private and named
> _as_immutable could be made public and named as_immutable or
> asImmutable.  I prefer the latter solution because it would give us a
> needed protocol -- "return an immutable snapshot of your current
> state" -- sets now do it, but it would be nice to have a way for all
> mutable types that are interested in cooperating in such "snapshot"
> operations to be able to (lists could return tuple(self), dicts could
> return 'frozendicts' if and when such beasts appear, etc) in such a
> uniform way.  One last possibility, there could instead be a builtin
> immutable(x) (or immutable_copy(x)) that knows about some builtin
> types (e.g. turns lists into tuples itself, returns already-immutable
> types unchanged) and tries to delegate to a new special method
> __as_immutable__ -- this would be at least as good as the second
> solution above and more in keeping with how several existing Python
> builtins already work.
>

I really like that idea.  Being able to use practically anything as a dict
key would be rather nice.  There doesn't need to be a guarantee that the
copy will keep up with the original, mutable object, just that it
corresponds at the immutable object's creation.  That eliminates that
possible hornet's nest.

If we could come up with an immutable dict (I suspect just removing
__setitem__() doesn't cut it =) we could take that and be able
to create immutable object copies by just making a snapshot of the
object's __dict__, __bases__, and such.

But the big use I see of an immutable_copy() function is being able to
take parameters and making sure you don't change them in-place.  You can,
of course, just make sure you don't change them, but this could eliminate
having to watch out for that and let you worry about other things.

-Brett



From guido@python.org  Mon Oct 14 00:48:33 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 13 Oct 2002 19:48:33 -0400
Subject: [Python-Dev] The -U flag
In-Reply-To: Your message of "Sun, 13 Oct 2002 21:48:08 +0200."
 <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>
Message-ID: <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>

> Since Python allows Unicode strings in sys.path, we are making
> progress on getting -U to work. When entering interactive mode, the
> stumbling block is
> 
> _idmap = ''
> for i in range(256): _idmap = _idmap + chr(i)
> del i
> 
> Here, _idmap is initialized with a Unicode string, and the chr(i)
> results are promoted to Unicode, which eventually causes a
> UnicodeErorr when you get past 127.
> 
> The work-around would be to write
> 
> _idmap = str('')
> for i in range(256): _idmap = _idmap + chr(i)
> del i
> 
> With that, we can enter interactive mode in python -U.
> 
> Is such a change acceptable?

Only with a comment that explains it -- otherwise the next person
looking at the code will remove it.

But didn't we at one point conclude that -U was never gonna work?  And
wasn't that why it's no longer documented?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct 14 00:52:05 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 13 Oct 2002 19:52:05 -0400
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: Your message of "Sun, 13 Oct 2002 17:38:38 EDT."
 <200210132138.g9DLccX06379@nyman.amk.ca>
References: <200210132138.g9DLccX06379@nyman.amk.ca>
Message-ID: <200210132352.g9DNq5N09900@pcp02138704pcs.reston01.va.comcast.net>

> I've just updated PEP 218 to match the sets.py module that's in CVS.  

Thanks!

> The original PEP text had a single Set class, and Set instances froze
> after their __hash__ was called so you could use a set as a dictionary
> key.  sets.py takes a more straightforward approach, with distinct Set
> and ImmutableSet classes.
> 
> Two things in the PEP need to be updated to take this change into account.
> 
> 1)
> The long-term proposal in PEP 218 lists a single built-in conversion
> function, set(iterable).  This made sense when there was a single Set
> class, but now this needs to be updated.  
> 
> How would instances of a built-in immutable set type be created?
> Would there be a second immutable_set() built-in, or would the set()
> function take an additional argument: set(iterable, immutable=True)?
> 
> 2)
> The PEP proposes {1,2,3} as the set notation and {-} for the empty
> set.  Would there be different syntax for an immutable and a mutable
> set?  
> 
> My thought is, "no"; {1,2,3} or {-} is always mutable, and an
> immutable set would be written as immutable_set({1,2,3}).  That's up
> to python-dev, though, but whatever gets chosen needs to be specified
> in the PEP.

I don't want to think about this right now -- and it's not yet
relevant since I expect this won't become an built-in type for a
couple more releases.  Can't you just add these questions to the PEP
as unresolved issues?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct 14 00:55:32 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 13 Oct 2002 19:55:32 -0400
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: Your message of "Sun, 13 Oct 2002 16:11:12 PDT."
 <Pine.SOL.4.44.0210131556580.3654-100000@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.44.0210131556580.3654-100000@death.OCF.Berkeley.EDU>
Message-ID: <200210132355.g9DNtWr09924@pcp02138704pcs.reston01.va.comcast.net>

> I don't see that as an automatic conclusion.  I prefer to view
> ImmutableSet as just as a Set with a property turned on.

But it supports a different set of methods!  I prefer to view it as a
separate type.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Raymond Hettinger" <python@rcn.com  Mon Oct 14 01:09:55 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Sun, 13 Oct 2002 20:09:55 -0400
Subject: [Python-Dev] PEP 218 update questions
References: <200210132138.g9DLccX06379@nyman.amk.ca>
Message-ID: <002501c27316$06f21b80$680ca044@oemcomputer>

> 1)
> The long-term proposal in PEP 218 lists a single built-in conversion
> function, set(iterable).  This made sense when there was a single Set
> class, but now this needs to be updated.  
> 
> How would instances of a built-in immutable set type be created?
> Would there be a second immutable_set() built-in, or would the set()
> function take an additional argument: set(iterable, immutable=True)?
> 

-1 on the property version.  As it stands now, immutables can be 
constructed directly from a class with its own methods or, more
likely, is an automagically created proxy used internally by
mutuable sets.  I think of immutable sets as regular sets that are
frozen for purposes of inclusion in another set -- there is almost
no reason for instantiatiating them directly (other than to save
coercion time for sets whose contents are known and fixed
at the time of instatiation).

> 2)
> The PEP proposes {1,2,3} as the set notation and {-} for the empty
> set.  Would there be different syntax for an immutable and a mutable
> set?  
> 
> My thought is, "no"; {1,2,3} or {-} is always mutable, and an
> immutable set would be written as immutable_set({1,2,3}).  That's up
> to python-dev, though, but whatever gets chosen needs to be specified
> in the PEP.

+1.  Immutable sets have their own weirdness.  No use reflecting
that in the core language syntax.  Besides, they may yet evolve 
further (perhaps with an animmutable.refresh() method).


Raymond Hettinger


From tdelaney@avaya.com  Mon Oct 14 01:27:18 2002
From: tdelaney@avaya.com (Delaney, Timothy)
Date: Mon, 14 Oct 2002 10:27:18 +1000
Subject: [Python-Dev] Python-dev Summary for 2002-10-01 through 2002-1
 0-13
Message-ID: <B43D149A9AB2D411971300B0D03D7E8BF0A667@natasha.auslabs.avaya.com>

> From: Brett Cannon [mailto:bac@OCF.Berkeley.EDU]
>
> .. _frameobject.c:
> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/d
> ist/src/Objects/frameobject.c

Might be worthwhile always including a TinyURL as well as the full URL to
deal with wrapping issues (I would suggested a Shorter Link, but Alex
Martelli showed me a better one).

For example, the above URL gets changed to:

    http://tinyurl.com/1ynh

so the resulting entry would look like

.. _frameobject.c: http://tinyurl.com/1ynh
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Object
s/frameobject.c

It also suggests that it may be worthwhile special-casing the reST parser
for this case (and http://makeashorterlink.com/ ... any others?).

Tim Delaney


From drifty@bigfoot.com  Mon Oct 14 01:57:36 2002
From: drifty@bigfoot.com (Brett Cannon)
Date: Sun, 13 Oct 2002 17:57:36 -0700 (PDT)
Subject: [Python-Dev] Python-dev Summary for 2002-10-01 through 2002-1
 0-13
In-Reply-To: <B43D149A9AB2D411971300B0D03D7E8BF0A667@natasha.auslabs.avaya.com>
Message-ID: <Pine.SOL.4.44.0210131753580.20875-100000@death.OCF.Berkeley.EDU>

[Delaney, Timothy]

> > From: Brett Cannon [mailto:bac@OCF.Berkeley.EDU]
>
> Might be worthwhile always including a TinyURL as well as the full URL to
> deal with wrapping issues (I would suggested a Shorter Link, but Alex
> Martelli showed me a better one).
>
> For example, the above URL gets changed to:
>
>     http://tinyurl.com/1ynh
>
> so the resulting entry would look like
>
> .. _frameobject.c: http://tinyurl.com/1ynh
> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Object
> s/frameobject.c
>

I could do that, but I have two concerns.  One is the extra amount of time
it woulkd take to do links for every long link (tend to be a decent
amount), and there is no guarantee that tinyurl will be around and thus
make the archived links break.  Now I know that the long links will break
at some point too, but I would like to minimize that problem.  And I
realize putting both in solves this, but I just don't love the idea that
half of all the links (redundent or not) could go bad because of one site
(if python.org goes down, Python will have gone down so I am not going to
worry about that).

-Brett



From martin@v.loewis.de  Mon Oct 14 05:32:14 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Oct 2002 06:32:14 +0200
Subject: [Python-Dev] The -U flag
In-Reply-To: <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>
 <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3u1jpy71t.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> But didn't we at one point conclude that -U was never gonna work?  

I'm not aware of a formal decision about that. I readily admit that it
wont' work in any foreseeable future, but I haven't given up for P3k :-)

This time, my interest was triggered by someone on c.l.p who could not
reverse a string, since it was in UTF-8; he was asking why Python is
so difficult for Swedish people.

> And wasn't that why it's no longer documented?

Yes: because it won't work in any foreseeable future.

I'll take your statement that "small" changes towards making -U work
are acceptable; I'll see how far I get with small changes until I get
stuck again.

Regards,
Martin


From eggsy@gmx.de  Mon Oct 14 05:50:07 2002
From: eggsy@gmx.de (Eric Johnson)
Date: Mon, 14 Oct 2002 06:50:07 +0200 (MEST)
Subject: [Python-Dev] Missing symbol in sre.py  (Python 2.2.2b1)
Message-ID: <20478.1034571007@www43.gmx.net>

There appears to still be a missing public symbol definition in the Python 
2.2.2b1 
version of sre.py 
 
The documentation implies that finditer should work and indeed it is 
defined 
in the module if sys.hexversion > 0x02020000 however it is not listed in 
the 
__all__ module variable. 
 
This means that if I use 
        import re 
I am unable to use re.finditer but if I use 
        import sre 
then I can use sre.finditer. 
 
I thought that the recommended approach was to use import re. 
 
I think this is  the outstanding bug 585882 re.finditer which was for 
Python 2.2.1. 
 
Eric 
 

-- 
+++ GMX - Mail, Messaging & more  http://www.gmx.net +++
NEU: Mit GMX ins Internet. Rund um die Uhr für 1 ct/ Min. surfen!



From aleaxit@yahoo.com  Mon Oct 14 07:17:58 2002
From: aleaxit@yahoo.com (Alex Martelli)
Date: Mon, 14 Oct 2002 08:17:58 +0200
Subject: [Python-Dev] PEP 218 update questions
In-Reply-To: <Pine.SOL.4.44.0210131556580.3654-100000@death.OCF.Berkeley.EDU>
References: <Pine.SOL.4.44.0210131556580.3654-100000@death.OCF.Berkeley.EDU>
Message-ID: <02101408175804.02129@arthur>

On Monday 14 October 2002 01:11, Brett Cannon wrote:
> [Alex Martelli]
>
> > Why expose a function rather than exposing the type directly?  I think
> > we should have Set, the type, rather than set, the function, as the
> > built-in -- so it can be inherited from as well as being constructed.
>
> That is what I meant; function is a factory function like dict() and
> friends are now.  Sorry if that wasn't clear.

I think of functions (including factory ones) and types (or classes) as
separate, e.g. because I can subclass the latter but not the former.  But
the docs sometimes aren't crystal clear on that because what used
to be factory functions (e.g. int) have recently become types.


> > This, in turn, suggests exposing type ImmutableSet too.
>
> I don't see that as an automatic conclusion.  I prefer to view
> ImmutableSet as just as a Set with a property turned on.

I think in Python that wouldn't work, because both Set and
ImmutableSet have methods the other shouldn't have -- so
if you inherit you'd better know which one you want, since
removing methods during inheritance is near-unfeasible.


> But tuple and lists do set a precedent for clearly separating a mutable
> and immutable version of something.  Probably should stick with that
> separation.

However tuples aren't "immutable lists" -- they don't have non-mutating
methods that lists do have, for example.  So the precedent's iffy.


> I really like that idea.  Being able to use practically anything as a
> dict key would be rather nice.  There doesn't need to be a guarantee that
> the copy will keep up with the original, mutable object, just that it
> corresponds at the immutable object's creation.  That eliminates that
> possible hornet's nest.

The needed axiom is that the original and copy are == at creation
time, and loosely speaking that apart from mutability they are more
or less equivalent objects in some sense or other (e.g. a mutable
one may be rebuilt from the immutable one later)


> If we could come up with an immutable dict (I suspect just removing
> __setitem__() doesn't cut it =) we could take that and be able
> to create immutable object copies by just making a snapshot of the
> object's __dict__, __bases__, and such.
>
> But the big use I see of an immutable_copy() function is being able to
> take parameters and making sure you don't change them in-place.  You can,
> of course, just make sure you don't change them, but this could eliminate
> having to watch out for that and let you worry about other things.

For that you can use copy.copy or if need be copy.deepcopy -- i.e. you
may not care if the function makes changes, as long as it's on a copy,
not on the original.  BTW this suggests that copy.immutablecopy, not a
builtin, is the right place to put the new hypothetical functionality (still
probably cooperating with an optional special method, like copy.copy
and copy.deepcopy do).  Would be weird if the function to make ordinary 
copies lived one place and the one to make immutable copies lived
somewhere else, I think.

BTW, I removed: "A.M. Kuchling" <amk@nyman.amk.ca> from the Cc
since it's bouncing...


Alex


From mal@lemburg.com  Mon Oct 14 08:37:06 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 14 Oct 2002 09:37:06 +0200
Subject: [Python-Dev] The -U flag
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de> <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DAA7422.8000703@lemburg.com>

Guido van Rossum wrote:
>>Since Python allows Unicode strings in sys.path, we are making
>>progress on getting -U to work. When entering interactive mode, the
>>stumbling block is
>>
>>_idmap = ''
>>for i in range(256): _idmap = _idmap + chr(i)
>>del i
>>
>>Here, _idmap is initialized with a Unicode string, and the chr(i)
>>results are promoted to Unicode, which eventually causes a
>>UnicodeErorr when you get past 127.
>>
>>The work-around would be to write
>>
>>_idmap = str('')
>>for i in range(256): _idmap = _idmap + chr(i)
>>del i
>>
>>With that, we can enter interactive mode in python -U.
>>
>>Is such a change acceptable?
> 
> 
> Only with a comment that explains it -- otherwise the next person
> looking at the code will remove it.

Wouldn't the following be faster ?

l = map(chr, range(256))
_idmap = str('').join(l)
del l

> But didn't we at one point conclude that -U was never gonna work?  And
> wasn't that why it's no longer documented?

I don't remember such a conclusion. The reason for having -U
in the first place was to test the Python standard lib for
being able to handle Unicode seemlessly. Obviously we are still
far from having reached that point, but it's a good way of
being able to test the compatibility.

It's no longer documented to reduce the number of bug-reports
we get for it. It was never meant to be used by users (at this
point), just by developers.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From sjoerd@acm.org  Mon Oct 14 08:57:05 2002
From: sjoerd@acm.org (Sjoerd Mullender)
Date: Mon, 14 Oct 2002 09:57:05 +0200
Subject: [Python-Dev] Why does the Windows installer use the shortname formof the install directory?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEPDBKAB.tim.one@comcast.net>
Message-ID: <200210140757.g9E7v5H06545@indus.ins.cwi.nl>

On Fri, Oct 11 2002 Tim Peters wrote:

> [Moore, Paul]
> > ...
> > That's fair enough. And it's not major enough that I'm pushing
> > for anyone to waste time on it. I just thought I'd mention it in
> > case it was a simple change to a checkbox, or something.
> 
> It *almost* is a checkbox!  In WISE's idea of "a scripting language", the
> relevant line looks like this:
> 
>    Set Variable MAINDIR to %MAINDIR%
> 
> There's not a visible clue that this isn't a no-op.  If you double-click on
> the line in the WISE GUI, though, a dialog box pops up, with a dropdown list
> labeled "Operation", one of whose entries is
> 
>     Convert to short filename
> 
> There isn't another variant that adds "but only if it contains a space".
> 
> Beyond that, I'm stuck -- since Adobe graciously updated my Acrobat Reader,
> every attempt to view the WISE reference docs (pdf format) hard-freezes
> (power cycle required) my box.  I *like* your suggestion to skip this unless
> there's an embedded space, I just seem unable to do anything about it now
> (but it would still be for 2.3 -- mucking with 2.2 is too dangerous).

You should be able to put an If Statement / End Statement around it
with the If Statement property box containing If Variable: MAINDIR
Contains The Value: <space>.  I haven't actually tested this, but I
have used this statement with other values.

-- Sjoerd Mullender <sjoerd@acm.org>


From guido@python.org  Mon Oct 14 13:16:30 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 08:16:30 -0400
Subject: [Python-Dev] The -U flag
In-Reply-To: Your message of "Mon, 14 Oct 2002 06:32:14 +0200."
 <m3u1jpy71t.fsf@mira.informatik.hu-berlin.de>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de> <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
 <m3u1jpy71t.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210141216.g9ECGU611229@pcp02138704pcs.reston01.va.comcast.net>

> > But didn't we at one point conclude that -U was never gonna work?  
> 
> I'm not aware of a formal decision about that. I readily admit that it
> wont' work in any foreseeable future, but I haven't given up for P3k :-)

Maybe MAL remembers after seeing this checkin comment to Modules/main.c:

----------------------------
revision 1.63
date: 2002/02/11 18:46:47;  author: lemburg;  state: Exp;  lines: +0 -1
Remove mentioning of -U option in "python -h" output.
----------------------------

> This time, my interest was triggered by someone on c.l.p who could not
> reverse a string, since it was in UTF-8; he was asking why Python is
> so difficult for Swedish people.

Yeah, he mailed me first -- I don't understand why his default
encoding isn'y Latin-1 though.

> > And wasn't that why it's no longer documented?
> 
> Yes: because it won't work in any foreseeable future.
> 
> I'll take your statement that "small" changes towards making -U work
> are acceptable; I'll see how far I get with small changes until I get
> stuck again.

Sure.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct 14 13:23:06 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 08:23:06 -0400
Subject: [Python-Dev] Missing symbol in sre.py (Python 2.2.2b1)
In-Reply-To: Your message of "Mon, 14 Oct 2002 06:50:07 +0200."
 <20478.1034571007@www43.gmx.net>
References: <20478.1034571007@www43.gmx.net>
Message-ID: <200210141223.g9ECN6W11276@pcp02138704pcs.reston01.va.comcast.net>

> There appears to still be a missing public symbol definition in the Python 
> 2.2.2b1 
> version of sre.py 
>  
> The documentation implies that finditer should work and indeed it is 
> defined 
> in the module if sys.hexversion > 0x02020000 however it is not listed in 
> the 
> __all__ module variable. 
>  
> This means that if I use 
>         import re 
> I am unable to use re.finditer but if I use 
>         import sre 
> then I can use sre.finditer. 
>  
> I thought that the recommended approach was to use import re. 
>  
> I think this is  the outstanding bug 585882 re.finditer which was for 
> Python 2.2.1. 

Thanks!  I fixed this just in time for the release.  I don't know
what Fredrik is doing these days, but fixing bugs in RE doesn't seem
to be a high priority for him... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Oct 14 13:39:10 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 08:39:10 -0400
Subject: [Python-Dev] Python 2.2.2 release tonight - checkin freeze on branch
Message-ID: <200210141239.g9ECdAC11799@pcp02138704pcs.reston01.va.comcast.net>

I'd like to do the 2.2.2 release tonight, unless Jack has any
unresolved problems.  (Jack, mail me ASAP if you do!)

Please, no more checkins to the release22-maint branch until the
release is done (with the exception of the Mac subtree, and Fred &
Tim).

I still have to update the Misc/NEWS file.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@cwi.nl  Mon Oct 14 14:07:34 2002
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Mon, 14 Oct 2002 15:07:34 +0200
Subject: [Python-Dev] Python 2.2.2 release tonight - checkin freeze on branch
In-Reply-To: <200210141239.g9ECdAC11799@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <E7B0BBAE-DF75-11D6-932A-0030655234CE@cwi.nl>

On Monday, Oct 14, 2002, at 14:39 Europe/Amsterdam, Guido van Rossum 
wrote:

> I'd like to do the 2.2.2 release tonight, unless Jack has any
> unresolved problems.  (Jack, mail me ASAP if you do!)

There's fix for test_email (skip it for MacPython) that's still sitting 
on my disk at home, but I'd say don't let that stop you.

And it may be a while before I can actually release MacPython 2.2.2, 
because I'm still waiting for a new freeware license for the VISE 
installer.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From pinard@iro.umontreal.ca  Mon Oct 14 14:45:48 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: Mon, 14 Oct 2002 09:45:48 -0400
Subject: [Python-Dev] Re: Assign to errno allowed?
In-Reply-To: <200209260059.g8Q0xmL06185@oma.cosc.canterbury.ac.nz> (Greg
 Ewing's message of "Thu, 26 Sep 2002 12:59:48 +1200 (NZST)")
References: <200209260059.g8Q0xmL06185@oma.cosc.canterbury.ac.nz>
Message-ID: <oq3cr9m8vn.fsf@titan.progiciels-bpi.ca>

[Greg Ewing]

>> #ifdef ...WINCE...
>             ^^^^^
>
> I wonder if Microsoft foresaw that abbreviation when
> they chose the name CE...

Speaking of, for many GNU projects, we needed a pre-processor symbol to
represent DJGPP and other ports to MS-DOS and Windows, so someone (most
probably me) took `DOS' and `WIN' and concatenated them, yielding:

    #if DOSWIN
     ...

It went for a few years.  One day, we were asked by the FSF to avoid this
symbol, as it suggested that DOS may win in some way.  At the time, this
surely meant a lot of places to change, I do not remember if we complied.
The fact is that I never had such a pun in head when I made the symbol...

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard


From nas@python.ca  Mon Oct 14 15:05:03 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 14 Oct 2002 07:05:03 -0700
Subject: [Python-Dev] Python-dev Summary for 2002-10-01 through 2002-1 0-13
In-Reply-To: <Pine.SOL.4.44.0210131753580.20875-100000@death.OCF.Berkeley.EDU>
References: <B43D149A9AB2D411971300B0D03D7E8BF0A667@natasha.auslabs.avaya.com> <Pine.SOL.4.44.0210131753580.20875-100000@death.OCF.Berkeley.EDU>
Message-ID: <20021014140503.GA27628@glacier.arctrix.com>

Brett Cannon wrote:
> I could do that, but I have two concerns.  One is the extra amount of time
> it woulkd take to do links for every long link (tend to be a decent
> amount), and there is no guarantee that tinyurl will be around and thus
> make the archived links break.

There's no reason we couldn't implement the same service on python.org.

  Neil


From guido@python.org  Mon Oct 14 15:28:49 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 10:28:49 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: Your message of "Mon, 14 Oct 2002 00:23:34 +0200."
 <3DA9F266.C87E5A58@ulb.ac.be>
References: <LNBBLJKPBEHFEDALKOLCEECPBLAB.tim.one@comcast.net>
 <3DA9F266.C87E5A58@ulb.ac.be>
Message-ID: <200210141428.g9EESnv12174@pcp02138704pcs.reston01.va.comcast.net>

I'd like to add that, until we decide on a Really Big Rewrite, the
length of all Python sequences must fit in a signed int.  So relying
on malloc returning NULL isn't really a viable approach: even if you
have, say, 2.5 GB of memory in your box, you still can't create a
string of that size.

I do agree that the distinction between OverflowError and MemoryError
is not worth splitting hairs over.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Mon Oct 14 18:34:15 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 14 Oct 2002 19:34:15 +0200
Subject: [Python-Dev] Missing symbol in sre.py (Python 2.2.2b1)
References: <20478.1034571007@www43.gmx.net>  <200210141223.g9ECN6W11276@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <01fd01c273a8$93561f30$ced241d5@hagrid>

guido wrote:

> I don't know what Fredrik is doing these days, but fixing bugs in RE
> doesn't seem to be a high priority for him... :-(

sorry, life got in the way.

and I have to admit that hacking on RE hasn't resulted in enough
sex, praise or money [1] lately to keep me motivated.  hopefully,
that'll change in time for 2.3...

cheers /F

1) http://blogs.salon.com/0001111/2002/09/21.html#a475



From martin@v.loewis.de  Mon Oct 14 19:05:57 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Oct 2002 20:05:57 +0200
Subject: [Python-Dev] The -U flag
In-Reply-To: <200210141216.g9ECGU611229@pcp02138704pcs.reston01.va.comcast.net>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>
 <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
 <m3u1jpy71t.fsf@mira.informatik.hu-berlin.de>
 <200210141216.g9ECGU611229@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m33cr8dhfe.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Yeah, he mailed me first -- I don't understand why his default
> encoding isn'y Latin-1 though.

He uses an sv_SE.UTF-8 locale, which appears to be the default on
Redhat 8, if you ask for a Swedish environment.

IMO, UTF-8 locales will replace everything else on Linux in the long
run. Otherwise, system-wide files (and file names) will be restricted
to ASCII forever, and people will not accept not having accented and
CJK characters in /etc/passwd, for their real names. I expect other
Unices to take a similar direction.

Regards,
Martin


From guido@python.org  Mon Oct 14 19:10:35 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 14:10:35 -0400
Subject: [Python-Dev] The -U flag
In-Reply-To: Your message of "Mon, 14 Oct 2002 20:05:57 +0200."
 <m33cr8dhfe.fsf@mira.informatik.hu-berlin.de>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de> <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net> <m3u1jpy71t.fsf@mira.informatik.hu-berlin.de> <200210141216.g9ECGU611229@pcp02138704pcs.reston01.va.comcast.net>
 <m33cr8dhfe.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210141810.g9EIAZ124143@pcp02138704pcs.reston01.va.comcast.net>

> IMO, UTF-8 locales will replace everything else on Linux in the long
> run. Otherwise, system-wide files (and file names) will be restricted
> to ASCII forever, and people will not accept not having accented and
> CJK characters in /etc/passwd, for their real names. I expect other
> Unices to take a similar direction.

We'll be in for lots of niggling little bug reports then, as RH 8.0
conquers the world... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Oct 14 19:20:05 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 14 Oct 2002 14:20:05 -0400
Subject: [Python-Dev] The -U flag
In-Reply-To: <200210141216.g9ECGU611229@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEIGBLAB.tim.one@comcast.net>

> Maybe MAL remembers after seeing this checkin comment to Modules/main.c:
>
> ----------------------------
> revision 1.63
> date: 2002/02/11 18:46:47;  author: lemburg;  state: Exp;  lines: +0 -1
> Remove mentioning of -U option in "python -h" output.
> ----------------------------

MAL did that after I whined a lot:  so long as -U was mentioned in the -h
blurb, users kept trying to use it -- that it didn't work, and spectacularly
so, gave them a wrong impression about Python, and wasted time on various
mailing lists explaining over & over that it wasn't "supposed to" work.



From martin@v.loewis.de  Mon Oct 14 21:04:26 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Oct 2002 22:04:26 +0200
Subject: [Python-Dev] The -U flag
In-Reply-To: <3DAA7422.8000703@lemburg.com>
References: <200210131948.g9DJm8E2005271@mira.informatik.hu-berlin.de>
 <200210132348.g9DNmXH09873@pcp02138704pcs.reston01.va.comcast.net>
 <3DAA7422.8000703@lemburg.com>
Message-ID: <m3bs5wss6t.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Wouldn't the following be faster ?
> 
> l = map(chr, range(256))
> _idmap = str('').join(l)
> del l

It is, by a factor of roughly 3 (on my system). Committed as posted.

Regards,
Martin


From BPettersen@NAREX.com  Mon Oct 14 23:30:58 2002
From: BPettersen@NAREX.com (Bjorn Pettersen)
Date: Mon, 14 Oct 2002 16:30:58 -0600
Subject: [Python-Dev] Python 2.2.2 release tonight - checkin freeze on branch
Message-ID: <60FB8BB7F0EFC7409B75EEEC13E2019201773D50@admin56.narex.com>

> On Monday, Oct 14, 2002, at 14:39 Europe/Amsterdam, Guido van Rossum=20
> wrote:
>=20
> > I'd like to do the 2.2.2 release tonight, unless Jack has any=20
> > unresolved problems.  (Jack, mail me ASAP if you do!)

I realize this is a little late, but I was wondering if 2.2.2 will
support VC7 (important for us Python-in-a-tie people ;-)  I searched the
mailing list, but the only thing I found were messages that it was fixed
on the 2.3 branch...

-- bjorn


From guido@python.org  Mon Oct 14 23:38:34 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 18:38:34 -0400
Subject: [Python-Dev] Python 2.2.2 release tonight - checkin freeze on branch
In-Reply-To: Your message of "Mon, 14 Oct 2002 16:30:58 MDT."
 <60FB8BB7F0EFC7409B75EEEC13E2019201773D50@admin56.narex.com>
References: <60FB8BB7F0EFC7409B75EEEC13E2019201773D50@admin56.narex.com>
Message-ID: <200210142238.g9EMcZB01255@pcp02138704pcs.reston01.va.comcast.net>

> I realize this is a little late, but I was wondering if 2.2.2 will
> support VC7 (important for us Python-in-a-tie people ;-) I searched
> the mailing list, but the only thing I found were messages that it
> was fixed on the 2.3 branch...

By VC7, do you refer to Microsoft Visual C++ 7.0?  I don't have a
copy, and neither does Tim.  Maybe the PBF can donate one?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Mon Oct 14 23:44:31 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 14 Oct 2002 17:44:31 -0500
Subject: [Python-Dev] Can Zope be built on MacOSX?
Message-ID: <15787.18639.177235.144826@montanaro.dyndns.org>

I downloaded Zope 2.6.0b1 today from zope.org and tried to build it with
Python from CVS, which failed, then Python 2.2.2 (also from CVS, but which
should be between 2.2.2b1 and 2.2.2final), which also failed.  Upon closer
examination, it seems 2.1.x is required for Zope 2.5, though the
doc/INSTALL.txt file says nothing about what Zope 2.6 requires.  I then
pulled the release21-maint branch of CVS and tried to build it on MacOSX.
It failed spectacularly trying to compile Python/dynload_next.c

Am I wasting my time trying to build Zope on MacOSX?

Skip


From lalo@laranja.org  Mon Oct 14 23:47:46 2002
From: lalo@laranja.org (Lalo Martins)
Date: Mon, 14 Oct 2002 20:47:46 -0200
Subject: [Python-Dev] Can Zope be built on MacOSX?
In-Reply-To: <15787.18639.177235.144826@montanaro.dyndns.org>
References: <15787.18639.177235.144826@montanaro.dyndns.org>
Message-ID: <20021014224746.GQ10221@laranja.org>

On Mon, Oct 14, 2002 at 05:44:31PM -0500, Skip Montanaro wrote:
> 
> I downloaded Zope 2.6.0b1 today from zope.org and tried to build it with
> Python from CVS, which failed, then Python 2.2.2 (also from CVS, but which
> should be between 2.2.2b1 and 2.2.2final), which also failed.  Upon closer
> examination, it seems 2.1.x is required for Zope 2.5, though the
> doc/INSTALL.txt file says nothing about what Zope 2.6 requires.  I then
> pulled the release21-maint branch of CVS and tried to build it on MacOSX.
> It failed spectacularly trying to compile Python/dynload_next.c
> 
> Am I wasting my time trying to build Zope on MacOSX?

This is not the correct list for this question, but then again I think you
know that quite well ;-)

Zope requires 2.1, this includes Zope 2.6. The first Python2.2 version is
scheduled to be 2.7.

I don't have OSX, but I heard about some difficulties. My coworker however
successfully compiled 2.1.3 (from the source release, not CVS) and Z2.6 from
CVS and reports it to work without a problem. He recommends latest gcc3.

[]s,
                                               |alo
                                               +----
--
            Those who trade freedom for security
               lose both and deserve neither.
--
http://www.laranja.org/                mailto:lalo@laranja.org
         pgp key: http://www.laranja.org/pessoal/pgp

Eu jogo RPG! (I play RPG)         http://www.eujogorpg.com.br/
Python Foundry Guide http://www.sf.net/foundry/python-foundry/


From guido@python.org  Mon Oct 14 23:56:12 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 18:56:12 -0400
Subject: [Python-Dev] Can Zope be built on MacOSX?
In-Reply-To: Your message of "Mon, 14 Oct 2002 17:44:31 CDT."
 <15787.18639.177235.144826@montanaro.dyndns.org>
References: <15787.18639.177235.144826@montanaro.dyndns.org>
Message-ID: <200210142256.g9EMuC208037@pcp02138704pcs.reston01.va.comcast.net>

[Skip]
> I downloaded Zope 2.6.0b1 today from zope.org and tried to build it
> with Python from CVS, which failed, then Python 2.2.2 (also from
> CVS, but which should be between 2.2.2b1 and 2.2.2final), which also
> failed.  Upon closer examination, it seems 2.1.x is required for
> Zope 2.5, though the doc/INSTALL.txt file says nothing about what
> Zope 2.6 requires.  I then pulled the release21-maint branch of CVS
> and tried to build it on MacOSX.  It failed spectacularly trying to
> compile Python/dynload_next.c

I'm hardly an expert on building Zope on MacOSX, but it's my
understanding that Zope 2.6 ought to work with Python 2.2.2 (as well
as 2.2.2b1) even though this is not officially supported.  Ditto for
CVS trunk ("Python 2.3").

I can also believe that Python 2.1.3 doesn't build on MacOSX -- MacOSX
simply didn't exist when we released 2.1.  AFAIK Jack hasn't really
backported all the work he did for MacOS to 2.1 -- and I doubt he ever
will.

> Am I wasting my time trying to build Zope on MacOSX?

I wouldn't say that.

What failures did you see with ~2.2.2?

(Maybe take this offline or to a Zope list?)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct 15 00:29:33 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 14 Oct 2002 19:29:33 -0400
Subject: [Python-Dev] RELEASED: Python 2.2.2 (final)
Message-ID: <200210142329.g9ENTXH13982@pcp02138704pcs.reston01.va.comcast.net>

Ta-daa!

The release of Python 2.2.2 is here.  This is a fully backwards
compatible bugfix release for Python 2.2.1.

Thanks to all who submitted last-minute bug reports and fixes -- and
thanks even more to those who didn't wait until the last minute! :-)

Python 2.2.2 has a large number of bug fixes in all areas of the
interpreter and the library; for a full account see the release notes,
available via the link below.  With special dispensation from the
"bugfixes only" rule, a brand new version of the email package is
also included: email 2.4.3.

The new release is available here:

    http://www.python.org/2.2.2/

It will soon be available from mirror sites all over the world -- for
a speedy download, try the mirror site nearest to you first:

    http://www.python.org/Mirrors.html

As usual, a Windows installer and a Unix/Linux source tarball are made
available, as well as tarballs of the documentation in various forms.
At the moment, no Mac version is available, though I expect that Jack
Jansen will produce one soon.  Likewise, no Linux RPMs are available
yet; I expect that Sean Reifschneider will prepare these shortly.

I'm especially grateful to Michael Hudson, Raymond Hettinger and Neal
Norwitz for their hard work on backporting the large volume of patches
from Python 2.3.  Without them this release would not have been
possible.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Oct 15 02:40:34 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 14 Oct 2002 21:40:34 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <3DA9F266.C87E5A58@ulb.ac.be>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEKDBLAB.tim.one@comcast.net>

[Armin Rigo]
> Let me quote from stringobject.c the "heartbreaking code such as
> string_repeat's":
>
>         nbytes = size * sizeof(char);
>         if (nbytes / sizeof(char) != (size_t)size ||
>             nbytes + sizeof(PyStringObject) <= nbytes)
>                 <OverflowError>
>         op = (PyStringObject *)
>                 PyObject_MALLOC(sizeof(PyStringObject) + nbytes);
>         if (op == NULL)
>                 <MemoryError>
>
> With the proper macro, the whole code above can be written as:
>
>         nbytes = Py_MUL_ADD(size, sizeof(char), sizeof(PyStringObject));
>         op = (PyStringObject *) PyObject_MALLOC(nbytes);
>         if (op == NULL)
>                 <MemoryError>
>
> which seems much cleaner to me.

It's not the appearance of the code that breaks my heart (although it
doesn't cheer it either <wink>), it's that when we "fix" one of these things
that pops up in real life, we add a pile of expensive tests and branches to
code that's been known to fall over only once in 10 years.  So Python gets
bigger and slower for everyone, and half the time there's no real cause
beyond that some wiener is playing "let's try to break it!".

> For this reason I do not believe it is a good idea here to define a
> macro a la Zope C:
>
>     DO_MULADD_OR(nbytes, size, sizeof(char), sizeof(PyStringObject),
> goto Overflow);
>
> for two reasons: there is no real need for a special "Overflow" case,

I don't know.  It depends on context, such as how many times a potential
overflow can occur within a single routine.  Probably not often on average.

> as long as the returned 'nbytes' is guaranteed to be un-malloc-able;

There is no nbytes that's guaranteed to bu un-malloc-able, unless you're
going to impose artificial restrictions on malloc.  It's not a happy thing
to burden our malloc wrappers with another layer of test-and-branch either.

> besides, no existing macro (e.g. _PyObject_VAR_SIZE) can support an
> extra "goto Overflow" case without breaking C code that depends on it.

Cool:  yet another reason to leave things alone <wink>.

> There is however a difference between the two above code fragments, in
> that the former will raise an OverflowError for "really too large
> values" and a MemoryError for "reasonable value but still too large for
> me to allocate right now".  Whether this is a correct behavior is
> debatable.

Some things aren't worth debating, though -- it doesn't matter to me which
exception we raise.

> I would argue that raising MemoryError in both cases is fine:

Fine by me too.

> after all, "123"*9999999999999999 could be perfectly fine in another
> Python implementation with lazy strings.  It is thus only a
> memory limit that makes this fail in the current implementation.
>
> Note that if we define MAX_SIZE to ((size_t)-1), the overflow check in
> the present stringobject.c could be more efficiently written down as:

(size_t)~0UL is safer, although there's really no wholly safe way to spell
the intent in C89.

>         if (size > (MAX_SIZE - sizeof(PyStringObject)) / sizeof(char))
>                 <OverflowError>
>
> where everything at the right of '>' reduces to a compile-time
> constant.  The Py_MUL_ADD macro can be implemented with this idea in
> mind, which works best if the 2nd and 3rd arguments are constants.

>> note that anything/non_constant can be incredibly expensive!

> Yes.  Couldn't a better overflow-checking algorithm be designed for
> these cases?  For example, to multiply the non-constants 'a' and 'b', we
> can be sure that there is no overflow if both 'a' and 'b' are small
> enough (which is the common case anyway).  I believe I can even think
> about a single Py_MUL_ADD macro that is as efficient as above with
> constant arguments, and otherwise uses this trick.

And more tests, and more branches.  Life will be much easier if we can limit
it to size_t thingies (which have the same size, and are unsigned0.

> About using 64-bit computations everywhere (as suggested by Christian):
> if the length of objects (e.g. lists) is still constrained to 31 bits,
> then it's fine; otherwise, the problem is the same.  Do all supported
> 64-bit platforms have a 32-bit 'int', and thus a 31-bit limit on object
> lengths?

No.  For example, some now-rare models of Cray boxes have no 32-bit types.
If 64-bit machines really are the wave of the future (I started using them
in the late 1970's, and am still waiting for the revolution <0.9 wink>),
it's no cure anyway, as Python in a truly 64-bit world will have to lose its
2GB-bounded of the world (as it's already painfully doing for files, and
maybe for mmaps).  Then the "detect overflow" problem returns with a
vengeance.

> By the way, this makes me think of a good efficient overflow-detection
> method for platforms with a "long long" type longer than size_t:
>
>   unsigned long long bytes = ((unsigned long long)a)*((unsigned long
> long)b)+c;
>   if (bytes > ((size_t)-1))
>     <overflow>;

This also implicitly assumes that:

1. size_t is no wider than an int; that's the kind of thing the
   "(size_t)~0UL would be safer" comment above was getting at.

2. long long is at least twice as wide as size_t.

> This compiles to very good assembly code on x86.
>
> Mmmh, in summary I would recommend that we define a common
> "multiply-and-add" macro, returning a size_t or OVERFLOW_VALUE in case
> of overflow, which is implemented in several different ways on different
> platforms, as selected by #if's.  The same macro is used for just
> "multiply" or just "add" by relying on the compiler's strengh reduction.

What a PITA.  Is it worth it?  If I don't have to do it, maybe <wink>.

BTW, when we allocate an object that participates in cyclic GC, there's Yet
Another little integer that gets added to the total size (to reflect the
"extra header" bytes unique to GC'ed objects), and so there's Yet Another
place potential overflow may go uncaught now.



From tim.one@comcast.net  Tue Oct 15 02:43:55 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 14 Oct 2002 21:43:55 -0400
Subject: [Python-Dev] Missing symbol in sre.py (Python 2.2.2b1)
In-Reply-To: <01fd01c273a8$93561f30$ced241d5@hagrid>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEKDBLAB.tim.one@comcast.net>

[Fredrik Lundh]
> sorry, life got in the way.
>
> and I have to admit that hacking on RE hasn't resulted in enough
> sex, praise or money [1] lately to keep me motivated.  hopefully,
> that'll change in time for 2.3...
>
> cheers /F
>
> 1) http://blogs.salon.com/0001111/2002/09/21.html#a475

Here you go, /F:  real, actual praise -- just like the fellow asked for.
What more could you want?  Swedes give all their money to the government,
and working on RE is plain incompatible with having orgasms <wink>.



From guido@python.org  Tue Oct 15 05:24:56 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 15 Oct 2002 00:24:56 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: Your message of "Mon, 14 Oct 2002 21:40:34 EDT."
 <LNBBLJKPBEHFEDALKOLCAEKDBLAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEKDBLAB.tim.one@comcast.net>
Message-ID: <200210150424.g9F4Ous17546@pcp02138704pcs.reston01.va.comcast.net>

I think this discussion is suffering from the desire to
over-generalize.

Armin, can you show us how to multiply two nonnegative ints together
with an overflow "block" as Tim suggested earlier?  I.e. a macro
callable as

  SAFE_MULTIPLY(destination, src1, src2, on_overflow);

meaning roughly

  destination = src1 * src2;
  if (<overflow detected>)
     on_overflow;

I currently use something like this as the overflow test in a few places:

  if (src1 && destination / src1 != src2)

but as Tim points out the division can kill us unless src1 is a
constant...  How would you spell this without doing a division?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Oct 15 06:29:36 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 15 Oct 2002 01:29:36 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <200210150424.g9F4Ous17546@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>

[Guido]
> ...
> I currently use something like this as the overflow test in a few places:
>
>   if (src1 && destination / src1 != src2)
>
> but as Tim points out the division can kill us unless src1 is a
> constant...  How would you spell this without doing a division?

Armin answered that in the case there's an efficient "unsigned long long"
type at least twice the width of the inputs.  Then you just do the product
in the fat precision and see whether any of the upper-half bits are set.

As a pragmatic matter, 2/5ths of that trick can work "most of the time":

#define YOWZA (sizeof(size_t) * 4U)

    if ((src1 | src2} >> YOWZA)
        hard to tell
    else
        overflow is impossible

would get out quick in *most* non-overflow cases.  The fast path there
succeeds if and only if both inputs are at most "half full".  For example,
in

    string * int

it's rare that len(string) >= 64K or int >= 64K.

For what to do in the other half, see the history of subtly buggy attempts
to detect overflow quickly in Python's int * int code.  Without a
double-width type to fall back on, I think it's impossible to do this both
correctly and quickly.

BTW, if we know we're not interested in anything other than 32-bit products,

    double _prod = (double)src1 * (double)src2;
    if (_prod < 4294967296.0) /* 2**32 */
        result = (size_t)_prod;
    else
        overflow;

does the trick reliably on all platforms I know of -- although not all
Python platforms support floating-point efficiently!

It's a good puzzle <wink>.



From thomas.heller@ion-tof.com  Tue Oct 15 11:02:56 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 15 Oct 2002 12:02:56 +0200
Subject: [Python-Dev] tempfile problems
Message-ID: <3cr8j9yn.fsf@ion-tof.com>

bdist_wininst currently contains this code:

    from tempfile import NamedTemporaryFile
    arc = NamedTemporaryFile(".zip")
    archive_basename = arc.name[:-4]
    fullname = self.distribution.get_fullname()
    arcname = self.make_archive(archive_basename, "zip",
                                root_dir=self.bdist_dir)

The corresponding checkin message is this:

revision 1.35
date: 2002/08/09 16:37:34;  author: gvanrossum;  state: Exp;  lines: +5 -4
Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Weinberg).  This changes all uses of deprecated tempfile functions to
the recommended ones.


The call to NamedTemporaryFile(".zip") crashes with an access
violation on Windows. I've submitted a bug # 623464 for this.
I assume this is meant instead:

    from tempfile import NamedTemporaryFile
    arc = NamedTemporaryFile(suffix = ".zip")
    archive_basename = arc.name[:-4]
    fullname = self.distribution.get_fullname()
    arcname = self.make_archive(archive_basename, "zip",
                                root_dir=self.bdist_dir)

but it doesn't work either: make_archive() fails with
'permission denied' because the file is already open.
This would work:

    from tempfile import NamedTemporaryFile
    arc = NamedTemporaryFile(suffix = ".zip")
    archive_basename = arc.name[:-4]
    arc.close()
    fullname = self.distribution.get_fullname()
    arcname = self.make_archive(archive_basename, "zip",
                                root_dir=self.bdist_dir)

but I'm not sure if it defeats the purpose of the change
(secure tempfiles).

Since distutils should still be compatible with older
Python versions, and those do not contain NamedTemporaryFile,
I suggest to back out this change.

Thomas



From mal@lemburg.com  Tue Oct 15 11:09:55 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 15 Oct 2002 12:09:55 +0200
Subject: [Python-Dev] tempfile problems
References: <3cr8j9yn.fsf@ion-tof.com>
Message-ID: <3DABE973.2080601@lemburg.com>

Thomas Heller wrote:
> bdist_wininst currently contains this code:
> 
>     from tempfile import NamedTemporaryFile
>     arc = NamedTemporaryFile(".zip")
>     archive_basename = arc.name[:-4]
>     fullname = self.distribution.get_fullname()
>     arcname = self.make_archive(archive_basename, "zip",
>                                 root_dir=self.bdist_dir)
> 
> The corresponding checkin message is this:
> 
> revision 1.35
> date: 2002/08/09 16:37:34;  author: gvanrossum;  state: Exp;  lines: +5 -4
> Massive changes from SF 589982 (tempfile.py rewrite, by Zack
> Weinberg).  This changes all uses of deprecated tempfile functions to
> the recommended ones.
> 
> 
> The call to NamedTemporaryFile(".zip") crashes with an access
> violation on Windows. I've submitted a bug # 623464 for this.
> I assume this is meant instead:
> 
>     from tempfile import NamedTemporaryFile
>     arc = NamedTemporaryFile(suffix = ".zip")
>     archive_basename = arc.name[:-4]
>     fullname = self.distribution.get_fullname()
>     arcname = self.make_archive(archive_basename, "zip",
>                                 root_dir=self.bdist_dir)
> 
> but it doesn't work either: make_archive() fails with
> 'permission denied' because the file is already open.
> This would work:
> 
>     from tempfile import NamedTemporaryFile
>     arc = NamedTemporaryFile(suffix = ".zip")
>     archive_basename = arc.name[:-4]
>     arc.close()
>     fullname = self.distribution.get_fullname()
>     arcname = self.make_archive(archive_basename, "zip",
>                                 root_dir=self.bdist_dir)
> 
> but I'm not sure if it defeats the purpose of the change
> (secure tempfiles).
> 
> Since distutils should still be compatible with older
> Python versions, and those do not contain NamedTemporaryFile,
> I suggest to back out this change.

+1

A temporary ZIP file doesn't introduce much of a security
risk anyway, even less when it's being created by a package
developer.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From Paolo Invernizzi <paoloinvernizzi@dmsware.com>  Tue Oct 15 11:58:04 2002
From: Paolo Invernizzi <paoloinvernizzi@dmsware.com> (Paolo Invernizzi)
Date: Tue, 15 Oct 2002 12:58:04 +0200
Subject: Re[2]: [Python-Dev] Python 2.2.2 release tonight - checkin freeze on branch
In-Reply-To: <60FB8BB7F0EFC7409B75EEEC13E2019201773D50@admin56.narex.com>
References: <60FB8BB7F0EFC7409B75EEEC13E2019201773D50@admin56.narex.com>
Message-ID: <13914848100.20021015125804@dmsware.com>

BP> I realize this is a little late, but I was wondering if 2.2.2 will
BP> support VC7 (important for us Python-in-a-tie people ;-)

I've just tried...
Python 2.2.2 does not compile with Visual Studio NET out-of-the-box...
It complains about missing largeint.h, so the fixes Mark (was Mark?) has done on
2.3 weren't backported to 2.2.2.

I haven't looked at it, but think that also the DLL_EXPORT/IMPORT
stuff is still broken on 2.2.2

-- 
Best regards,
Paolo



From xscottg@yahoo.com  Tue Oct 15 18:24:09 2002
From: xscottg@yahoo.com (Scott Gilbert)
Date: Tue, 15 Oct 2002 10:24:09 -0700 (PDT)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>
Message-ID: <20021015172409.50190.qmail@web40110.mail.yahoo.com>

The following is really ugly (until you put it in a macro), but would work:

#define HALF_BITS (sizeof(size_t) * 4U)
#define LO_MASK ((((size_t)1) << HALF_BITS)-1)
#define HI_MASK (LO_MASK<<HALF_BITS)

  if (
      /* Tim's fast path */
      ((src1 | src2) & HI_MASK) && (
      /* Ugly slower path */
      (((src1&LO_MASK) * (src2>>HALF_BITS)) & HI_MASK) |
      (((src1>>HALF_BITS) * (src2&LO_MASK)) & HI_MASK) |
      (src1>>HALF_BITS) * (src2>>HALF_BITS))
  ) {
     goto overflow;
  }


Of course it's hoping for some common subexpression elimination from the
compiler.








__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com


From jason@tishler.net  Tue Oct 15 18:51:32 2002
From: jason@tishler.net (Jason Tishler)
Date: Tue, 15 Oct 2002 13:51:32 -0400
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: <200210011519.g91FJwI21585@odiug.zope.com>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
 <200210011338.g91Dclo08817@odiug.zope.com>
 <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>
 <200210011519.g91FJwI21585@odiug.zope.com>
Message-ID: <20021015175131.GB1536@tishler.net>

Martin,

On Tue, Oct 01, 2002 at 11:19:58AM -0400, Guido van Rossum wrote:
> > > If it's more work to keep support around than to drop it, sure.
> > 
> > I'm planning a number of changes which can be simplified if I don't
> > have that many versions to support.
> 
> OK, +1.

This is probably too little, too late, but Cygwin is still stuck at Tcl
8.0.  Would you be willing to keep this support in?

Note that I've tried to encourage Cygwin to upgrade to a more recent
version:

    http://cygwin.com/ml/cygwin/2002-10/msg00798.html

Thanks,
Jason


From jason@tishler.net  Tue Oct 15 19:02:33 2002
From: jason@tishler.net (Jason Tishler)
Date: Tue, 15 Oct 2002 14:02:33 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <200210050018.g950IFO32089@pcp02138704pcs.reston01.va.comcast.net>
References: <200210041940.g94Jeph29549@pcp02138704pcs.reston01.va.comcast.net>
 <LCEPIIGDJPKCOIHOBJEPIEMEGLAA.mhammond@skippinet.com.au>
 <20021004235039.GA97477@gargamel.ghaering.test>
 <200210050018.g950IFO32089@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021015180232.GC1536@tishler.net>

On Fri, Oct 04, 2002 at 08:18:15PM -0400, Guido van Rossum wrote:
> We've already absorbed a sheer endless set of patches to make it work
> on CYGWIN.

Ouch! :,)

Nevertheless, the Cygwin community appreciates the continued support.

Jason


From tim.one@comcast.net  Tue Oct 15 21:36:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 15 Oct 2002 16:36:19 -0400
Subject: [Python-Dev] [win32] Killing MSVC's _alloca
In-Reply-To: <20021015180232.GC1536@tishler.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHIEAIDMAA.tim.one@comcast.net>

[Guido]
> We've already absorbed a sheer endless set of patches to make it work
> on CYGWIN.

[Jason Tishler]
> Ouch! :,)
>
> Nevertheless, the Cygwin community appreciates the continued support.

The Python community appreciates your support too, Jason!  Guido wasn't
being pejorative there, just objectively accurate:  "sheer endless" is Dutch
for "about 12" <wink>.



From martin@v.loewis.de  Tue Oct 15 21:57:58 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Oct 2002 22:57:58 +0200
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: <20021015175131.GB1536@tishler.net>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
 <200210011338.g91Dclo08817@odiug.zope.com>
 <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>
 <200210011519.g91FJwI21585@odiug.zope.com>
 <20021015175131.GB1536@tishler.net>
Message-ID: <m3ptubbesp.fsf@mira.informatik.hu-berlin.de>

Jason Tishler <jason@tishler.net> writes:

> This is probably too little, too late, but Cygwin is still stuck at Tcl
> 8.0.  Would you be willing to keep this support in?

No, I've already made the changes to drop this support, and added
support for UCS-4 mode in _tkinter.

> Note that I've tried to encourage Cygwin to upgrade to a more recent
> version:
> 
>     http://cygwin.com/ml/cygwin/2002-10/msg00798.html

That seems the right way to go. They may have changed Cygwin by the
time Python 2.3 is released. If not, users wishing to use Cygwin
_tkinter can use Mumit Khan's Tcl 8.3/8.4 ports:

ftp://ftp.nanotech.wisc.edu/pub/khan/tcl/tcltk-8.3.4-cygwin/

Regards,
Martin



From tim.one@comcast.net  Tue Oct 15 23:25:27 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 15 Oct 2002 18:25:27 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <20021015172409.50190.qmail@web40110.mail.yahoo.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEANDMAA.tim.one@comcast.net>

[Scott Gilbert]
> The following is really ugly (until you put it in a macro), but
> would work:
>
> #define HALF_BITS (sizeof(size_t) * 4U)
> #define LO_MASK ((((size_t)1) << HALF_BITS)-1)
> #define HI_MASK (LO_MASK<<HALF_BITS)
>
>   if (
>       /* Tim's fast path */
>       ((src1 | src2) & HI_MASK) && (
>       /* Ugly slower path */
>       (((src1&LO_MASK) * (src2>>HALF_BITS)) & HI_MASK) |
>       (((src1>>HALF_BITS) * (src2&LO_MASK)) & HI_MASK) |
>       (src1>>HALF_BITS) * (src2>>HALF_BITS))
>   ) {
>      goto overflow;
>   }

I'll repeat that it's hard to do this correctly and quickly (c.f. earlier
comments about the history of Python's subtly buggy attempts).

This kind of approach is viewing src1 and src2 as 2-digit numbers in base X
= 2**HALF_BITS:

    src1 = A*X + B
    src2 = C*X + D

then viewing their product as

highpart:      A*C*X**2 +
               xxxxxxxx
middlepart1:          A*D*X +
                   xxxxxxxx
middlepart2:          B*C*X +
                   xxxxxxxx
lowpart:                    B*D
                       xxxxxxxx
               ----------------
               xxxxxxxxxxxxxxxx
               <danger>< safe >

Your second line checks whether the high half of middlepart2 is 0; your
third line whether the high half of middlepart1 is 0; your last line whether
highpart is 0.  But they can all be 0 and *still* have overflow:  the sum

    low half of middlepart1 +
    low half of middlepart2 +
    high half of lowpart

can carry over into the <danger> zone.  Supposing we were looking at 16-bit
products, here's an example:

        0x01ff *
        0x00ff
        ------
      0000
        00ff
        0000
          fe01
      --------
      0001fd01
         ^ oops

Hint:  before posting something like this, prototype it in Python, and do an
exhaustive check of all possible pairs of multiplicands in an artificially
small base.  Very few buggy algorithms survive that, even for very small
artificial bases.



From guido@python.org  Wed Oct 16 01:47:17 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 15 Oct 2002 20:47:17 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
Message-ID: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>

I just found a place in my own code that broke because stat now
returns floats for mtime etc.  I was using a Zope "OIBTree" object,
which is a dict-like extension type mapping strings to ints; it
very reasonbly doesn't like it if you assign a float to it.

I'm not say that because *my* code breaks, this is a bad feature.  But
I am worried that we'll find more programs using external data types
that won't accept floats (I'm thinking of Numpy arrays), and I'm now
not so sure if this is acceptable breakage...  Apparently the fact
that it "works" on the Mac isn't really a great proof.

Maybe we can introduce a variant of the stat() function that returns
floats, or alternative field names that are only available when using
attributes (not when using the tuple-ish API)?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 02:46:32 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 15 Oct 2002 21:46:32 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: Your message of "Tue, 15 Oct 2002 01:29:36 EDT."
 <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>
Message-ID: <200210160146.g9G1kWn20397@pcp02138704pcs.reston01.va.comcast.net>

> if we know we're not interested in anything other than 32-bit products,
> 
>     double _prod = (double)src1 * (double)src2;
>     if (_prod < 4294967296.0) /* 2**32 */
>         result = (size_t)_prod;
>     else
>         overflow;
> 
> does the trick reliably on all platforms I know of -- although not
> all Python platforms support floating-point efficiently!

I like this one, as well as the one using 64-bit precision.  I wonder
if we could collect a bunch of different macros and make them tunable
for different platforms?  (Including a "don't check for overflow"
version for people who wanna play fast and loose. :-)  The trick will
to ensure that the macro arguments are used exactly once in all
versions (otherwise switching macros might unveal a new class of
bugs).

Apart from *how* to check, there's a variety of cases depending on the
types of the input and output variables.

There's the ob_size calculation for sequence repeat, which takes a C
int and a C long and should return something that fits in a C int; and
there's the malloc size calculation for various ones, which should
take an int and a (small constant) size_t and return a size_t.

So maybe we need two macros: one for int*long->int, and one for
int*size_t->size_t?  With several different implementations, chosen by
configure.

But first we'd have to see of there are enough places to benefit from
such elaborate machinery.

There are also additions, usually of small constants, which generally
operate on the size_t.  These are easier to test: if you add a small
positive constant to a size_t, and the result is smaller than the
original size_t, an overflow occurred.  Similar for plain ints.  So
I'm not sure we need much help with these.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Oct 16 03:09:47 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 15 Oct 2002 22:09:47 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15788.51819.500992.678200@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Maybe we can introduce a variant of the stat() function that
    GvR> returns floats, or alternative field names that are only
    GvR> available when using attributes (not when using the tuple-ish
    GvR> API)?

+1 on the latter idea.  Since the st_mtime, st_atime, and st_ctime
attributes, and the tuple-ish API are documented as returning ints, I
think you shouldn't change that.  Provide different field names for
float values, e.g. f_mtime, f_ctime, f_atime, or maybe just mtime,
atime, and ctime... ?

-Barry


From xscottg@yahoo.com  Wed Oct 16 04:41:20 2002
From: xscottg@yahoo.com (Scott Gilbert)
Date: Tue, 15 Oct 2002 20:41:20 -0700 (PDT)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEANDMAA.tim.one@comcast.net>
Message-ID: <20021016034120.46693.qmail@web40101.mail.yahoo.com>

[Tim Peters]
> 
> I'll repeat that it's hard to do this correctly and quickly (c.f. earlier
> comments about the history of Python's subtly buggy attempts).
> 
[...]
> 
>         0x01ff *
>         0x00ff
>         ------
>       0000
>         00ff
>         0000
>           fe01
>       --------
>       0001fd01
>          ^ oops
> 

That was sloppy on my part.  Thanks for showing me my error.  (That's as
gracious as I can be while saying "D'Oh!" on the inside.  :-)

Clearly you know the long answer to this problem.  It could involve taking
the ugly expression I had and adding 3 more terms to see if it adds up to a
problem.  Or if all those integer multiplies make it less attractive than
just going to double, then taking the log2() of each operand and counting
the bits to see if there is a problem.  Or using the PyLong_* machinery. 
Or something similar.  And you've got your fast path past the test for the
common case, so it looks like you're just hesitant to make the big ugly
macro...








__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com


From greg@cosc.canterbury.ac.nz  Wed Oct 16 06:13:04 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Oct 2002 18:13:04 +1300 (NZDT)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEANDMAA.tim.one@comcast.net>
Message-ID: <200210160513.g9G5D4003141@oma.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> I'll repeat that it's hard to do this correctly and quickly (c.f. earlier
> comments about the history of Python's subtly buggy attempts).

The really tragic thing about all this is that most
machines have perfectly good facilities for detecting
overflow at the machine level, but no commonly used
HLL that I know of provides access to them, for some 
unfathomable reason.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Wed Oct 16 06:29:18 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 16 Oct 2002 01:29:18 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <200210160513.g9G5D4003141@oma.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEOJBLAB.tim.one@comcast.net>

[Greg Ewing]
> The really tragic thing about all this is that most
> machines have perfectly good facilities for detecting
> overflow at the machine level, but no commonly used
> HLL that I know of provides access to them, for some
> unfathomable reason.

Well, that's where it bifurcates Yet Again -- Pentiums have a perfectly
usable "give me the high bits of a 32x32->64 bit product" instruction, and
it's just a couple lines of inline assembler under MSVC or gcc (both make
such stuff easy to get at, embedded in C code).

But as I said at the start, I ignore these things until they pop up in real
life.  So few *have* popped in real life that there hasn't been any point to
creating mounds of platform-dependent #ifdef'ed infrastructure.



From tismer@tismer.com  Wed Oct 16 06:36:41 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 15 Oct 2002 22:36:41 -0700
Subject: [Python-Dev] Stackless for 2.2.2 ready!
Message-ID: <3DACFAE9.60905@tismer.com>

Hi Python-community,

The Stackless Python beta has been ported to the
Python 2.2.2 release, one day after the announce.

It could have been even faster, but I had to work
a lot on typeobject.c, which had been backported from 2.3.
My own patches to that are now very tiny and will be
submitted, soon.

Changes to Stackless:
The option to build statically has been re-enabled.
Supported platforms are now
  Windows,
  Mac OS X,
  Linux-X86,
  Cygwin,
  SunOS Sparc with gcc,
  PPC Unix,
  S390 & S390x.
A scheduler object with overridable strategies has been
prepared (functionality will be added these days)
Metatypes can have slots (se above).

See http://www.stackless.com/ for details, CVS access etc.
Binaries for Python 2.2.2 have been built and include _tkinter.

CVS access to the last 2.2.1 version is possible, check out
with the tag "last_2_2_1".

good luck - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tim.one@comcast.net  Wed Oct 16 06:44:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 16 Oct 2002 01:44:02 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <200210160146.g9G1kWn20397@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEOKBLAB.tim.one@comcast.net>

[Guido]
> ...
> There are also additions, usually of small constants, which generally
> operate on the size_t.  These are easier to test: if you add a small
> positive constant to a size_t, and the result is smaller than the
> original size_t, an overflow occurred.

More generally, if x and y are both size_t, and sizeof(size_t) >=
sizeof(unsigned int), x+y overflowed if and only if

    x+y < x

Checking

    x+y < y

is equivalent -- you can compare the sum to either input; doesn't matter.

> Similar for plain ints.

Signed ints are harder.  If the signs differ, overflow is impossible.  If
the signs are the same, overflow occurred if and only if the sign of the
result differs from the common input sign.

    sum = x+y
    overflow iff ((sum ^ x) & (sum ^ y)) < 0

> So I'm not sure we need much help with these.

Well, they're all obscure as hell.




From martin@v.loewis.de  Wed Oct 16 06:57:17 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 07:57:17 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <15788.51819.500992.678200@gargle.gargle.HOWL>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <15788.51819.500992.678200@gargle.gargle.HOWL>
Message-ID: <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>

barry@python.org (Barry A. Warsaw) writes:

> +1 on the latter idea.  Since the st_mtime, st_atime, and st_ctime
> attributes, and the tuple-ish API are documented as returning ints, I
> think you shouldn't change that.  Provide different field names for
> float values, e.g. f_mtime, f_ctime, f_atime, or maybe just mtime,
> atime, and ctime... ?

The previous suggestion was st_mtimef, st_ctimef, ...

Regards,
Martin


From martin@v.loewis.de  Wed Oct 16 07:08:29 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 08:08:29 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> I just found a place in my own code that broke because stat now
> returns floats for mtime etc.  I was using a Zope "OIBTree" object,
> which is a dict-like extension type mapping strings to ints; it
> very reasonbly doesn't like it if you assign a float to it.

In this case, could you fix your code by truncating/rounding, or by
using a different container?

> Maybe we can introduce a variant of the stat() function that returns
> floats, or alternative field names that are only available when using
> attributes (not when using the tuple-ish API)?

Assuming that in all cases of breakage it would be possible to easily
"fix" the code somehow, I'd like to have a migration plan, following
PEP 5: I.e. together with adding a new feature for float time stamps,
we should deprecate the old feature (of int time stamps) - perhaps not
right now, but as a pending deprecation. Then, after some time, we
should drop the old feature and continue with just the new one.

So I'd like to see a solution which does not look ugly when the
deprecated feature is gone and just the new feature stays.

Perhaps a flag "times_are_float" would be the right approach?
N: flag defaults to False, users need to ask for floats explicitly
N+1: omitting the flag is deprecated
N+2: flag defaults to True, users need to ask for ints explicitly
N+5: the flag is removed

There are variations to this plan, adding a phase where it is an error
to omit the flag, or where a pending deprecation is added.

Regards,
Martin


From mal@lemburg.com  Wed Oct 16 08:09:13 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 09:09:13 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DAD1099.1040507@lemburg.com>

Guido van Rossum wrote:
> I just found a place in my own code that broke because stat now
> returns floats for mtime etc.  I was using a Zope "OIBTree" object,
> which is a dict-like extension type mapping strings to ints; it
> very reasonbly doesn't like it if you assign a float to it.
> 
> I'm not say that because *my* code breaks, this is a bad feature.  But
> I am worried that we'll find more programs using external data types
> that won't accept floats (I'm thinking of Numpy arrays), and I'm now
> not so sure if this is acceptable breakage...  Apparently the fact
> that it "works" on the Mac isn't really a great proof.
> 
> Maybe we can introduce a variant of the stat() function that returns
> floats, or alternative field names that are only available when using
> attributes (not when using the tuple-ish API)?

If you're suggesting a new API, why not have it return one of
the new datetime objects ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From arigo@tunes.org  Wed Oct 16 09:11:18 2002
From: arigo@tunes.org (Armin Rigo)
Date: Wed, 16 Oct 2002 10:11:18 +0200 (CEST)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <200210160146.g9G1kWn20397@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <Pine.OSF.3.96.1021016100251.1277343A-100000@aster2.ulb.ac.be>

Hello Guido,

On Tue, 15 Oct 2002, Guido van Rossum wrote:
> I like this one, as well as the one using 64-bit precision.  I wonder
> if we could collect a bunch of different macros and make them tunable
> for different platforms?

Doable.  I experimented with some macros that do (unsigned int)*(unsigned
int)->size_t, where the macro to use is selected according to #if's based
on the relative size of int, long and long long.

> (...) The trick will
> to ensure that the macro arguments are used exactly once in all
> versions (otherwise switching macros might unveal a new class of
> bugs).

Harder...

> Apart from *how* to check, there's a variety of cases depending on the
> types of the input and output variables.

True as well... Why are C macros so limited?...

> There are also additions, usually of small constants, which generally
> operate on the size_t.

If we are sure these are small, we can just add without any check if we
reserve a whole range of values for "overflow", e.g. ((size_t)~0x7fff) to
((size_t)~0). Not too clean however.


Armin



From oren-py-d@hishome.net  Wed Oct 16 09:15:32 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Wed, 16 Oct 2002 04:15:32 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>
References: <200210150424.g9F4Ous17546@pcp02138704pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCKEKPBLAB.tim.one@comcast.net>
Message-ID: <20021016081532.GA49294@hishome.net>

On Tue, Oct 15, 2002 at 01:29:36AM -0400, Tim Peters wrote:
> BTW, if we know we're not interested in anything other than 32-bit products,
> 
>     double _prod = (double)src1 * (double)src2;
>     if (_prod < 4294967296.0) /* 2**32 */
>         result = (size_t)_prod;
>     else
>         overflow;

s/_prod;/src1 * src2;/

Re-calculating the product with integer multiplication is probably faster 
than converting a double to integer on any sane CPU. 

I wonder if the old SPARCs that don't have an integer multiply instruction 
actually have a floating point to integer conversion that is faster than 
integer mul. Wasting "expensive" multiplications this way still feels 
blasphemous :)

	Oren


From recomnd@pandanic.com  Wed Oct 16 08:25:35 2002
From: recomnd@pandanic.com (pandaNIC)
Date: Wed, 16 Oct 2002 15:25:35 +0800
Subject: [Python-Dev] Recommend "Register domain with only $14.95"
Message-ID: <2002101672535654CBOLDNQEHKHE@M5Mailer>

This is a multi-part message in MIME format

--=_NextPart_2rfkindysadvnqw3nerasdf
Content-Type: text/plain;charset="windows-1252"
Content-Transfer-Encoding: quoted-printable


Somebody send you this mail to recommend our domain registration service=
:


register/renew or transfer domain (com/net/org/biz/info/us) with only $1=
4.95 per year.


go to http://www.pandanic.com for more details, and check out the other =
services, such as email/url forwarding and DNS manager.




(this mail will be sent only by once)



----------------------------------------
http://www.pandanic.com

--=_NextPart_2rfkindysadvnqw3nerasdf
Content-Type: text/html; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dwindows-=
1252">
<META content=3D"MSHTML 6.00.2719.2200" name=3DGENERATOR></HEAD>
<BODY><DIV>&nbsp;</DIV>
<DIV>Somebody send you this mail to recommend our domain registration=20=

service:</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>register/renew or transfer domain (com/net/org/biz/info/us) with on=
ly=20
$14.95 per year.</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>go to <A href=3D"http://www.pandanic.com">http://www.pandanic.com</=
A> for=20
more details, and check out the other services, such as email/url forwar=
ding and=20
DNS manager.</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>(this mail will be sent only by once)</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<DIV>----------------------------------------</DIV>
<DIV><A href=3D"http://www.pandanic.com">http://www.pandanic.com</A></DI=
V>
<DIV>&nbsp;</DIV></BODY></HTML>



--=_NextPart_2rfkindysadvnqw3nerasdf--


From loewis@informatik.hu-berlin.de  Wed Oct 16 10:00:03 2002
From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=)
Date: 16 Oct 2002 11:00:03 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <3DAD1099.1040507@lemburg.com>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <3DAD1099.1040507@lemburg.com>
Message-ID: <j4k7kig3n0.fsf@informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> If you're suggesting a new API, why not have it return one of
> the new datetime objects ?

Wouldn't that require to implement this type first?

Regards,
Martin


From mal@lemburg.com  Wed Oct 16 10:40:25 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 11:40:25 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>	<3DAD1099.1040507@lemburg.com> <j4k7kig3n0.fsf@informatik.hu-berlin.de>
Message-ID: <3DAD3409.6060005@lemburg.com>

Martin v. L=F6wis wrote:
> "M.-A. Lemburg" <mal@lemburg.com> writes:
>=20
>=20
>>If you're suggesting a new API, why not have it return one of
>>the new datetime objects ?
>=20
>=20
> Wouldn't that require to implement this type first?

AFAIK, PythonLabs is working on this. Even though development
seems to have stalled recently...

See CVS:/python/nondist/sandbox/datetime/

--=20
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Wed Oct 16 10:46:22 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 11:46:22 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
Message-ID: <3DAD356E.5080308@lemburg.com>

I was wondering whether this is an oversight or intended. Buffer
objects can certainly be compared to strings on a byte-by-byte
basis, so the compare result looks like a (long standing)
bug to me.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mwh@python.net  Wed Oct 16 11:50:05 2002
From: mwh@python.net (Michael Hudson)
Date: 16 Oct 2002 11:50:05 +0100
Subject: [Python-Dev] Memory size overflows
In-Reply-To: Greg Ewing's message of "Wed, 16 Oct 2002 18:13:04 +1300 (NZDT)"
References: <200210160513.g9G5D4003141@oma.cosc.canterbury.ac.nz>
Message-ID: <2mof9uslnm.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Tim Peters <tim.one@comcast.net>:
> 
> > I'll repeat that it's hard to do this correctly and quickly (c.f. earlier
> > comments about the history of Python's subtly buggy attempts).
> 
> The really tragic thing about all this is that most
> machines have perfectly good facilities for detecting
> overflow at the machine level, but no commonly used
> HLL that I know of provides access to them, for some 
> unfathomable reason.

Well, if we go that route we can do floating point properly too.

An impression I've got is that because noone actually uses the
overflow detection stuff, chip designers are placing less empahasis on
implementing them efficiently.  A particularly unpleasant circle :-(

Cheers,
M.

-- 
  If comp.lang.lisp *is* what vendors are relying on to make or 
  break Lisp sales, that's more likely the problem than is the 
  effect of any one of us on such a flimsy marketing strategy...
                                      -- Kent M Pitman, comp.lang.lisp


From mwh@python.net  Wed Oct 16 11:51:39 2002
From: mwh@python.net (Michael Hudson)
Date: 16 Oct 2002 11:51:39 +0100
Subject: [Python-Dev] Memory size overflows
In-Reply-To: Armin Rigo's message of "Wed, 16 Oct 2002 10:11:18 +0200 (CEST)"
References: <Pine.OSF.3.96.1021016100251.1277343A-100000@aster2.ulb.ac.be>
Message-ID: <2mlm4ysll0.fsf@starship.python.net>

Armin Rigo <arigo@tunes.org> writes:

> True as well... Why are C macros so limited?...

To remind us all that the world is a harsh place.

Now lets rewrite Python in common lisp.  Or pyrex.

Cheers,
M.

-- 
  I'm okay with intellegent buildings, I'm okay with non-sentient
  buildings. I have serious reservations about stupid buildings.
     -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)


From guido@python.org  Wed Oct 16 13:25:46 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 08:25:46 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 08:08:29 +0200."
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>

> > I just found a place in my own code that broke because stat now
> > returns floats for mtime etc.  I was using a Zope "OIBTree" object,
> > which is a dict-like extension type mapping strings to ints; it
> > very reasonbly doesn't like it if you assign a float to it.
> 
> In this case, could you fix your code by truncating/rounding, or by
> using a different container?

I added int() around the st_mtime reference, because there's no
"OFBTree" data type, and because second resolution was good enough.

> > Maybe we can introduce a variant of the stat() function that returns
> > floats, or alternative field names that are only available when using
> > attributes (not when using the tuple-ish API)?
> 
> Assuming that in all cases of breakage it would be possible to easily
> "fix" the code somehow, I'd like to have a migration plan, following
> PEP 5: I.e. together with adding a new feature for float time stamps,
> we should deprecate the old feature (of int time stamps) - perhaps not
> right now, but as a pending deprecation. Then, after some time, we
> should drop the old feature and continue with just the new one.
> 
> So I'd like to see a solution which does not look ugly when the
> deprecated feature is gone and just the new feature stays.
> 
> Perhaps a flag "times_are_float" would be the right approach?
> N: flag defaults to False, users need to ask for floats explicitly
> N+1: omitting the flag is deprecated
> N+2: flag defaults to True, users need to ask for ints explicitly
> N+5: the flag is removed
> 
> There are variations to this plan, adding a phase where it is an error
> to omit the flag, or where a pending deprecation is added.

Agreed, but I don't like adding and then deleting a "temporary" new
argument to stat() and fstat().

In this particular case, I think having a global flag should be good
enough.  We can easily rig the standard library not to depend on its
value.  It should be settable by calling a function, e.g.

  os.stat_times(int)

or

  os.stat_times(float)

If you think passing the actual type objects is too cute, we can call
it stat_float_times() and pass a bool.

Without an argument it could return the current status, with the twist
that if the platform doesn't support float times it will always return
int.

Or maybe we can make it return the precision.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 13:29:29 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 08:29:29 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 09:09:13 +0200."
 <3DAD1099.1040507@lemburg.com>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <3DAD1099.1040507@lemburg.com>
Message-ID: <200210161229.g9GCTTS21887@pcp02138704pcs.reston01.va.comcast.net>

> If you're suggesting a new API, why not have it return one of
> the new datetime objects ?

That would require much more changes to the user code to use the new
feature.  And in many cases a basic timestamp is just what you need --
all you usually do is comparing them, occasionally printing them, and
both are done just as easily using the int or float form.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 13:39:05 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 08:39:05 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 11:46:22 +0200."
 <3DAD356E.5080308@lemburg.com>
References: <3DAD356E.5080308@lemburg.com>
Message-ID: <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>

> I was wondering whether this is an oversight or intended. Buffer
> objects can certainly be compared to strings on a byte-by-byte
> basis, so the compare result looks like a (long standing)
> bug to me.

I'd consider it a feature, designed to convey the subliminal message
"the buffer type should be deprecated". :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Oct 16 14:04:48 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 15:04:48 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DAD63F0.4080409@lemburg.com>

Guido van Rossum wrote:
>>I was wondering whether this is an oversight or intended. Buffer
>>objects can certainly be compared to strings on a byte-by-byte
>>basis, so the compare result looks like a (long standing)
>>bug to me.
> 
> 
> I'd consider it a feature, designed to convey the subliminal message
> "the buffer type should be deprecated". :-)

Fine, but what alternative is there which meets the following
requirements:

* signals "this data is binary data"
* compares just fine to strings
* gets accepted by all APIs which use the buffer interface to
   access the data
* has a C API which can be used in extensions
* is available in Python 2.1.x and up

(other than rolling my own mxBinary type...)

What I'd basically need is a type which simply wraps up any
string data, plays nice with 8-bit strings and signals
the binary nature of the content.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From barry@python.org  Wed Oct 16 14:06:16 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 16 Oct 2002 09:06:16 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15789.25672.851902.133899@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> In this particular case, I think having a global flag should
    GvR> be good enough.  We can easily rig the standard library not
    GvR> to depend on its value.  It should be settable by calling a
    GvR> function, e.g.

    GvR>   os.stat_times(int)

    GvR> or

    GvR>   os.stat_times(float)

    GvR> If you think passing the actual type objects is too cute, we
    GvR> can call it stat_float_times() and pass a bool.

So then why not just make this an optional argument flag on the
os.stat() call itself?  That way you could migrate some code at a
different rate than other code (a global flag might be fine for your
application, but break some 3rd party library you're using).

-Barry


From barry@python.org  Wed Oct 16 14:07:23 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 16 Oct 2002 09:07:23 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <15788.51819.500992.678200@gargle.gargle.HOWL>
 <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15789.25739.530553.355804@gargle.gargle.HOWL>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    >> +1 on the latter idea.  Since the st_mtime, st_atime, and
    >> st_ctime attributes, and the tuple-ish API are documented as
    >> returning ints, I think you shouldn't change that.  Provide
    >> different field names for float values, e.g. f_mtime, f_ctime,
    >> f_atime, or maybe just mtime, atime, and ctime... ?

    MvL> The previous suggestion was st_mtimef, st_ctimef, ...

Names which differ by one character at the end of the name are error
prone, IME.  Plus it makes stuff like dabbrev-expand harder to use. :)

-Barry


From martin@v.loewis.de  Wed Oct 16 14:15:55 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 15:15:55 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> In this particular case, I think having a global flag should be good
> enough.  We can easily rig the standard library not to depend on its
> value.

Sounds good. Unless I hear otherwise, I'll add stat_float_times(bool).

That still leaves the question of a migration plan, assuming that
users, in general, would appreciate to silently get subsecond
timestamps where possible. Here is a proposal:

- 2.3: just add stat_float_times. In the documentation, warn that the
       value will change in the future.
- 2.4: make stat_float_times(True) the default. Applications that break
       would need to add stat_float_times(False) in a central place.

This has no phase were a warning is emitted, but any scheme that adds
a warning means that the user first has to add something (to silence
the warning), and then to remove something (when the default changes -
although the option might stay forever).

Regards,
Martin



From martin@v.loewis.de  Wed Oct 16 14:30:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 15:30:40 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <3DAD63F0.4080409@lemburg.com>
References: <3DAD356E.5080308@lemburg.com>
 <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>
 <3DAD63F0.4080409@lemburg.com>
Message-ID: <m37kgih5of.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> * signals "this data is binary data"
> * compares just fine to strings
> * gets accepted by all APIs which use the buffer interface to
>    access the data
> * has a C API which can be used in extensions
> * is available in Python 2.1.x and up

I believe the string type meets all these requirements.

Regards,
Martin


From jacobs@penguin.theopalgroup.com  Wed Oct 16 14:44:27 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 16 Oct 2002 09:44:27 -0400 (EDT)
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de>
Message-ID: <Pine.LNX.4.44.0210160941140.20115-100000@penguin.theopalgroup.com>

On 16 Oct 2002, Martin v. Loewis wrote:
> Guido van Rossum <guido@python.org> writes:
> 
> > In this particular case, I think having a global flag should be good
> > enough.  We can easily rig the standard library not to depend on its
> > value.
> 
> Sounds good. Unless I hear otherwise, I'll add stat_float_times(bool).

Please don't!  The global setting isn't sufficient since many of us access
many 3rd party modules.  Why not simply create properies that return the
int/float versions from the same structure and store the float versions? 
That seems to be an instant win for all use-cases.

My 2e-2cents,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From martin@v.loewis.de  Wed Oct 16 15:09:10 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 16:09:10 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <Pine.LNX.4.44.0210160941140.20115-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210160941140.20115-100000@penguin.theopalgroup.com>
Message-ID: <m3wuoifpbt.fsf@mira.informatik.hu-berlin.de>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> > Sounds good. Unless I hear otherwise, I'll add stat_float_times(bool).
> 
> Please don't!  The global setting isn't sufficient since many of us access
> many 3rd party modules.  

Why is that insufficient? As long as you have modules that require int
time stamps, you can't switch to float time stamps. If you find you
are using such a module, tell the module author about that, and wait
for a corrected module.

> Why not simply create properies that return the int/float versions
> from the same structure and store the float versions?

How can the property know whether to return an int or a float?

Regards,
Martin


From mal@lemburg.com  Wed Oct 16 15:12:13 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 16:12:13 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
References: <3DAD356E.5080308@lemburg.com>	<200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>	<3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3DAD73BD.2020805@lemburg.com>

Martin v. Loewis wrote:
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> 
>>* signals "this data is binary data"
>>* compares just fine to strings
>>* gets accepted by all APIs which use the buffer interface to
>>   access the data
>>* has a C API which can be used in extensions
>>* is available in Python 2.1.x and up
> 
> 
> I believe the string type meets all these requirements.

Except one which was implicit: how to tell binary data from
text data. This information can sometimes be deduced
from the string content provided you know what text data
means to you, but this doesn't always work, since sometimes
binary data happens to look like text data (ie. use only
character ordinals as data bytes).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Oct 16 15:20:57 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 10:20:57 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 15:15:55 +0200."
 <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net> <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de> <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210161420.g9GEKvD01247@odiug.zope.com>

> > In this particular case, I think having a global flag should be good
> > enough.  We can easily rig the standard library not to depend on its
> > value.
> 
> Sounds good. Unless I hear otherwise, I'll add stat_float_times(bool).
> 
> That still leaves the question of a migration plan, assuming that
> users, in general, would appreciate to silently get subsecond
> timestamps where possible. Here is a proposal:
> 
> - 2.3: just add stat_float_times. In the documentation, warn that the
>        value will change in the future.
> - 2.4: make stat_float_times(True) the default. Applications that break
>        would need to add stat_float_times(False) in a central place.
> 
> This has no phase were a warning is emitted, but any scheme that adds
> a warning means that the user first has to add something (to silence
> the warning), and then to remove something (when the default changes -
> although the option might stay forever).

Sounds good.

I wonder if we could let the tuple-ish interface return ints forever
(or until it's no longer supported).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 15:28:52 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 10:28:52 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 09:06:16 EDT."
 <15789.25672.851902.133899@gargle.gargle.HOWL>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net> <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de> <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <15789.25672.851902.133899@gargle.gargle.HOWL>
Message-ID: <200210161428.g9GESqE01333@odiug.zope.com>

Barry:
> So then why not just make this an optional argument flag on the
> os.stat() call itself?  That way you could migrate some code at a
> different rate than other code (a global flag might be fine for your
> application, but break some 3rd party library you're using).

And Kevin:

> Please don't!  The global setting isn't sufficient since many of us
> access many 3rd party modules.

Most code doesn't care; most 3rd party code will work either way.
Martin proposed adding an option to the stat() call itself first, but
that seems a rather heavy-handed way to ensure backwards compatibility
for such a very rare situation.

If you are using a 3rd party library that breaks due to the floats,
you can disable the floats until you've got time to fix it.  You can
live with the ints a little longer.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 15:31:02 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 10:31:02 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 16:12:13 +0200."
 <3DAD73BD.2020805@lemburg.com>
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net> <3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de>
 <3DAD73BD.2020805@lemburg.com>
Message-ID: <200210161431.g9GEV2R01348@odiug.zope.com>

> >>* signals "this data is binary data"
> >>* compares just fine to strings
> >>* gets accepted by all APIs which use the buffer interface to
> >>   access the data
> >>* has a C API which can be used in extensions
> >>* is available in Python 2.1.x and up
> > 
> > 
> > I believe the string type meets all these requirements.
> 
> Except one which was implicit: how to tell binary data from
> text data. This information can sometimes be deduced
> from the string content provided you know what text data
> means to you, but this doesn't always work, since sometimes
> binary data happens to look like text data (ie. use only
> character ordinals as data bytes).

I don't understand why you need to signal "this is binary data" while
at the same time you want to be able to compare to strings.

Also, since buffer objects *can't* be compared to strings right now,
and you require compatibility with 2.1, there is no solution that
satisfies your requirements, so I conclude you're just being
"difficult". :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Wed Oct 16 15:28:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 16:28:04 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <200210161420.g9GEKvD01247@odiug.zope.com>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de>
 <200210161420.g9GEKvD01247@odiug.zope.com>
Message-ID: <m3it02fogb.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> I wonder if we could let the tuple-ish interface return ints forever
> (or until it's no longer supported).

That would not be easy to implement. The structseq type does not
support unnamed sequence members (yet), which we'ld need: the int
members would have only indices, and the float members only names.  

It probably could be made work by allowing NULL names in a
PyStructSequence_Field. I'll try to do this.

Regards,
Martin


From martin@v.loewis.de  Wed Oct 16 15:31:38 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 16:31:38 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <3DAD73BD.2020805@lemburg.com>
References: <3DAD356E.5080308@lemburg.com>
 <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>
 <3DAD63F0.4080409@lemburg.com>
 <m37kgih5of.fsf@mira.informatik.hu-berlin.de>
 <3DAD73BD.2020805@lemburg.com>
Message-ID: <m3elaqfoad.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Except one which was implicit: how to tell binary data from
> text data. 

I see. In this case,

class BinaryString(str):
  pass

should do. The string type *is* the type for binary data. For
character data, unicode is the type.

Regards,
Martin


From guido@python.org  Wed Oct 16 15:41:40 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 10:41:40 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 16:28:04 +0200."
 <m3it02fogb.fsf@mira.informatik.hu-berlin.de>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net> <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de> <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net> <m3d6qaikxg.fsf@mira.informatik.hu-berlin.de> <200210161420.g9GEKvD01247@odiug.zope.com>
 <m3it02fogb.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210161441.g9GEfew01452@odiug.zope.com>

> > I wonder if we could let the tuple-ish interface return ints forever
> > (or until it's no longer supported).
> 
> That would not be easy to implement. The structseq type does not
> support unnamed sequence members (yet), which we'ld need: the int
> members would have only indices, and the float members only names.  
> 
> It probably could be made work by allowing NULL names in a
> PyStructSequence_Field. I'll try to do this.

That sounds good to me.  Or you could "subclass" the PyStructSequence
class with something that special-cases the st_?time attributes.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Wed Oct 16 15:50:14 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 16 Oct 2002 10:50:14 -0400 (EDT)
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <m3wuoifpbt.fsf@mira.informatik.hu-berlin.de>
Message-ID: <Pine.LNX.4.44.0210161048290.20961-100000@penguin.theopalgroup.com>

On 16 Oct 2002, Martin v. Loewis wrote:
> Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> 
> > > Sounds good. Unless I hear otherwise, I'll add stat_float_times(bool).
> > 
> > Please don't!  The global setting isn't sufficient since many of us access
> > many 3rd party modules.  
> 
> Why is that insufficient? As long as you have modules that require int
> time stamps, you can't switch to float time stamps. If you find you
> are using such a module, tell the module author about that, and wait
> for a corrected module.

Why break all modules when only one needs the old behavior?  Think about how
from __future__ import is set up.

> > Why not simply create properies that return the int/float versions
> > from the same structure and store the float versions?
> 
> How can the property know whether to return an int or a float?

By name -- anyone who wants a floating point value can request
fst_mtime vs. st_mtime, or whatever.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Wed Oct 16 16:01:19 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 11:01:19 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 10:50:14 EDT."
 <Pine.LNX.4.44.0210161048290.20961-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210161048290.20961-100000@penguin.theopalgroup.com>
Message-ID: <200210161501.g9GF1JO01579@odiug.zope.com>

> Why break all modules when only one needs the old behavior?  Think
> about how from __future__ import is set up.

Because this is IMO a *very* small change, for which I don't want to
invoke such heavy guns.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Wed Oct 16 16:12:05 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 16 Oct 2002 11:12:05 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <20021016131701.12398.86665.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEACCMAA.gsw@agere.com>

Guido van Rossum wrote:
> I.e. a macro callable as
>   SAFE_MULTIPLY(destination, src1, src2, on_overflow);
> meaning roughly
>   destination = src1 * src2;
>   if (<overflow detected>)
>      on_overflow;

> There are also additions...These are easier to test: if you add a small
> positive constant to a size_t, and the result is smaller than the
> original size_t, an overflow occurred.

Why not use the same trick for multiplication? For src1,src2 > 0,
dest should always be >= MAX(src1,src2). SAFE_MULTIPLY could be
implemented something like this:

#define HALF_BITS (sizeof(size_t) * 4U)
#define LO_MASK ((((size_t)1) << (HALF_BITS))-1)
#define HI_MASK (~(LO_MASK))
#define MAX(a,b) (((a) >= (b)) ? (a) : (b))

#define SAFE_MULTIPLY(dest,src1,src2,on_error)  \
    {                                           \
        size_t _x = src1;                       \
        size_t _y = src2;                       \
        size_t _dest = _x * _y;                 \
                                                \
        if (_x && _y && ((_x|_y) & HI_MASK))    \
        {                                       \
            if (_dest < MAX(_x,_y))             \
            {                                   \
                on_error;                       \
            }                                   \
        }                                       \
                                                \
        dest = _dest;                           \
    }

-Jerry Williams



From barry@python.org  Wed Oct 16 16:09:10 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 16 Oct 2002 11:09:10 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <15789.25672.851902.133899@gargle.gargle.HOWL>
 <200210161428.g9GESqE01333@odiug.zope.com>
Message-ID: <15789.33046.223487.447141@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Most code doesn't care; most 3rd party code will work either
    GvR> way.  Martin proposed adding an option to the stat() call
    GvR> itself first, but that seems a rather heavy-handed way to
    GvR> ensure backwards compatibility for such a very rare
    GvR> situation.

    GvR> If you are using a 3rd party library that breaks due to the
    GvR> floats, you can disable the floats until you've got time to
    GvR> fix it.  You can live with the ints a little longer.

But you may end up ping ponging between floats and ints as you import
new libraries into your app.  If, as you surmise, most code won't
care, then that won't be a real problem.  But if it is, you might see
code like this:

    ostate = os.stat_times()
    os.stat_times(True)
    try:
        # do something requiring float times
    finally:
	os.stat_times(ostate)

Maybe it won't matter in practice, but global state like that always
tickles the gag reflex in me. ;)

-Barry


From martin@v.loewis.de  Wed Oct 16 16:21:26 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 17:21:26 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <Pine.LNX.4.44.0210161048290.20961-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0210161048290.20961-100000@penguin.theopalgroup.com>
Message-ID: <m3vg42e7ex.fsf@mira.informatik.hu-berlin.de>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> Why break all modules when only one needs the old behavior?  Think about how
> from __future__ import is set up.

Nothing will break; that's the entire purpose of this change.

> By name -- anyone who wants a floating point value can request
> fst_mtime vs. st_mtime, or whatever.

Been there, done that (posixmodule.c 2.254). Please understand that,
after the transitional phase, fst_mtime is what everyone will use.
This is plain ugly, since it is inconsistent with the names of all
other fields. So all we need to do is to accommodate the transition,
and we do that with Guido's proposed change.

Regards,
Martin



From mal@lemburg.com  Wed Oct 16 16:24:20 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 17:24:20 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
References: <3DAD356E.5080308@lemburg.com>	<200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>	<3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de>  	<3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com>
Message-ID: <3DAD84A4.6020008@lemburg.com>

Guido van Rossum wrote:
>>>>* signals "this data is binary data"
>>>>* compares just fine to strings
>>>>* gets accepted by all APIs which use the buffer interface to
>>>>  access the data
>>>>* has a C API which can be used in extensions
>>>>* is available in Python 2.1.x and up
>>>
>>>
>>>I believe the string type meets all these requirements.
>>
>>Except one which was implicit: how to tell binary data from
>>text data. This information can sometimes be deduced
>>from the string content provided you know what text data
>>means to you, but this doesn't always work, since sometimes
>>binary data happens to look like text data (ie. use only
>>character ordinals as data bytes).
> 
> I don't understand why you need to signal "this is binary data" while
> at the same time you want to be able to compare to strings.

Because I use buffer objects to wrap string data to say
"this is binary data" to a database. When fetching the
same data back from the database I return a string and
I found the quirk mentioned in the subject while writing
a unit test for this. It's not a showstopper.

The above just was a hint not to deprecate the buffer
object until we've come up with a decent replacement
that's easy to adapt in existing code.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From martin@v.loewis.de  Wed Oct 16 16:25:19 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 17:25:19 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <15789.33046.223487.447141@gargle.gargle.HOWL>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <15789.25672.851902.133899@gargle.gargle.HOWL>
 <200210161428.g9GESqE01333@odiug.zope.com>
 <15789.33046.223487.447141@gargle.gargle.HOWL>
Message-ID: <m3r8eqe78g.fsf@mira.informatik.hu-berlin.de>

barry@python.org (Barry A. Warsaw) writes:

> But you may end up ping ponging between floats and ints as you import
> new libraries into your app.  If, as you surmise, most code won't
> care, then that won't be a real problem.  

With Guido's most recent suggestion (put ints into the tuple), it is
even more likely that nothing will break if floats are activated (we
should investigate the two known incidents to find out whether they
were using the tuple interface).

>         # do something requiring float times

I think this is a central point here: Why would anything *require*
float times? If the system does not support subsecond timestamps, or
Python does not expose that (e.g. Windows), you will get floats, but
you won't get fractions of a second.

> Maybe it won't matter in practice, but global state like that always
> tickles the gag reflex in me. ;)

I completely agree about reflexes and general feelings. However, we
assume that this indeed won't matter in practice.

Regards,
Martin



From jepler@unpythonic.net  Wed Oct 16 16:26:10 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Wed, 16 Oct 2002 10:26:10 -0500
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFEEACCMAA.gsw@agere.com>
References: <20021016131701.12398.86665.Mailman@mail.python.org> <GBEGLOMMCLDACBPKDIHFEEACCMAA.gsw@agere.com>
Message-ID: <20021016152609.GC9288@unpythonic.net>

On Wed, Oct 16, 2002 at 11:12:05AM -0400, Gerald S. Williams wrote:
> Guido van Rossum wrote:
> > I.e. a macro callable as
> >   SAFE_MULTIPLY(destination, src1, src2, on_overflow);
> > meaning roughly
> >   destination = src1 * src2;
> >   if (<overflow detected>)
> >      on_overflow;
> 
> > There are also additions...These are easier to test: if you add a small
> > positive constant to a size_t, and the result is smaller than the
> > original size_t, an overflow occurred.
> 
> Why not use the same trick for multiplication? For src1,src2 > 0,
> dest should always be >= MAX(src1,src2). SAFE_MULTIPLY could be
> implemented something like this:
[...]

Is this correct for all operands?  Consider 4-bit types, with valid
values 0..15.
    3*9 == 27, but 27%16 == 11 and 11>max(3, 9)
There seem to be 15 distinct operand pairs for which this fails in the
4-bit case.
    
    def check(m):
        for x in range(m):
            for y in range(x, m):
                z = x*y
                w = z % m
                if z != w and w>x and w>y: yield x, y, z, w

    >>> print list(check(16))
    [(3, 9, 27, 11), (3, 10, 30, 14), (4, 6, 24, 8), (4, 7, 28, 12), (4,
    11, 44, 12), (5, 5, 25, 9), (5, 6, 30, 14), (5, 9, 45, 13), (6, 7,
    42, 10), (6, 10, 60, 12), (6, 13, 78, 14), (7, 9, 63, 15), (7, 11,
    77, 13), (10, 11, 110, 14), (11, 13, 143, 15)]

    >>> print len(list(check(256))
    9915

Jeff


From guido@python.org  Wed Oct 16 16:36:29 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 11:36:29 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 11:09:10 EDT."
 <15789.33046.223487.447141@gargle.gargle.HOWL>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net> <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de> <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net> <15789.25672.851902.133899@gargle.gargle.HOWL> <200210161428.g9GESqE01333@odiug.zope.com>
 <15789.33046.223487.447141@gargle.gargle.HOWL>
Message-ID: <200210161536.g9GFaTI01841@odiug.zope.com>

> Maybe it won't matter in practice, but global state like that always
> tickles the gag reflex in me. ;)

Believe me, mine too -- if I thought it was a real problem.  But I
don't.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Oct 16 16:31:02 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 16 Oct 2002 11:31:02 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de>
 <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net>
 <15789.25672.851902.133899@gargle.gargle.HOWL>
 <200210161428.g9GESqE01333@odiug.zope.com>
 <15789.33046.223487.447141@gargle.gargle.HOWL>
 <m3r8eqe78g.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15789.34358.965278.784246@gargle.gargle.HOWL>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    >> But you may end up ping ponging between floats and ints as you
    >> import new libraries into your app.  If, as you surmise, most
    >> code won't care, then that won't be a real problem.

    MvL> With Guido's most recent suggestion (put ints into the
    MvL> tuple), it is even more likely that nothing will break if
    MvL> floats are activated (we should investigate the two known
    MvL> incidents to find out whether they were using the tuple
    MvL> interface).

Cool, that sounds like the best compromise.

-Barry


From guido@python.org  Wed Oct 16 16:40:25 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 11:40:25 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 17:24:20 +0200."
 <3DAD84A4.6020008@lemburg.com>
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net> <3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de> <3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com>
 <3DAD84A4.6020008@lemburg.com>
Message-ID: <200210161540.g9GFePD01901@odiug.zope.com>

> The above just was a hint not to deprecate the buffer
> object until we've come up with a decent replacement
> that's easy to adapt in existing code.

So, apart from the Python 2.1 requirement, subclassing str does the
trick, right?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 16 16:41:41 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 11:41:41 -0400
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: Your message of "Wed, 16 Oct 2002 17:25:19 +0200."
 <m3r8eqe78g.fsf@mira.informatik.hu-berlin.de>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net> <m3bs5usyoy.fsf@mira.informatik.hu-berlin.de> <200210161225.g9GCPkc21859@pcp02138704pcs.reston01.va.comcast.net> <15789.25672.851902.133899@gargle.gargle.HOWL> <200210161428.g9GESqE01333@odiug.zope.com> <15789.33046.223487.447141@gargle.gargle.HOWL>
 <m3r8eqe78g.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210161541.g9GFffv01914@odiug.zope.com>

> With Guido's most recent suggestion (put ints into the tuple), it is
> even more likely that nothing will break if floats are activated (we
> should investigate the two known incidents to find out whether they
> were using the tuple interface).

I was using the tuple interface -- this code needed to work under
Python 2.1.

> I completely agree about reflexes and general feelings. However, we
> assume that this indeed won't matter in practice.

Amen.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Oct 16 16:45:02 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 17:45:02 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
References: <3DAD356E.5080308@lemburg.com>	<200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>	<3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de>	<3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com>  	<3DAD84A4.6020008@lemburg.com> <200210161540.g9GFePD01901@odiug.zope.com>
Message-ID: <3DAD897E.6060800@lemburg.com>

Guido van Rossum wrote:
>>The above just was a hint not to deprecate the buffer
>>object until we've come up with a decent replacement
>>that's easy to adapt in existing code.
> 
> 
> So, apart from the Python 2.1 requirement, subclassing str does the
> trick, right?

Right.

Would be nice if there were a standard builtin, e.g. binary(),
for this and maybe some support code to go with it in C (e.g.
the type object would be nice to have at C level).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Oct 16 17:01:47 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 12:01:47 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 17:45:02 +0200."
 <3DAD897E.6060800@lemburg.com>
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net> <3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de> <3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com> <3DAD84A4.6020008@lemburg.com> <200210161540.g9GFePD01901@odiug.zope.com>
 <3DAD897E.6060800@lemburg.com>
Message-ID: <200210161601.g9GG1mM02180@odiug.zope.com>

> > So, apart from the Python 2.1 requirement, subclassing str does the
> > trick, right?
> 
> Right.
> 
> Would be nice if there were a standard builtin, e.g. binary(),
> for this and maybe some support code to go with it in C (e.g.
> the type object would be nice to have at C level).

I disagree.  There are a thousand different applications, and yours
seems rather unusual to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Wed Oct 16 17:00:06 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 16 Oct 2002 12:00:06 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <20021016152609.GC9288@unpythonic.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEACCMAA.gsw@agere.com>

Jeff Epler wrote:
> Is this correct for all operands?  Consider 4-bit types, with valid
> values 0..15.
>     3*9 == 27, but 27%16 == 11 and 11>max(3, 9)

You're right. It's a case of "fingers quicker than the
brain". There may be something worth investigating in
that type of relationship, but that isn't the answer.
I came up with it as I was responding with a somewhat
more working answer.

What I was originally going to post before my mental
lapse was this:

One solution to what Guido asked for is as follows. It
avoids the division, but I don't think it's worth it
(it doesn't make much difference on my platform):

#define FULL_BITS (sizeof(size_t) * 8U)
#define TOP_BIT (((size_t)1) << ((FULL_BITS)-1))
#define HALF_BITS (sizeof(size_t) * 4U)
#define LO_MASK ((((size_t)1) << (HALF_BITS))-1)
#define HI_MASK (~(LO_MASK))

#define SAFE_MULTIPLY(dest,src1,src2,on_error)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        \
        dest = _x * _y;\
        \
        if ((_x|_y) & HI_MASK)\
        {\
            size_t h;\
            size_t l;\
            size_t mask;\
            size_t low_bit;\
            size_t low_bit_mask;\
            \
            if (_x >= _y)\
            {\
                h = _x;\
                l = _y;\
            }\
            else\
            {\
                h = _y;\
                l = _x;\
            }\
            \
            for (low_bit=0,mask=TOP_BIT; mask & HI_MASK; ++low_bit,(mask>>=1))\
            {\
                if (h & mask)\
                {\
                    break;\
                }\
            }\
            \
            low_bit_mask = (size_t)1 << low_bit;\
            \
            do\
            {\
                if (l & mask)\
                {\
                    on_error;\
                }\
                mask >>= 1;\
            }\
            while (!(mask & low_bit_mask));\
        }\
    }

Well, at least it demonstrates how to avoid multiply-invoking
its parameters. :-)

-Jerry


From mal@lemburg.com  Wed Oct 16 17:15:02 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 16 Oct 2002 18:15:02 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
References: <3DAD356E.5080308@lemburg.com>	<200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>	<3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de>	<3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com>	<3DAD84A4.6020008@lemburg.com> <200210161540.g9GFePD01901@odiug.zope.com>  	<3DAD897E.6060800@lemburg.com> <200210161601.g9GG1mM02180@odiug.zope.com>
Message-ID: <3DAD9086.1040204@lemburg.com>

Guido van Rossum wrote:
>>>So, apart from the Python 2.1 requirement, subclassing str does the
>>>trick, right?
>>
>>Right.
>>
>>Would be nice if there were a standard builtin, e.g. binary(),
>>for this and maybe some support code to go with it in C (e.g.
>>the type object would be nice to have at C level).
> 
> 
> I disagree.  There are a thousand different applications, and yours
> seems rather unusual to me.

It's not at all unusual if you interface to databases. These offer
you three choices: character data, Unicode data and binary data
and each of these is handled slightly differently.

We currently don't have any notion of separating character data from
binary except the difference between Unicode and strings. Using
Unicode for character data only and reserving strings for
binary data would be nice, except that practice shows that this
doesn't always work because not all tools in the chain are
ready for Unicode just yet (including Python's stdlib itself).

Nevermind, I'll roll my own,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Oct 16 17:26:59 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 12:26:59 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 18:15:02 +0200."
 <3DAD9086.1040204@lemburg.com>
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net> <3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de> <3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com> <3DAD84A4.6020008@lemburg.com> <200210161540.g9GFePD01901@odiug.zope.com> <3DAD897E.6060800@lemburg.com> <200210161601.g9GG1mM02180@odiug.zope.com>
 <3DAD9086.1040204@lemburg.com>
Message-ID: <200210161626.g9GGQxM08939@odiug.zope.com>

> >>Would be nice if there were a standard builtin, e.g. binary(),
> >>for this and maybe some support code to go with it in C (e.g.
> >>the type object would be nice to have at C level).
> > 
> > I disagree.  There are a thousand different applications, and yours
> > seems rather unusual to me.
> 
> It's not at all unusual if you interface to databases. These offer
> you three choices: character data, Unicode data and binary data
> and each of these is handled slightly differently.

I figure that most apps will be happy to return 8-bit strings for
binary data; that's what 8-bit strings were explicitly designed to
support.

> We currently don't have any notion of separating character data from
> binary except the difference between Unicode and strings. Using
> Unicode for character data only and reserving strings for
> binary data would be nice, except that practice shows that this
> doesn't always work because not all tools in the chain are
> ready for Unicode just yet (including Python's stdlib itself).

No, we'll eventually need a separate data type, but I doubt it needs
to be as compatible with the current 8-bit string type as your
requirements state.

> Nevermind, I'll roll my own,

You're welcome.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From xscottg@yahoo.com  Wed Oct 16 17:27:23 2002
From: xscottg@yahoo.com (Scott Gilbert)
Date: Wed, 16 Oct 2002 09:27:23 -0700 (PDT)
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <3DAD9086.1040204@lemburg.com>
Message-ID: <20021016162723.13099.qmail@web40102.mail.yahoo.com>

--- "M.-A. Lemburg" <mal@lemburg.com> wrote:
> 
> We currently don't have any notion of separating character data from
> binary except the difference between Unicode and strings. Using
> Unicode for character data only and reserving strings for
> binary data would be nice, except that practice shows that this
> doesn't always work because not all tools in the chain are
> ready for Unicode just yet (including Python's stdlib itself).
> 
> Nevermind, I'll roll my own,
>

I don't know if you've seen PEP 296, but I still hope to finish it and have
it accepted in time for the Python 2.3 release.  That doesn't meet your
requirement of working with Python 2.1 however.









__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com


From tim.one@comcast.net  Wed Oct 16 17:31:44 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 16 Oct 2002 12:31:44 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <20021016081532.GA49294@hishome.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGECCDMAA.tim.one@comcast.net>

[Tim]
> BTW, if we know we're not interested in anything other than
> 32-bit products,
>
>     double _prod = (double)src1 * (double)src2;
>     if (_prod < 4294967296.0) /* 2**32 */
>         result = (size_t)_prod;
>     else
>         overflow;

[Oren Tirosh]
> s/_prod;/src1 * src2;/
>
> Re-calculating the product with integer multiplication is probably faster
> than converting a double to integer on any sane CPU.

This is so.  Better, because it allows more instruction-level parallelism:

    double dprod = (double)src1 * (double)src2;
    size_t iprod = src1 * src2;
    if (dprod <  4294967296.0) /* 2**32 */
        result = iprod;
    else
        /* overflow */

> I wonder if the old SPARCs that don't have an integer multiply
> instruction actually have a floating point to integer conversion
> that is faster than integer mul.

I'll let someone with an old SPARC worry about that.

> Wasting "expensive" multiplications this way still feels blasphemous :)

Well, we could simulate multiplication by hand, with a bit-at-a-time
shift-and-add loop.  This reduces the detect-mul-overflow case to the
detect-add-overflow case.  Bonus:  on some odd boxes, it would be faster
<wink/sigh>.



From tim.one@comcast.net  Wed Oct 16 17:37:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 16 Oct 2002 12:37:54 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <Pine.OSF.3.96.1021016100251.1277343A-100000@aster2.ulb.ac.be>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMECCDMAA.tim.one@comcast.net>

[Guido]
> The trick will to ensure that the macro arguments are used exactly
> once in all versions (otherwise switching macros might unveal a new
> class of> > bugs).

[Armin Rigo]
> Harder...

It's trivial if you define the macro to take an lvalue result argument.
Then the macro can expand to a block, and so declare block-local temp
variables, including, if desired, temps that are merely initialized to other
macro arguments.  Such temps can be reused as often as you like without
semantic consquence.  Contorting this into expression rvalue form is a
challenge, but a worthier challenge is yielding to common sense here <wink>.



From martin@v.loewis.de  Wed Oct 16 18:11:21 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Oct 2002 19:11:21 +0200
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <200210161601.g9GG1mM02180@odiug.zope.com>
References: <3DAD356E.5080308@lemburg.com>
 <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net>
 <3DAD63F0.4080409@lemburg.com>
 <m37kgih5of.fsf@mira.informatik.hu-berlin.de>
 <3DAD73BD.2020805@lemburg.com>
 <200210161431.g9GEV2R01348@odiug.zope.com>
 <3DAD84A4.6020008@lemburg.com>
 <200210161540.g9GFePD01901@odiug.zope.com>
 <3DAD897E.6060800@lemburg.com>
 <200210161601.g9GG1mM02180@odiug.zope.com>
Message-ID: <m34rbme2bq.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Would be nice if there were a standard builtin, e.g. binary(),
> > for this and maybe some support code to go with it in C (e.g.
> > the type object would be nice to have at C level).
> 
> I disagree.  There are a thousand different applications, and yours
> seems rather unusual to me.

I do think there should be a string type for binary data, and that the
standard string type should become Unicode one day. There was past
discussion about this, at which proponents suggest that there should
be even binary literals.

Of course, all I/O would use the binary, unless an encoding was
specified when creating the stream.

Regards,
Martin



From guido@python.org  Wed Oct 16 18:19:07 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 16 Oct 2002 13:19:07 -0400
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: Your message of "Wed, 16 Oct 2002 19:11:21 +0200."
 <m34rbme2bq.fsf@mira.informatik.hu-berlin.de>
References: <3DAD356E.5080308@lemburg.com> <200210161239.g9GCd5v22010@pcp02138704pcs.reston01.va.comcast.net> <3DAD63F0.4080409@lemburg.com> <m37kgih5of.fsf@mira.informatik.hu-berlin.de> <3DAD73BD.2020805@lemburg.com> <200210161431.g9GEV2R01348@odiug.zope.com> <3DAD84A4.6020008@lemburg.com> <200210161540.g9GFePD01901@odiug.zope.com> <3DAD897E.6060800@lemburg.com> <200210161601.g9GG1mM02180@odiug.zope.com>
 <m34rbme2bq.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210161719.g9GHJ7E09833@odiug.zope.com>

> I do think there should be a string type for binary data, and that the
> standard string type should become Unicode one day. There was past
> discussion about this, at which proponents suggest that there should
> be even binary literals.
> 
> Of course, all I/O would use the binary, unless an encoding was
> specified when creating the stream.

Maybe it's time for a PEP outlining and detailing this view of the future?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Wed Oct 16 21:30:55 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 16 Oct 2002 16:30:55 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFMEACCMAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFCEAECMAA.gsw@agere.com>

Actually, my original solution doesn't seem to stand up
to comprehensive testing, either. Essentially all it was
doing was checking to see if the sum of the high-order
bit positions was less than the number of bits. I created
a Python equivalent for smaller width numbers. At a width
of 6 bits, the following combinations aren't reported as
overflows but should be:

(6, 13) (6, 14) (6, 15) (7, 11)
(7, 12) (7, 13) (7, 14) (7, 15)

Perhaps there is some property of the false negatives
that can be exploited.

I don't know if it helps, but there are certainly a number
of ways to narrow the field:

  if (src1 & HIMASK) && (src2 & HIMASK), overflow=True
  if !(src1 & HIMASK) && !(src2 & HIMASK), overflow=False
  if !src1 || !src2, overflow=False
  if dest < MAX(src1,src2), overflow=True
  if log2(src1) + log2(src2) >= width, overflow=True
  (the integer form of log2 can be done quite cheaply by
   exploiting known properties of src1 and src2 at this
   point)

But you'd still need to filter out false negatives from
that field. Perhaps there's a way, but divides would have
to be really expensive for this to be worth it.

-Jerry


From greg@cosc.canterbury.ac.nz  Wed Oct 16 22:32:48 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Oct 2002 10:32:48 +1300 (NZDT)
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210162132.g9GLWmX09153@oma.cosc.canterbury.ac.nz>

martin@v.loewis.de (Martin v. Loewis):

> The previous suggestion was st_mtimef, st_ctimef, ...

For some reason, st_fmtime and st_fctime seem easier
to read and pronounce to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Oct 16 22:39:23 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Oct 2002 10:39:23 +1300 (NZDT)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <20021016081532.GA49294@hishome.net>
Message-ID: <200210162139.g9GLdNQ09171@oma.cosc.canterbury.ac.nz>

> Re-calculating the product with integer multiplication is probably faster 
> than converting a double to integer on any sane CPU. 

Um... what would be "sane" about that?

I don't know much about the Pentium. I do know that
the PowerPC doesn't have any way of moving a value
from a float to an integer register without going
through memory... but I regard that as a decidedly
INsane feature!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From gsw@agere.com  Thu Oct 17 05:36:23 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 17 Oct 2002 00:36:23 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFCEAECMAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFMEAECMAA.gsw@agere.com>

I felt obliged to post a working copy of SAFE_MULTIPLY. On
my platform, it's slower than simply dividing by one of the
multiplicands and comparing to the other, but it does get
the job done without division. This is what I started out
to write before my brain short-circuited.

Here's a simplified Python representation of the algorithm:

def check(x,y,m):
    xrem, xlog = math.frexp(x)
    yrem, ylog = math.frexp(y)

    if xlog + ylog == m + 1:
	return (xrem * yrem >= 0.5)
    else:
	return xlog + ylog > m + 1


And here it is in C:

#define FULL_BITS (sizeof(size_t) * 8U)
#define TOP_BIT (((size_t)1) << ((FULL_BITS)-1))
#define HALF_BITS (sizeof(size_t) * 4U)

#define SAFE_MULTIPLY(dest,src1,src2,on_overflow)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        size_t _dest = _x * _y;\
        size_t h;\
        int    hlog;\
        size_t l;\
        int    llog;\
        size_t mask;\
        \
        dest = _dest;\
        \
        if (_x >= _y)\
        {\
            h = _x;\
            l = _y;\
        }\
        else\
        {\
            h = _y;\
            l = _x;\
        }\
        \
        if (l)\
        {\
            for ((mask=TOP_BIT),(hlog=FULL_BITS-1); !(mask & h); mask >>= 1)\
            {\
                --hlog;\
            }\
            if (hlog >= HALF_BITS)\
            {\
                for (llog = hlog; !(mask & l); mask >>= 1)\
                {\
                    --llog;\
                }\
                if (((hlog + llog) > (FULL_BITS - 1)) ||\
                    (((hlog + llog) == (FULL_BITS - 1)) &&\
                     (((h >> 1) * l + ((h & 1) ? (l >> 1) : 0)) >= TOP_BIT)))\
                {\
                    on_overflow;\
                }\
            }\
        }\
    }

-Jerry


From tismer@tismer.com  Thu Oct 17 07:03:43 2002
From: tismer@tismer.com (Christian Tismer)
Date: Wed, 16 Oct 2002 23:03:43 -0700
Subject: [Python-Dev] Ann: Stackless technology merge under the way
Message-ID: <3DAE52BF.9040304@tismer.com>

Hi friends,

Stackless is changing heavily, once again.
After yesterday's move to Python 2.2.2,
I am now working on an integration of the old
technology (software only) and the new (modifying
the hardware stack).

This has a number of advantages:
- porting of existing applications, which depend on the
   old changes to Python's interpreter, becomes much easier.
   They tend to rely on a Python version of 2.0 or lower
   and use the old structures at the same time. Supporting
   these, again, makes it possible to do stepwise upgrades.
- existing applications like thread pickling (written by
   TwinSun.com) will be enabled again and merged into
   Stackless as a feature. While limited to
   plain frame contexts (no C function or method calls
   in between), this is very worth doing.
- the core Stackless code is using C stack manipulations
   right now in order to reduce stack size. This is not
   guaranteed to work with every C extension, since it is
   hard to check whether a foreign C call is sitting on the
   stack and relying on variables which must stay in place.
   The old technology was only able to switch Python stacks,
   when no recursive C calls were existing. This locks switches
   of C extensions automagically.
- the new technology can be removed from the Python core
   and be only used for the stackless extension, for tasklet
   switching. This is not decided yet, but an option.
- the amount of C stack space to me moved around is reduced
   drastically by this hybrid approach, since most of these
   vanish automatically. Although stack compression has been
   prepared (and will be supported), this is more time efficient.

The reason while I have no longer a problem to make this
decision is Python's current state. After generators have been
introduced, the core code already has a lot of properties
which I need to enable the old technology with ease.

Furthermore, this support code will not again try to cover
everything, but just the minimum to avoid most recursion.
It is neither planned to support stackless map or apply,
nor are continuations desired. The target are applications
which implemented coroutines on top of the old kernel.

This hybrid implementation has started today and will continue
over a couple of weeks.
The last "plain hardware switching" version is available under
the CVS tag "last_hardware_only", which is based on the
final 2.2.2 version.

I regard this move as the beginning of the
re-union of my brain halves    :-)

continuation-less - ly y'rs chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tino.lange@isg.de  Thu Oct 17 09:38:39 2002
From: tino.lange@isg.de (Tino Lange)
Date: Thu, 17 Oct 2002 10:38:39 +0200
Subject: [Python-Dev] PyEval_EvalCode()
Message-ID: <3DAE770F.5030304@isg.de>

Hi!

This prototype is not available when including "Python.h" (Using v2.1.3)
Why did you exclude it from the "official" API prototypes?

I found it in "eval.h", which has to be included manually, besides 
"Python.h".

Isn't that PyEval_EvalCode() the regular API function to run CodeObjects 
from Py_CompileString() ?

Cheers,

Tino



From arigo@tunes.org  Thu Oct 17 11:57:33 2002
From: arigo@tunes.org (Armin Rigo)
Date: Thu, 17 Oct 2002 12:57:33 +0200 (CEST)
Subject: [Python-Dev] Ann: Stackless technology merge under the way
In-Reply-To: <3DAE52BF.9040304@tismer.com>
Message-ID: <Pine.OSF.3.96.1021017125529.954218A-100000@aster1.ulb.ac.be>

Hello Christian,

On Wed, 16 Oct 2002, Christian Tismer wrote:
> Stackless is changing heavily, once again.

Good news.  It might be worth pointing out that even if continuations are
not directly supported, it should be possible to do them with thread
pickling.  Unpickling a given thread state several times does the trick.


Armin



From jason@tishler.net  Thu Oct 17 13:18:16 2002
From: jason@tishler.net (Jason Tishler)
Date: Thu, 17 Oct 2002 08:18:16 -0400
Subject: [Python-Dev] Dropping support for Tcl 8.0 and 8.1
In-Reply-To: <m3ptubbesp.fsf@mira.informatik.hu-berlin.de>
References: <200210010607.g9167Mns001471@mira.informatik.hu-berlin.de>
 <200210011338.g91Dclo08817@odiug.zope.com>
 <m3u1k6195a.fsf@mira.informatik.hu-berlin.de>
 <200210011519.g91FJwI21585@odiug.zope.com> <20021015175131.GB1536@tishler.net>
 <m3ptubbesp.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20021017121816.GC2104@tishler.net>

Martin,

On Tue, Oct 15, 2002 at 10:57:58PM +0200, Martin v. Loewis wrote:
> Jason Tishler <jason@tishler.net> writes:
> > This is probably too little, too late, but Cygwin is still stuck at
> > Tcl 8.0.  Would you be willing to keep this support in?
> 
> No, I've already made the changes to drop this support, and added
> support for UCS-4 mode in _tkinter.

Although the above is disappointing, your position is quite
understandable.  See below...

> > Note that I've tried to encourage Cygwin to upgrade to a more recent
> > version:
> > 
> >     http://cygwin.com/ml/cygwin/2002-10/msg00798.html
> 
> That seems the right way to go.

Agreed.

> They may have changed Cygwin by the time Python 2.3 is released.

Unfortunately, the above is unlikely.  There seems to be "issues" with
more recent Tcl versions under Cygwin.  This implies that Cygwin Python
2.3+ will no longer provide a pre-built _tkinter module until the issues
have been resolved. :,(

> If not, users wishing to use Cygwin _tkinter can use Mumit Khan's Tcl
> 8.3/8.4 ports:
> 
> ftp://ftp.nanotech.wisc.edu/pub/khan/tcl/tcltk-8.3.4-cygwin/

The above appears to be the most likely workaround.

Jason


From gsw@agere.com  Thu Oct 17 15:07:12 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 17 Oct 2002 10:07:12 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFMEAECMAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEAFCMAA.gsw@agere.com>

Here's a version of my previous algorithm with the checks added
back in to improve overall performance. This one runs about as
fast as the divide-and-compare algorithm for my test cases. :-)

OK, I'm done tweaking now...

#define FULL_BITS (sizeof(size_t) * 8U)
#define TOP_BIT (((size_t)1) << ((FULL_BITS)-1))
#define HALF_BITS (sizeof(size_t) * 4U)
#define MID_BIT (((size_t)1) << ((HALF_BITS)-1))
#define LO_MASK ((((size_t)1) << (HALF_BITS))-1)
#define HI_MASK (~(LO_MASK))

#define SAFE_MULTIPLY(dest,src1,src2,on_overflow)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        size_t _dest = _x * _y;\
        \
        dest = _dest;\
        \
        if ((_x | _y) & HI_MASK)\
        {\
            size_t h;\
            int    hlog;\
            size_t l;\
            int    llog;\
            size_t mask;\
            \
            if (_x >= _y)\
            {\
                h = _x;\
                l = _y;\
            }\
            else\
            {\
                h = _y;\
                l = _x;\
            }\
            \
            if ((h & HI_MASK) && (l))\
            {\
                if (l & HI_MASK)\
                {\
                    hlog = llog = FULL_BITS;\
                }\
                else\
                {\
                    for ((mask=TOP_BIT),(hlog=FULL_BITS-1);!(mask&h);mask>>=1)\
                    {\
                        --hlog;\
                    }\
                    for ((mask=MID_BIT),(llog=HALF_BITS-1);!(mask&l);mask>>=1)\
                    {\
                        --llog;\
                    }\
                }\
                if ((hlog + llog > FULL_BITS - 1) ||\
                    ((hlog + llog == FULL_BITS - 1) && !(_dest & TOP_BIT)))\
                {\
                    on_overflow;\
                }\
            }\
        }\
    }

-Jerry


From arigo@tunes.org  Thu Oct 17 16:16:42 2002
From: arigo@tunes.org (Armin Rigo)
Date: Thu, 17 Oct 2002 17:16:42 +0200 (CEST)
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFEEAFCMAA.gsw@agere.com>
Message-ID: <Pine.OSF.3.96.1021017171508.1300123A-100000@aster2.ulb.ac.be>

Hello,

On Thu, 17 Oct 2002, Gerald S. Williams wrote:
> Here's a version of my previous algorithm with the checks added
> back in to improve overall performance. This one runs about as
> fast as the divide-and-compare algorithm for my test cases. :-)

We might be running into code bloats thought.  If we go for such big
macros, I believe we should design them so that they run fast in the
common case and call a helper function otherwise. 


Armin



From guido@python.org  Thu Oct 17 16:18:22 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 17 Oct 2002 11:18:22 -0400
Subject: [Python-Dev] PyEval_EvalCode()
In-Reply-To: Your message of "Thu, 17 Oct 2002 10:38:39 +0200."
 <3DAE770F.5030304@isg.de>
References: <3DAE770F.5030304@isg.de>
Message-ID: <200210171518.g9HFIMc12781@odiug.zope.com>

> This prototype is not available when including "Python.h" (Using v2.1.3)
> Why did you exclude it from the "official" API prototypes?

Because it's unlikely that someone writing a Python extension would
ever have a need for it.

> I found it in "eval.h", which has to be included manually, besides 
> "Python.h".
> 
> Isn't that PyEval_EvalCode() the regular API function to run
> CodeObjects from Py_CompileString() ?

Probably -- I've never used it this way.  But why aren't you using
PyRun_String()?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Thu Oct 17 17:06:28 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 17 Oct 2002 12:06:28 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <Pine.OSF.3.96.1021017171508.1300123A-100000@aster2.ulb.ac.be>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEAGCMAA.gsw@agere.com>

Armin Rigo wrote:
> We might be running into code bloats thought.  If we go for such big
> macros, I believe we should design them so that they run fast in the
> common case and call a helper function otherwise. 

Easily done, but first you have to ask yourself if it's
really ever more efficient than this:

#define SAFE_MULTIPLY(dest,src1,src2,on_overflow)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        size_t _dest = _x * _y;\
        \
        dest = _dest;\
        \
        if ((_x | _y) & HI_MASK)\
        {\
            if ((_dest / _x) != _y)\
            {\
                on_overflow;\
            }\
        }\
    }

If so, here's a macro/helper version:

#define FULL_BITS (sizeof(size_t) * 8U)
#define TOP_BIT (((size_t)1) << ((FULL_BITS)-1))
#define HALF_BITS (sizeof(size_t) * 4U)
#define MID_BIT (((size_t)1) << ((HALF_BITS)-1))
#define LO_MASK ((((size_t)1) << (HALF_BITS))-1)
#define HI_MASK (~(LO_MASK))

#define SAFE_MULTIPLY(dest,src1,src2,on_overflow)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        size_t _dest = _x * _y;\
        \
        dest = _dest;\
        \
        if ((_x | _y) & HI_MASK)\
        {\
            if (_safe_multiply_check_for_overflow(_x,_y,_dest))\
            {\
                on_overflow;\
            }\
        }\
    }

int
_safe_multiply_check_for_overflow(
    size_t x,
    size_t y,
    size_t dest)
{
    size_t h;
    size_t l;

    if (x >= y)
    {
        h = x;
        l = y;
    }
    else
    {
        h = y;
        l = x;
    }

    if ((h & HI_MASK) && (l))
    {
        int    hlog;
        int    llog;

        if (l & HI_MASK)
        {
            hlog = llog = FULL_BITS;
        }
        else
        {
            size_t mask;

            for ((mask=TOP_BIT),(hlog=FULL_BITS-1);!(mask&h);mask>>=1)
            {
                --hlog;
            }
            for ((mask=MID_BIT),(llog=HALF_BITS-1);!(mask&l);mask>>=1)
            {
                --llog;
            }
        }
        if ((hlog + llog > FULL_BITS - 1) ||
            ((hlog + llog == FULL_BITS - 1) && !(dest & TOP_BIT)))
        {
            return 1;
        }
    }

    return 0;
}

Of course there are also the alternatives of using floating point
numbers (assuming the mantissa has as many bits as size_t), or a
double-width intermediate representation, if available.

There are also some other tricks that can be used to optimize this
solution. The time-consuming part is finding out if the sum of the
log2's of the multiplicands (i.e., highest bit positions) is less
than, greater than, or equal to <bits in size_t> - 1. But I said I
was done tweaking for now. :-)

-Jerry


From Tino.Lange@isg.de  Thu Oct 17 18:09:08 2002
From: Tino.Lange@isg.de (Tino Lange)
Date: Thu, 17 Oct 2002 19:09:08 +0200
Subject: [Python-Dev] PyEval_EvalCode()
References: <3DAE770F.5030304@isg.de> <200210171518.g9HFIMc12781@odiug.zope.com>
Message-ID: <3DAEEEB4.4EFE10CB@isg.de>

Guido van Rossum wrote:

Hi!

Thanks for your reply!

> > PyEval_EvalCode()
> > This prototype is not available when including "Python.h" (Using v2.1.3)
> > Why did you exclude it from the "official" API prototypes? 
> Because it's unlikely that someone writing a Python extension would
> ever have a need for it.

Sure?
I'd like to use it in a program that embeds a Python interpreter.
The code snippets get compiled and checked by that function and can be
called as a code object afterwards.
 
> > Isn't that PyEval_EvalCode() the regular API function to run
> > CodeObjects from Py_CompileString() ? 
> Probably -- I've never used it this way.

At least a solution like that is described in the lastest "Programming
Python" bible by Mark Lutz.
(covering Python 2)

> But why aren't you using
> PyRun_String()?

Speed, efficiency.
If the code get's executed quite often (i. e. multiple times a second)
it's worth precompiling (and maybe optimizing by Py_OptimizeFlag = 1),
or? PyRun_String() will compile and check _every_ time - wasting a lot
of resources.

Cheers

Tino


From guido@python.org  Thu Oct 17 18:29:00 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 17 Oct 2002 13:29:00 -0400
Subject: [Python-Dev] PyEval_EvalCode()
In-Reply-To: Your message of "Thu, 17 Oct 2002 19:09:08 +0200."
 <3DAEEEB4.4EFE10CB@isg.de>
References: <3DAE770F.5030304@isg.de> <200210171518.g9HFIMc12781@odiug.zope.com>
 <3DAEEEB4.4EFE10CB@isg.de>
Message-ID: <200210171729.g9HHT0r21574@odiug.zope.com>

> > > PyEval_EvalCode()
> > > This prototype is not available when including "Python.h" (Using v2.1.3)
> > > Why did you exclude it from the "official" API prototypes? 
> > Because it's unlikely that someone writing a Python extension would
> > ever have a need for it.
> 
> Sure?
> I'd like to use it in a program that embeds a Python interpreter.
> The code snippets get compiled and checked by that function and can be
> called as a code object afterwards.
>  
> > > Isn't that PyEval_EvalCode() the regular API function to run
> > > CodeObjects from Py_CompileString() ? 
> > Probably -- I've never used it this way.
> 
> At least a solution like that is described in the lastest "Programming
> Python" bible by Mark Lutz.
> (covering Python 2)
> 
> > But why aren't you using
> > PyRun_String()?
> 
> Speed, efficiency.
> If the code get's executed quite often (i. e. multiple times a second)
> it's worth precompiling (and maybe optimizing by Py_OptimizeFlag = 1),
> or? PyRun_String() will compile and check _every_ time - wasting a lot
> of resources.

Then feel free to import eval.h.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Oct 17 19:15:27 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 17 Oct 2002 14:15:27 -0400
Subject: [Python-Dev] creating struct-seq types from Python
Message-ID: <15790.65087.725796.331958@grendel.zope.com>

Martin,

I've addressed this to you since you seem to be playing with the
relevant code these days, but others may be interested as well.

Have you looked at what it would take to create new "structure
sequence" types from Python code?  This seems like a reasonable thing
to do, especially when an existing API returns a long(ish) tuple of
fields.  I'd like to be able to use it for my proposed urlparse
changes (http://www.python.org/sf/624325), but there seems to be no
way to get to it.  What's needed to do something currently feels like
too much code for what little is being done (see the patch if unsure
of what I mean).

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tismer@tismer.com  Thu Oct 17 19:24:56 2002
From: tismer@tismer.com (Christian Tismer)
Date: Thu, 17 Oct 2002 11:24:56 -0700
Subject: [Python-Dev] Ann: Stackless technology merge under the way
References: <Pine.OSF.3.96.1021017125529.954218A-100000@aster1.ulb.ac.be>
Message-ID: <3DAF0078.4060404@tismer.com>

Armin Rigo wrote:
> Hello Christian,
> 
> On Wed, 16 Oct 2002, Christian Tismer wrote:
> 
>>Stackless is changing heavily, once again.
> 
> 
> Good news.  It might be worth pointing out that even if continuations are
> not directly supported, it should be possible to do them with thread
> pickling.  Unpickling a given thread state several times does the trick.

This is correct.
Actually, re-enabling continuations would not be *that*
problem. What it needs it a copy-on-write like wrapper
around frames, to make them duplicate when necessary.
But I really don't want to go this path again,
last but not least to avoid giving Guido an argument :-)

a man cannot step into the same river twice -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From martin@v.loewis.de  Thu Oct 17 19:28:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 17 Oct 2002 20:28:54 +0200
Subject: [Python-Dev] Re: creating struct-seq types from Python
In-Reply-To: <15790.65087.725796.331958@grendel.zope.com>
References: <15790.65087.725796.331958@grendel.zope.com>
Message-ID: <m3hefkudg9.fsf@mira.informatik.hu-berlin.de>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> Have you looked at what it would take to create new "structure
> sequence" types from Python code?

Not really, but I think all you need to do is to expose
PyStructSequence_InitType. I would recommend an interface like

struct_seq(name, doc, n_in_sequence, (fields))

where fields is a list of (name,doc) tuples. You will need to
temporarily allocate a PyStructSequence_Field array of len(fields)+1
elements, and put the PyStructSequence_Desc on the stack. You will
also need to dynamically allocate a PyTypeObject which you return.

I would put this into the new module.

Regards,
Martin



From fdrake@acm.org  Thu Oct 17 19:54:06 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 17 Oct 2002 14:54:06 -0400
Subject: [Python-Dev] Re: creating struct-seq types from Python
In-Reply-To: <m3hefkudg9.fsf@mira.informatik.hu-berlin.de>
References: <15790.65087.725796.331958@grendel.zope.com>
 <m3hefkudg9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15791.1870.843775.819662@grendel.zope.com>

Martin v. Loewis writes:
 > Not really, but I think all you need to do is to expose
 > PyStructSequence_InitType. I would recommend an interface like
 > 
 > struct_seq(name, doc, n_in_sequence, (fields))

Sounds like a plan!  I've assigned this to me, since I'm the one
asking for this.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From skip@pobox.com  Thu Oct 17 19:42:48 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 17 Oct 2002 13:42:48 -0500
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <15788.51819.500992.678200@gargle.gargle.HOWL>
 <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15791.1192.105898.349109@montanaro.dyndns.org>

    BAW> Provide different field names for float values, e.g. f_mtime,
    BAW> f_ctime, f_atime, or maybe just mtime, atime, and ctime... ?

    Martin> The previous suggestion was st_mtimef, st_ctimef, ...

Why not simply provide an alternative method, os.fstat()?  Seems cleaner to
me.  The field names are hard enough to remember as it is...

Skip



From skip@pobox.com  Thu Oct 17 19:56:47 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 17 Oct 2002 13:56:47 -0500
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <15791.1192.105898.349109@montanaro.dyndns.org>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <15788.51819.500992.678200@gargle.gargle.HOWL>
 <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
 <15791.1192.105898.349109@montanaro.dyndns.org>
Message-ID: <15791.2031.219470.410776@montanaro.dyndns.org>

    BAW> Provide different field names for float values, e.g. f_mtime,
    BAW> f_ctime, f_atime, or maybe just mtime, atime, and ctime... ?

    Martin> The previous suggestion was st_mtimef, st_ctimef, ...

    Skip> Why not simply provide an alternative method, os.fstat()?  Seems
    Skip> cleaner to me.  The field names are hard enough to remember as it
    Skip> is...

Doh!  Engage brain before hitting the <send> key...  Of course there is
already an fstat() function.  I'll shut up now.

Skip



From gsw@agere.com  Thu Oct 17 23:03:39 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Thu, 17 Oct 2002 18:03:39 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFEEAGCMAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFOEAGCMAA.gsw@agere.com>

I wrote:
> But I said I was done tweaking for now. :-)

Well, I was done tweaking, but then I decided to try out one more
performance tweak. This is in desperate need of commenting, but
it's finally faster than checking via division on my machine:

#define FULL_BITS (sizeof(size_t) * 8U)
#define TOP_BIT (((size_t)1) << ((FULL_BITS)-1))
#define HALF_BITS (sizeof(size_t) * 4U)
#define MID_BIT (((size_t)1) << ((HALF_BITS)-1))
#define LO_MASK ((((size_t)1) << (HALF_BITS))-1)
#define HI_MASK (~(LO_MASK))

#define Q_BITS (sizeof(size_t) * 2U)
#define TQ_BITS (sizeof(size_t) * 6U)
#define TQ_BIT (((size_t)1) << ((TQ_BITS)-1))
#define LTQ_MASK ((((size_t)1) << (TQ_BITS))-1)
#define TQ_MASK (~(LTQ_MASK))

#define SAFE_MULTIPLY(dest,src1,src2,on_overflow)\
    {\
        size_t _x = src1;\
        size_t _y = src2;\
        size_t _dest = _x * _y;\
        \
        dest = _dest;\
        \
        if ((_x | _y) & HI_MASK)\
        {\
            if (_safe_multiply_check_for_overflow(_x,_y,_dest))\
            {\
                on_overflow;\
            }\
        }\
    }

int
_safe_multiply_check_for_overflow(
    size_t h,
    size_t l,
    size_t dest)
{
    if (l > h)
    {
        size_t temp = l;
        l = h;
        h = temp;
    }

    if ((h & HI_MASK) && (l))
    {
        size_t lgt;
        size_t leq;

        if (l & HI_MASK)
        {
            lgt = (size_t)(-1);
            leq = 0;
        }
        else
        {
            size_t hbit;
            size_t mask;

            if (h & TQ_MASK)
            {
                for ((mask=TOP_BIT),(hbit=0);!(mask&h);mask>>=1)
                {
                    ++hbit;
                }
            }
            else
            {
                for ((mask=TQ_BIT),(hbit=Q_BITS);!(mask&h);mask>>=1)
                {
                    ++hbit;
                }
            }
            leq = 1 << hbit;
            mask = leq - 1;
            lgt = ~mask ^ leq;
        }
        if ((l & lgt) || ((l & leq) && !(dest & TOP_BIT)))
        {
            return 1;
        }
    }

    return 0;
}

-Jerry


From tim.one@comcast.net  Thu Oct 17 23:29:39 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 17 Oct 2002 18:29:39 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOEAGCMAA.gsw@agere.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEFLDMAA.tim.one@comcast.net>

[Gerald S. Williams]
> Well, I was done tweaking, but then I decided to try out one more
> performance tweak. This is in desperate need of commenting, but
> it's finally faster than checking via division on my machine:
> ...

If you've actually got the energy to time these things <wink>, how about
adapting the int-mul overflow checking code in Python's intobject.c's
int_mul() function too?  The CVS (ditto 2.2.2) version

+ does rely on floating point

+ doesn't use division

+ does work for signed integral types

+ doesn't assume that the number of mantissa bits in a double
  is enough to hold a full-precision integral product

+ does assume that the largest integral product is within the
  dynamic range of a double

It could be simplified a little if it were specialized to unsigned mult.



From neal@metaslash.com  Fri Oct 18 00:01:08 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 17 Oct 2002 19:01:08 -0400
Subject: [Python-Dev] Valgrind on 2.2.2
Message-ID: <20021017230108.GL9241@epoch.metaslash.com>

I ran valgrind 1.0.3 on Python 2.2.2.  Well, almost 2.2.2, it was from
12 Oct.  After 90+ minutes and over 512 MB of RAM, there are no
new/major issues to report.

The complete valgrind log (231k) can be found here: 
        http://www.metaslash.com/py/valgrind-2_2_2.txt

I've cleaned up the log (200k) to remove some of the uninteresting stuff:
        http://www.metaslash.com/py/valgrind-clean-2_2_2.txt

There are a few small memory leaks.  I think most of the leaks are
related to threads.  The report contains memory still in use as well
as leaks.  To find memory leaks, search for ' lost ' without quotes.
It's quite possible some of the memory still in use is due to
reference leaks.

Note: one common leak is from readline.  Some of the read errors are
from GLIBC.

I had to skip test_pty, I think it caused the tests to hang.
test_commands and test_popen2 fail when running valgrind.  test_popen2
is due to valgrind output going to stdout/stderr.

Neal


From guido@python.org  Fri Oct 18 01:38:40 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 17 Oct 2002 20:38:40 -0400
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: Your message of "Thu, 17 Oct 2002 19:01:08 EDT."
 <20021017230108.GL9241@epoch.metaslash.com>
References: <20021017230108.GL9241@epoch.metaslash.com>
Message-ID: <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>

> I ran valgrind 1.0.3 on Python 2.2.2.  Well, almost 2.2.2, it was from
> 12 Oct.  After 90+ minutes and over 512 MB of RAM, there are no
> new/major issues to report.
> 
> The complete valgrind log (231k) can be found here: 
>         http://www.metaslash.com/py/valgrind-2_2_2.txt
> 
> I've cleaned up the log (200k) to remove some of the uninteresting stuff:
>         http://www.metaslash.com/py/valgrind-clean-2_2_2.txt

I looked at this one only.

> There are a few small memory leaks.  I think most of the leaks are
> related to threads.  The report contains memory still in use as well
> as leaks.  To find memory leaks, search for ' lost ' without quotes.
> It's quite possible some of the memory still in use is due to
> reference leaks.

This seems to be the most worrysome:

==28827== 268980 bytes in 15 blocks are possibly lost in loss record 100 of 106
==28827==    at 0x400487CD: realloc (vg_clientfuncs.c:270)
==28827==    by 0x80994DC: _PyObject_GC_Resize (Modules/gcmodule.c:917)
==28827==    by 0x80BD46E: PyFrame_New (Objects/frameobject.c:264)
==28827==    by 0x80782C3: PyEval_EvalCodeEx (Python/ceval.c:2390)
==28827==    by 0x807AA43: fast_function (Python/ceval.c:3173)
==28827==    by 0x80779B5: eval_frame (Python/ceval.c:2034)
==28827==    by 0x807892B: PyEval_EvalCodeEx (Python/ceval.c:2595)
==28827==    by 0x807AA43: fast_function (Python/ceval.c:3173)
==28827==    by 0x80779B5: eval_frame (Python/ceval.c:2034)
==28827==    by 0x807892B: PyEval_EvalCodeEx (Python/ceval.c:2595)

There are a few other records mentioning GC_Resize, but this one is
the biggest.  Could it be that the free frame list is botched?  OTOH,
what does "possibly lost" really mean?

There are also a few fingers pointing in the direction of weakref_ref,
e.g.

==28827== 520 bytes in 14 blocks are possibly lost in loss record 48 of 106
==28827==    at 0x400481B4: malloc (vg_clientfuncs.c:100)
==28827==    by 0x8099519: _PyObject_GC_New (Modules/gcmodule.c:868)
==28827==    by 0x8067BA5: PyWeakref_NewRef (Objects/weakrefobject.c:37)
==28827==    by 0x8066119: add_subclass (Objects/typeobject.c:2249)
==28827==    by 0x8061F29: PyType_Ready (Objects/typeobject.c:2219)
==28827==    by 0x80605A8: type_new (Objects/typeobject.c:1280)
==28827==    by 0x805EDA4: type_call (Objects/typeobject.c:183)
==28827==    by 0x80ABB0C: PyObject_Call (Objects/abstract.c:1688)
==28827==    by 0x807A34F: PyEval_CallObjectWithKeywords (Python/ceval.c:3058)
==28827==    by 0x80AB0C6: PyObject_CallFunction (Objects/abstract.c:1679)

Of course many of these could be caused by a single leak that drops a
pointer to a container -- then everything owned by that container is
also leaked.

I noticed this one:

==28713== 572 bytes in 15 blocks are possibly lost in loss record 39 of 78
==28713==    at 0x400481B4: malloc (vg_clientfuncs.c:100)
==28713==    by 0x8099519: _PyObject_GC_New (Modules/gcmodule.c:868)
==28713==    by 0x80B2D09: PyMethod_New (Objects/classobject.c:2008)
==28713==    by 0x80AF837: instance_getattr2 (Objects/classobject.c:702)
==28713==    by 0x80AF73A: instance_getattr1 (Objects/classobject.c:676)
==28713==    by 0x80B30F1: instance_getattr (Objects/classobject.c:715)
==28713==    by 0x80577A2: PyObject_GetAttr (Objects/object.c:1108)
==28713==    by 0x80B1731: half_cmp (Objects/classobject.c:1503)
==28713==    by 0x80B1937: instance_compare (Objects/classobject.c:1572)
==28713==    by 0x8055A6E: try_3way_compare (Objects/object.c:477)

which led me to an easy-to-fix leak in half_cmp(), both in 2.2.2 and 2.3:

diff -c -c -r2.154.8.1 classobject.c
*** classobject.c	13 Jun 2002 21:36:35 -0000	2.154.8.1
--- classobject.c	18 Oct 2002 00:36:06 -0000
***************
*** 1507,1514 ****
  	}
  
  	args = Py_BuildValue("(O)", w);
! 	if (args == NULL)
  		return -2;
  
  	result = PyEval_CallObject(cmp_func, args);
  	Py_DECREF(args);
--- 1507,1516 ----
  	}
  
  	args = Py_BuildValue("(O)", w);
! 	if (args == NULL) {
! 		Py_DECREF(cmp_func);
  		return -2;
+ 	}
  
  	result = PyEval_CallObject(cmp_func, args);
  	Py_DECREF(args);

but somehow I don't think that caused the report, because this exit
can only be taken if there's a memory error.  (Hm...  or if w == NULL
upon entry?  How could that happen?)

A similar one is on half_binop.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Oct 18 01:51:17 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 17 Oct 2002 20:51:17 -0400
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: Your message of "Thu, 17 Oct 2002 20:38:40 EDT."
 <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
References: <20021017230108.GL9241@epoch.metaslash.com>
 <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210180051.g9I0pH612923@pcp02138704pcs.reston01.va.comcast.net>

> There are also a few fingers pointing in the direction of weakref_ref,
> e.g.
> 
> ==28827== 520 bytes in 14 blocks are possibly lost in loss record 48 of 106
> ==28827==    at 0x400481B4: malloc (vg_clientfuncs.c:100)
> ==28827==    by 0x8099519: _PyObject_GC_New (Modules/gcmodule.c:868)
> ==28827==    by 0x8067BA5: PyWeakref_NewRef (Objects/weakrefobject.c:37)
> ==28827==    by 0x8066119: add_subclass (Objects/typeobject.c:2249)
> ==28827==    by 0x8061F29: PyType_Ready (Objects/typeobject.c:2219)
> ==28827==    by 0x80605A8: type_new (Objects/typeobject.c:1280)
> ==28827==    by 0x805EDA4: type_call (Objects/typeobject.c:183)
> ==28827==    by 0x80ABB0C: PyObject_Call (Objects/abstract.c:1688)
> ==28827==    by 0x807A34F: PyEval_CallObjectWithKeywords (Python/ceval.c:3058)
> ==28827==    by 0x80AB0C6: PyObject_CallFunction (Objects/abstract.c:1679)
> 
> Of course many of these could be caused by a single leak that drops a
> pointer to a container -- then everything owned by that container is
> also leaked.

This one seems simple:

diff -c -c -r2.126.4.25 typeobject.c
*** typeobject.c	11 Oct 2002 00:22:22 -0000	2.126.4.25
--- typeobject.c	18 Oct 2002 00:50:31 -0000
***************
*** 2249,2256 ****
  	while (--i >= 0) {
  		ref = PyList_GET_ITEM(list, i);
  		assert(PyWeakref_CheckRef(ref));
! 		if (PyWeakref_GET_OBJECT(ref) == Py_None)
! 			return PyList_SetItem(list, i, new);
  	}
  	i = PyList_Append(list, new);
  	Py_DECREF(new);
--- 2249,2259 ----
  	while (--i >= 0) {
  		ref = PyList_GET_ITEM(list, i);
  		assert(PyWeakref_CheckRef(ref));
! 		if (PyWeakref_GET_OBJECT(ref) == Py_None) {
! 			i = PyList_SetItem(list, i, new);
! 			Py_DECREF(new);
! 			return i;
! 		}
  	}
  	i = PyList_Append(list, new);
  	Py_DECREF(new);

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Fri Oct 18 09:25:13 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 18 Oct 2002 10:25:13 +0200
Subject: [Python-Dev] Incorporating bsddb3
Message-ID: <200210180825.g9I8PDK7015010@mira.informatik.hu-berlin.de>

I have changes in my sandbox to incorporate bsddb3 into Python
proper. This essentially is a copy of bsddb3 3.4.0, with the following
modifications:
- the extension module is called _bsddb (not bsddb3._db)
- the package is called bsddb (not bsddb3)

Both test_bsddb and test_anydbm continue to work unmodified.

Is it ok to commit these changes?

Regards,
Martin


From martin@v.loewis.de  Fri Oct 18 07:37:45 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Oct 2002 08:37:45 +0200
Subject: [Python-Dev] float atime/mtime/ctime - a bad idea?
In-Reply-To: <15791.1192.105898.349109@montanaro.dyndns.org>
References: <200210160047.g9G0lHx20255@pcp02138704pcs.reston01.va.comcast.net>
 <15788.51819.500992.678200@gargle.gargle.HOWL>
 <m3fzv7rkn6.fsf@mira.informatik.hu-berlin.de>
 <15791.1192.105898.349109@montanaro.dyndns.org>
Message-ID: <m3r8eos152.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

>     Martin> The previous suggestion was st_mtimef, st_ctimef, ...
> 
> Why not simply provide an alternative method, os.fstat()?

Because os.fstat already exists, and also has the option of either
returning floating point time stamps or integer time stamps - just
like os.stat and os.lstat.

Regards,
Martin


From guido@python.org  Fri Oct 18 15:03:40 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 18 Oct 2002 10:03:40 -0400
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: Your message of "Thu, 17 Oct 2002 20:51:17 EDT."
 <200210180051.g9I0pH612923@pcp02138704pcs.reston01.va.comcast.net>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
 <200210180051.g9I0pH612923@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <200210181403.g9IE3fO17121@pcp02138704pcs.reston01.va.comcast.net>

> This one seems simple:
[patch to typeobject.c]

Deceptively so -- I forgot that PyList_SetItem(list, i, new) steals a
reference to new. :-(

The search is still on...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Oct 18 23:02:44 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 18 Oct 2002 17:02:44 -0500
Subject: [Python-Dev] To -I(nclude) or not to -I(nclude), that is the question...
Message-ID: <15792.34052.962582.864797@montanaro.dyndns.org>

In installing mxBase 2.0.4 on my MacOS 10.2.1 system I get warnings like the
following:

    gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Wno-long-double -no-cpp-precomp -Imx/Queue/mxQueue -I/Users/skip/local/include/python2.3 -I/usr/include -I/usr/local/include -c mx/Queue/mxQueue/mxQueue.c -o build/temp.darwin-6.1-Power Macintosh-2.3/mx/Queue/mxQueue/mxQueue/mxQueue.o
    cc1: warning: changing search order for system directory "/usr/local/include"
    cc1: warning:   as it has already been specified as a non-system directory
    cc1: warning: changing search order for system directory "/usr/include"
    cc1: warning:   as it has already been specified as a non-system directory

This warning bothers me a bit, as it suggests I'm screwing up the compiler's
notions about header file search order.  Has anyone else seen this and
investigated how to get rid of this problem?  This is related to bug
http://python.org/sf/589427 (which was assigned to me).  It's due to the
gen_preprocess_options() function in distutils/ccompiler.py.  This warning
seems to be related to gcc version >= 3.1.

I have a quick hack in my local copy of distutils/ccompiler.py.  At the
bottom of gen_preprocess_options() I replaced

    for dir in include_dirs:
        pp_opts.append ("-I%s" % dir)

with 

    pp_opts.extend (gen_preprocess_includes(include_dirs))

and added these two functions to the file:

    def gen_preprocess_includes_macosx_gcc(dirs):
        """GCC on MacOSX complains if /usr/include or /usr/local/include are
        mentioned in -I.
        """
        pp_opts = []
        for dir in dirs:
            if dir not in ("/usr/include", "/usr/local/include"):
                pp_opts.append ("-I%s" % dir)
        return pp_opts

    def gen_preprocess_includes(dirs):
        """Generate the -I flags for a compile command."""
        if sys.platform == "darwin":
            return gen_preprocess_includes_macosx_gcc(dirs)

        pp_opts = []
        for dir in dirs:
            pp_opts.append ("-I%s" % dir)
        return pp_opts

This is an obscure solution, at best.  I'd prefer to at least test the
compiler and version.  How can I tell what compiler and version will be used
to compile files, or can't I can this level?  (Seems to me that
gen_preprocess_options begs to be a method of the CCompiler class.)

Thx,

Skip


From barry@python.org  Fri Oct 18 23:10:55 2002
From: barry@python.org (Barry A. Warsaw)
Date: Fri, 18 Oct 2002 18:10:55 -0400
Subject: [Python-Dev] To -I(nclude) or not to -I(nclude), that is the question...
References: <15792.34052.962582.864797@montanaro.dyndns.org>
Message-ID: <15792.34543.263408.609946@gargle.gargle.HOWL>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> This warning bothers me a bit, as it suggests I'm screwing up
    SM> the compiler's notions about header file search order.  Has
    SM> anyone else seen this and investigated how to get rid of this
    SM> problem?  This is related to bug http://python.org/sf/589427
    SM> (which was assigned to me).  It's due to the
    SM> gen_preprocess_options() function in distutils/ccompiler.py.
    SM> This warning seems to be related to gcc version >= 3.1.

I seem to remember that Martin said the gcc folks agreed that this
behavior was a bug on their part.  I tried building Python on Linux
with gcc 3.1 several months ago and had the same problems.  I dropped
back to 2.96 and haven't looked back. :)

Search around in python-dev for a discussion on this, but
unfortunately I don't remember the exact time frame.

-Barry


From brian@sweetapp.com  Sat Oct 19 01:10:55 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 17:10:55 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <15792.34543.263408.609946@gargle.gargle.HOWL>
Message-ID: <000401c27703$fe5cd860$21795418@dell1700>

1. Is there any reason why builtin methods cannot be proxied? 

2. It would be handy for my application if a callback could be triggered
   when an object has no more weak references attached to it. 

It seems like my application could be a fairly common one:

# C library pseudocode
def c_library_func(): # C code
    while 1:
        o = create_complex_object()
	  user_call_back(o)
        del o

# Python bindings pseudocode
def python_bindings_user_call_back (o): # C code
    py_o = create_python_wrapper_object(o)
    proxy = PyWeakref_NewProxy(py_o)
    python_function(py_o)
    Py_DECREF(proxy)
    Py_DECREF(py_o) # This will kill the proxy


# Evil Python user code
evil = None
def python_function(o):
	global evil
	o.foo()
	evil = o

start(python_function)
evil.foo() # Nice exception because evil is a dead proxy

# More evil Python user code

more_evil = None
def python_function(o):
	global more_evil
	o.foo()
	more_evil = o.foo

start(python_function)
more_evil() # Crash because the underlying data structures that
		# the Python wrapper object depends on are dead 

My current solution to this problem is to create my own callable object
type that supports weakrefs. That object is then used to wrap the real
bound method object e.g.

def getattr(self, name): # This is C code
	callable = MyCallAble_New(Py_FindMethod(...);
	objects_to_kill_after_py_func_call.add(callable);
	return PyWeakref_NewProxy(callable);

Avoiding this hassle is the reason for my question.  

Pruning callable objects that the user is done with is the reason for my
request.

Cheers,
Brian



From guido@python.org  Sat Oct 19 01:23:50 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 18 Oct 2002 20:23:50 -0400
Subject: [Python-Dev] Weakref design questions
In-Reply-To: Your message of "Fri, 18 Oct 2002 17:10:55 PDT."
 <000401c27703$fe5cd860$21795418@dell1700>
References: <000401c27703$fe5cd860$21795418@dell1700>
Message-ID: <200210190023.g9J0Nou25728@pcp02138704pcs.reston01.va.comcast.net>

> 1. Is there any reason why builtin methods cannot be proxied? 

The instances of a type need to have a pointer added if the type is to
support weak refs to its instances.  We've only done this for the
obvious candidates like user-defined class instances and a few
others.  You must weigh the cost of the extra pointer vs. the utility
of being able to use weak refs for a particular type.

> 2. It would be handy for my application if a callback could be triggered
>    when an object has no more weak references attached to it. 
> 
> It seems like my application could be a fairly common one:
> 
> # C library pseudocode
> def c_library_func(): # C code
>     while 1:
>         o = create_complex_object()
> 	  user_call_back(o)
>         del o
> 
> # Python bindings pseudocode
> def python_bindings_user_call_back (o): # C code
>     py_o = create_python_wrapper_object(o)
>     proxy = PyWeakref_NewProxy(py_o)
>     python_function(py_o)
>     Py_DECREF(proxy)
>     Py_DECREF(py_o) # This will kill the proxy
> 
> 
> # Evil Python user code
> evil = None
> def python_function(o):
> 	global evil
> 	o.foo()
> 	evil = o
> 
> start(python_function)
> evil.foo() # Nice exception because evil is a dead proxy
> 
> # More evil Python user code
> 
> more_evil = None
> def python_function(o):
> 	global more_evil
> 	o.foo()
> 	more_evil = o.foo
> 
> start(python_function)
> more_evil() # Crash because the underlying data structures that
> 		# the Python wrapper object depends on are dead 
> 
> My current solution to this problem is to create my own callable object
> type that supports weakrefs. That object is then used to wrap the real
> bound method object e.g.
> 
> def getattr(self, name): # This is C code
> 	callable = MyCallAble_New(Py_FindMethod(...);
> 	objects_to_kill_after_py_func_call.add(callable);
> 	return PyWeakref_NewProxy(callable);
> 
> Avoiding this hassle is the reason for my question.  
> 
> Pruning callable objects that the user is done with is the reason for my
> request.

I'm sorry, but I don't understand why you're using weak references
here at all.  Is it really the proxy function that you're after?  If
you're worried about the complex object 'o' being referenced after the
C library has killed it (a ligitimate concern), while the Python
wrapper can be kept alive using the scenario you show, the Python
wrapper py_o should set its pointer to 'o' to NULL when o's lifetime
ends (or a little before) and make sure that methods on the wrapper
raise an exception when the reference is NULL.

This is how all well-behaved wrapper objects behave; e.g. file and
socket objects in Python check if the file is closed in each method
implementation.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From dave@boost-consulting.com  Sat Oct 19 01:39:04 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 18 Oct 2002 20:39:04 -0400
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000401c27703$fe5cd860$21795418@dell1700>
References: <000401c27703$fe5cd860$21795418@dell1700>
Message-ID: <u8z0vntxz.fsf@boost-consulting.com>

Brian Quinlan <brian@sweetapp.com> writes:

> 1. Is there any reason why builtin methods cannot be proxied? 
> 
> 2. It would be handy for my application if a callback could be triggered
>    when an object has no more weak references attached to it. 
> 
> It seems like my application could be a fairly common one:
> 
> # C library pseudocode
> def c_library_func(): # C code
>     while 1:
>         o = create_complex_object()
> 	  user_call_back(o)
>         del o
> 
> # Python bindings pseudocode
> def python_bindings_user_call_back (o): # C code
>     py_o = create_python_wrapper_object(o)
>     proxy = PyWeakref_NewProxy(py_o)
>     python_function(py_o)
>     Py_DECREF(proxy)
>     Py_DECREF(py_o) # This will kill the proxy

What's the point of the proxy? You never use it for anything. I guess
you must've meant:

     python_function(proxy)

above.

> # Evil Python user code
> evil = None
> def python_function(o):
> 	global evil
> 	o.foo()
> 	evil = o
> 
> start(python_function)
> evil.foo() # Nice exception because evil is a dead proxy

Hum. In Boost.Python the default is to copy the C++ object when
passing it to a Python callback, though it's possible for the user to
explicitly say "just build a Python object around a reference to the
C++ object -- Python code beware of lifetime issues". I guess I like
your idea, though, as a safer alternative for non-copyable or for very
expensive-to-copy C++ objects.

> # More evil Python user code
> 
> more_evil = None
> def python_function(o):
> 	global more_evil
> 	o.foo()
> 	more_evil = o.foo
> 
> start(python_function)
> more_evil() # Crash because the underlying data structures that
> 		# the Python wrapper object depends on are dead 

Hum. 

  >>> class userlist(list): pass # just to get weakref ability
  ...
  >>> l = userlist() + [ 1, 2, 3 ]
  >>> type(l)        # Interesting (buggy?) behavior with 2.2.1
  <type 'list'>
  >>> l = userlist() # Try again
  >>> l += [1,2,3]
  >>> type(l)        # Better
  <class '__main__.userlist'>
  >>> p = weakref.proxy(l)
  >>> a = p.append
  >>> p
  [1, 2, 3]
  >>> a
  <built-in method append of userlist object at 0x0093F328>
  >>> del l
  >>> p              # The list is still alive!
  [1, 2, 3]
  >>> a              # Because it (not the proxy) is bound into a
  <built-in method append of userlist object at 0x0093F328>
  >>> a(2)           # a works without crashing
  >>> p
  [1, 2, 3, 2]

How does this differ from what's happening for you?

> Pruning callable objects that the user is done with is the reason
> for my request.

Hmm, it kinda seems like you want to prune some of them before the
user's done with them. Don't users expect objects to stay alive when
they hold references?

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html



From brian@sweetapp.com  Sat Oct 19 02:00:28 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 18:00:28 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <200210190023.g9J0Nou25728@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <000501c2770a$ea8fb4e0$21795418@dell1700>

Guido van Rossum wrote:
> The instances of a type need to have a pointer added if the 
> type is to support weak refs to its instances.  We've only 
> done this for the obvious candidates like user-defined class 
> instances and a few others.  You must weigh the cost of the 
> extra pointer vs. the utility of being able to use weak refs 
> for a particular type.

Fair enough. 

> I'm sorry, but I don't understand why you're using weak references
> here at all.  Is it really the proxy function that you're after? 

I don't understand the question. 

> If
> you're worried about the complex object 'o' being referenced after the
> C library has killed it (a ligitimate concern), while the Python
> wrapper can be kept alive using the scenario you show, the Python
> wrapper py_o should set its pointer to 'o' to NULL when o's lifetime
> ends (or a little before) and make sure that methods on the wrapper
> raise an exception when the reference is NULL.

That is definitely one possible way to do it. However, I am wrapping a
complete DOM, with dozens of objects containing, collectively, hundreds
of methods. 

Adding an explicit check to each method seemed like a lot more pain than
using proxies.

Avoiding explicit checks also offers a potential performance advantage
because sometimes the objects are owned and no checking is required. In
that case I simply return the object directly without using a proxy.

> This is how all well-behaved wrapper objects behave; e.g. file and
> socket objects in Python check if the file is closed in each method
> implementation.

They contain far fewer methods. Also, there is more centralization is
file and socket objects i.e. they know themselves if they are in a valid
state or not, this is not true of my DOM objects.

Cheers,
Brian



From brian@sweetapp.com  Sat Oct 19 02:21:23 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 18:21:23 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <u8z0vntxz.fsf@boost-consulting.com>
Message-ID: <000601c2770d$d6c80c70$21795418@dell1700>

David Abrahams:
> What's the point of the proxy? You never use it for anything. I guess
> you must've meant:

>     python_function(proxy)

> above.

<blush> Nice catch.

> Hum. In Boost.Python the default is to copy the C++ object when
> passing it to a Python callback, though it's possible for the user
> to explicitly say "just build a Python object around a reference to 
> the C++ object -- Python code beware of lifetime issues". I guess I 
> like your idea, though, as a safer alternative for non-copyable or 
> for very expensive-to-copy C++ objects.

DOMs, the data structures in question, can be obscenely expensive to
copy.

And I firmly believe that Python code should never be able to cause a
crash.

[nice demonstration deleted]
> How does this differ from what's happening for you?

When you created the bound method p.append, you caused the reference
count of p to be increased. 

In my case, since Python is not capable of altering the lifetime of p
(it's dead with the C library says that it is dead), this is a bad
thing.

So I proxy the bound methods and kill them when p dies.

> Hmm, it kinda seems like you want to prune some of them before the
> user's done with them. 

I actually presented two different desires at once and know I'm paying
the price. 

There are two reasons that I might want to delete a proxied object:

1. Its lifetime is over and I don't want anyone to access it
2. There are no proxy objects that refer to it, making it inaccessible
(since I hold the only actual reference)

Reason #1 is why I could like builtin methods to support weakrefs.
Reason #2 is why I would like a callback when the last weakref dies.

> Don't users expect objects to stay alive when they hold references?

Probably. But that is not always an option.

Cheers,
Brian



From dave@boost-consulting.com  Sat Oct 19 02:42:28 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 18 Oct 2002 21:42:28 -0400
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000601c2770d$d6c80c70$21795418@dell1700>
References: <000601c2770d$d6c80c70$21795418@dell1700>
Message-ID: <u3cr3nr0b.fsf@boost-consulting.com>

Brian Quinlan <brian@sweetapp.com> writes:

> David Abrahams:
> > What's the point of the proxy? You never use it for anything. I guess
> > you must've meant:
> 
> >     python_function(proxy)
> 
> > above.
> 
> <blush> Nice catch.
> 
> > Hum. In Boost.Python the default is to copy the C++ object when
> > passing it to a Python callback, though it's possible for the user
> > to explicitly say "just build a Python object around a reference to 
> > the C++ object -- Python code beware of lifetime issues". I guess I 
> > like your idea, though, as a safer alternative for non-copyable or 
> > for very expensive-to-copy C++ objects.
> 
> DOMs, the data structures in question, can be obscenely expensive to
> copy.

This may not be possible for you, but in C++ That would generally be a
"handle" class with reference semantics (using a reference-counted
smart pointer, for example) so that "copying" the DOM really just
amounted to bumping a reference count somewhere.

> And I firmly believe that Python code should never be able to cause a
> crash.

Me too, but some of my users insist on being able to trade safety for
performance.

> [nice demonstration deleted]
> > How does this differ from what's happening for you?
> 
> When you created the bound method p.append, you caused the reference
> count of p to be increased. 

I think I caused the reference count of l to be increased. AFAIK, p
isn't keeping the list alive.

> In my case, since Python is not capable of altering the lifetime of p
> (it's dead with the C library says that it is dead), this is a bad
> thing.

Yeah, I see that now.

> So I proxy the bound methods and kill them when p dies.

I don't quite understand what "proxy the bound methods" means.

> > Hmm, it kinda seems like you want to prune some of them before the
> > user's done with them.
> 
> I actually presented two different desires at once and know I'm paying
> the price. 
> 
> There are two reasons that I might want to delete a proxied object:
> 
> 1. Its lifetime is over and I don't want anyone to access it

OK.

> 2. There are no proxy objects that refer to it, making it inaccessible
> (since I hold the only actual reference)
> 
> Reason #1 is why I could like builtin methods to support weakrefs.
> Reason #2 is why I would like a callback when the last weakref dies.

So you want a weakrefref. Well, I think this problem can be solved by
applying the Fundamental Theorem of Software Engineering: apply an
extra level of indirection. But Guido's approach of having the wrapper
methods check for a null DOM* seems like a reasonable one to
me. Incidentally, that's what you automatically get from Boost.Python
when your object is held by std::auto_ptr<T>, which is a smart pointer
class that can release ownership. I think I want to do something
similar for boost::weak_ptr, but I haven't gotten around to it yet.

> > Don't users expect objects to stay alive when they hold references?
> 
> Probably. But that is not always an option.

That's clear to me now. Very interesting thread; thanks for posting it
here!

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html



From brian@sweetapp.com  Sat Oct 19 03:51:18 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 19:51:18 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <u3cr3nr0b.fsf@boost-consulting.com>
Message-ID: <000c01c2771a$66023170$21795418@dell1700>

David Abrahams:
> > So I proxy the bound methods and kill them when p dies.

> I don't quite understand what "proxy the bound methods" means.

Most extension types have a function like this:

PyObject *
MyObject_getattr(MyObject*    self, 
                 char *       name)
{    
	/* check for attributes */
	...

      /* ok, not an attribute, now check the methods
	return Py_FindMethod(
            MyObject_methods, 
            (PyObject *) self, 
            name);
}

Py_FindMethod() returns a PyCFunctionObject. The PyCFunctionObject will
own a reference to the MyObject "self". 

Since "self" has a limited lifetime this would be bad. So we could do
this:

PyObject *
MyObject_getattr(MyObject*    self, 
                 char *       name)
{    
	/* check for attributes */
	...

      /* ok, not an attribute, now check the methods
	method = Py_FindMethod(
            MyObject_methods, 
            (PyObject *) self, 
            name);

	if (method != NULL) {
		add_to_list_of_objects_to_kill(method)
		return PyWeakref_NewProxy(method);
	}

	return NULL;
}

But we can't quite do this because builtin functions are not proxyable.

> So you want a weakrefref. Well, I think this problem can be solved 
> by applying the Fundamental Theorem of Software Engineering: apply 
> an extra level of indirection. 

That's what I do. I was just wondering if there is any reason not to
make C functions weakref/proxyable.

> But Guido's approach of having the wrapper methods check for a null
DOM*
> seems like a reasonable one to me. 

It's not unreasonable and it is a bit simpler. But it is more work and
reduces performance slightly in the case where no check need be
performed i.e. the object is owned by Python.

> That's clear to me now. Very interesting thread; thanks for posting 
> it here!

Yeah, my first non-stupid post to python-dev.

Cheers,
Brian



From martin@v.loewis.de  Sat Oct 19 04:39:09 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Oct 2002 05:39:09 +0200
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000501c2770a$ea8fb4e0$21795418@dell1700>
References: <000501c2770a$ea8fb4e0$21795418@dell1700>
Message-ID: <m34rbjdrmq.fsf@mira.informatik.hu-berlin.de>

Brian Quinlan <brian@sweetapp.com> writes:

> That is definitely one possible way to do it. However, I am wrapping a
> complete DOM, with dozens of objects containing, collectively, hundreds
> of methods. 
> 
> Adding an explicit check to each method seemed like a lot more pain than
> using proxies.

You don't have to add it to every method. You can perform the check in
tp_getattro before performing the method lookup. Alternatively, you
can change the ob_type of the object to simply drop the methods that
are not available anymore.

Regards,
Martin


From brian@sweetapp.com  Sat Oct 19 04:54:45 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 20:54:45 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <m34rbjdrmq.fsf@mira.informatik.hu-berlin.de>
Message-ID: <000d01c27723$432da220$21795418@dell1700>

Martin wrote:
> You don't have to add it to every method. You can perform the check in
> tp_getattro before performing the method lookup. 

That would be dangerous! See my original "more evil" example. 

> Alternatively, you can change the ob_type of the object to simply drop

> the methods that are not available anymore.

I like this strategy! But I still think that this is more painful/less
elegant than using proxies.

Cheers,
Brian



From dave@boost-consulting.com  Sat Oct 19 04:46:45 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 18 Oct 2002 23:46:45 -0400
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000d01c27723$432da220$21795418@dell1700>
References: <000d01c27723$432da220$21795418@dell1700>
Message-ID: <uiszzm6oq.fsf@boost-consulting.com>

Brian Quinlan <brian@sweetapp.com> writes:

> Martin wrote:
> > You don't have to add it to every method. You can perform the check in
> > tp_getattro before performing the method lookup. 
> 
> That would be dangerous! See my original "more evil" example. 

Yeah, doesn't work for that case.

> > Alternatively, you can change the ob_type of the object to simply drop
> 
> > the methods that are not available anymore.
> 
> I like this strategy! But I still think that this is more painful/less
> elegant than using proxies.

I don't see how that works either. If you have two objects of the same
type, they may die at different times. If the type drops its methods
all the objects become disabled. Furthermore, I think it still doesn't
help with "more evil", since nobody's touching the type at that point
- the method has already been looked up and kept alive by binding it
to the underlying object.

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html



From brian@sweetapp.com  Sat Oct 19 05:03:10 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 21:03:10 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000d01c27723$432da220$21795418@dell1700>
Message-ID: <000e01c27724$7098df30$21795418@dell1700>

Brian Quinlan wrote:
>> Alternatively, you can change the ob_type of the object to simply 
>> drop the methods that are not available anymore.

> I like this strategy! But I still think that this is more 
> painful/less elegant than using proxies.

I'm an idiot. This won't work either for the same reason that I gave
before.

Cheers,
Brian



From martin@v.loewis.de  Sat Oct 19 05:18:12 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Oct 2002 06:18:12 +0200
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <000401c27703$fe5cd860$21795418@dell1700>
References: <000401c27703$fe5cd860$21795418@dell1700>
Message-ID: <m3of9rcb97.fsf@mira.informatik.hu-berlin.de>

Brian Quinlan <brian@sweetapp.com> writes:

> My current solution to this problem is to create my own callable object
> type that supports weakrefs. That object is then used to wrap the real
> bound method object e.g.
> 
> def getattr(self, name): # This is C code
> 	callable = MyCallAble_New(Py_FindMethod(...);
> 	objects_to_kill_after_py_func_call.add(callable);
> 	return PyWeakref_NewProxy(callable);
> 
> Avoiding this hassle is the reason for my question.  

I notice that you don't need the Weakref property of those proxies at
all: You know precisely the set of all objects to consider when the
underlying C object has gone away.

So you merely want a proxy, not a weak proxy: both for the entire
object, and for the methods. So for this code, you can save the proxy,
and return your callable object. Make the null pointer check in its
tp_call slot, and don't kill it after py func call, but merely clear
the pointer.

I also notice that you rely on the fact that Python code has no way to
find out the underlying object of a weak proxy. I think this is a weak
assumption - there is no guarantee that this is not possible, or might
not be possible in the future.

Regards,
Martin


From brian@sweetapp.com  Sat Oct 19 05:44:39 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 18 Oct 2002 21:44:39 -0700
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <m3of9rcb97.fsf@mira.informatik.hu-berlin.de>
Message-ID: <001101c2772a$3bef5e70$21795418@dell1700>

Martin wrote:

> So you merely want a proxy, not a weak proxy: both for the 
> entire object, and for the methods. So for this code, you 
> can save the proxy, and return your callable object. Make 
> the null pointer check in its tp_call slot, and don't kill 
> it after py func call, but merely clear the pointer.

As you say, I need a proxy for my methods and a proxy for my objects. 

Creating my own proxy type for methods is easy, since the only
interesting thing that must be proxied is __call__.

Creating my own proxy type for arbitrary objects is harder because I
must create a proxy for every slot that I use. That requires duplication
a lot of the work that already lives in weakrefobject.c

Also, for both method and object proxies, I have to invent a mechanism
to signal that the original object is dead. weakrefobject.c already
defines a nice mechanism.

> I also notice that you rely on the fact that Python code has no way 
> to find out the underlying object of a weak proxy. I think this is a
weak
> assumption - there is no guarantee that this is not possible, or might
> not be possible in the future.

This is a concern. If I am mistaken in my assumption, please let me know
and I will either invent my own proxy type or use the
check-on-every-method-call technique.

Cheers,
Brian



From martin@v.loewis.de  Sat Oct 19 07:37:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Oct 2002 08:37:04 +0200
Subject: [Python-Dev] Weakref design questions
In-Reply-To: <001101c2772a$3bef5e70$21795418@dell1700>
References: <001101c2772a$3bef5e70$21795418@dell1700>
Message-ID: <m3bs5rc4tr.fsf@mira.informatik.hu-berlin.de>

Brian Quinlan <brian@sweetapp.com> writes:

> > I also notice that you rely on the fact that Python code has no
> > way to find out the underlying object of a weak proxy. I think
> > this is a weak assumption - there is no guarantee that this is not
> > possible, or might not be possible in the future.

> This is a concern. If I am mistaken in my assumption, please let me know
> and I will either invent my own proxy type or use the
> check-on-every-method-call technique.

I'm not exactly sure what your assumption is. If it is "in Python 2.3,
there is no way to unwrap a proxy except by writing an extension
module", then your assumption is correct.

If your assumption is "there is a guarantee that there never will be
such mechanism", your assumption is incorrect.

Regards,
Martin



From mal@lemburg.com  Sat Oct 19 11:12:34 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 19 Oct 2002 12:12:34 +0200
Subject: [Python-Dev] Re: Moving to Unicode 3.2 ([Python-checkins] python/dist/src/Objects
 unicodectype.c,2.11,2.12	unicodetype_db.h,1.4,1.5)
References: <E182ZjA-0002ii-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <3DB13012.3040604@lemburg.com>

loewis@users.sourceforge.net wrote:
 > Update of /cvsroot/python/python/dist/src/Objects
 > In directory usw-pr-cvs1:/tmp/cvs-serv10274/Objects
 >
 > Modified Files:
 > 	unicodectype.c unicodetype_db.h
 > Log Message:
 > Update to Unicode 3.2 database.

I haven't seen any messages about this on python-dev. Did I miss
something ?

The switch from Unicode 3.0 is a big one since 3.2 introduces
non-BMP character points for the first time.

I also don't think that it is a good idea to ship the Unicode
3.2 database while the code behaves as defined in Unicode 3.0.

And last not least, I'd like to be asked before you make such
changes.

Thanks,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mal@lemburg.com  Sat Oct 19 11:18:43 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 19 Oct 2002 12:18:43 +0200
Subject: [Python-Dev] To -I(nclude) or not to -I(nclude), that is the
 question...
References: <15792.34052.962582.864797@montanaro.dyndns.org>
Message-ID: <3DB13183.9000206@lemburg.com>

Skip Montanaro wrote:
> In installing mxBase 2.0.4 on my MacOS 10.2.1 system I get warnings like the
> following:
> 
>     gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Wno-long-double -no-cpp-precomp -Imx/Queue/mxQueue -I/Users/skip/local/include/python2.3 -I/usr/include -I/usr/local/include -c mx/Queue/mxQueue/mxQueue.c -o build/temp.darwin-6.1-Power Macintosh-2.3/mx/Queue/mxQueue/mxQueue/mxQueue.o
>     cc1: warning: changing search order for system directory "/usr/local/include"
>     cc1: warning:   as it has already been specified as a non-system directory
>     cc1: warning: changing search order for system directory "/usr/include"
>     cc1: warning:   as it has already been specified as a non-system directory
> 
> This warning bothers me a bit, as it suggests I'm screwing up the compiler's
> notions about header file search order.  Has anyone else seen this and
> investigated how to get rid of this problem? 

I've had a few bug reports related to this, but all of them
were from Solaris users. The -I/usr/include causes the GCC compiler
to pick up a system stdarg.h header file which causes a compile
error (GCC ships with its own stdarg.h files).

The report for MacOS is new, though. Perhaps this is a generic GCC problem ?
(#include <stdarg.h> should look in the compiler dirs first and only
then scan the additional -I paths)

> This is related to bug
> http://python.org/sf/589427 (which was assigned to me).  It's due to the
> gen_preprocess_options() function in distutils/ccompiler.py.  This warning
> seems to be related to gcc version >= 3.1.
> 
> I have a quick hack in my local copy of distutils/ccompiler.py.  At the
> bottom of gen_preprocess_options() I replaced
> 
>     for dir in include_dirs:
>         pp_opts.append ("-I%s" % dir)
> 
> with 
> 
>     pp_opts.extend (gen_preprocess_includes(include_dirs))
> 
> and added these two functions to the file:
> 
>     def gen_preprocess_includes_macosx_gcc(dirs):
>         """GCC on MacOSX complains if /usr/include or /usr/local/include are
>         mentioned in -I.
>         """
>         pp_opts = []
>         for dir in dirs:
>             if dir not in ("/usr/include", "/usr/local/include"):
>                 pp_opts.append ("-I%s" % dir)
>         return pp_opts
> 
>     def gen_preprocess_includes(dirs):
>         """Generate the -I flags for a compile command."""
>         if sys.platform == "darwin":
>             return gen_preprocess_includes_macosx_gcc(dirs)
> 
>         pp_opts = []
>         for dir in dirs:
>             pp_opts.append ("-I%s" % dir)
>         return pp_opts
> 
> This is an obscure solution, at best.  I'd prefer to at least test the
> compiler and version.  How can I tell what compiler and version will be used
> to compile files, or can't I can this level?  (Seems to me that
> gen_preprocess_options begs to be a method of the CCompiler class.)
> 
> Thx,
> 
> Skip
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Sat Oct 19 11:54:44 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 19 Oct 2002 06:54:44 -0400
Subject: [Python-Dev] Weakref design questions
In-Reply-To: Your message of "Sat, 19 Oct 2002 05:39:09 +0200."
 <m34rbjdrmq.fsf@mira.informatik.hu-berlin.de>
References: <000501c2770a$ea8fb4e0$21795418@dell1700>
 <m34rbjdrmq.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210191054.g9JAsiG26920@pcp02138704pcs.reston01.va.comcast.net>

[Brian]
> > That is definitely one possible way to do it. However, I am
> > wrapping a complete DOM, with dozens of objects containing,
> > collectively, hundreds of methods.
> > 
> > Adding an explicit check to each method seemed like a lot more
> > pain than using proxies.

[Martin]
> You don't have to add it to every method. You can perform the check in
> tp_getattro before performing the method lookup. Alternatively, you
> can change the ob_type of the object to simply drop the methods that
> are not available anymore.

No, his second example defeated that:

  def evil_python_callback(o):
     global evil_method
     evil_method = o.meth

  def called_later():
     evil_method() # SegFault

This does the getattr when it's allowed, but makes the call later.

To Brian: I really don't want to make bound methods proxyable.  (And
it wouldn't help you anyway until Python 2.3 is released.)  If you
really don't want to modify each method, you can write your own
callable type and use that to wrap the methods; this can then check
whether the object is still alive before passing on the call.  If you
think this is too expensive, go back to checking in each method --
since a check is needed anyway, you can't get cheaper than that.  (A
weakref proxy isn't free either.)  To signal invalidity, the Python
wrapper would have to set the 'o' pointer to NULL.

I also agree with Martin's observation that weakref proxies aren't
intended to make the original object undiscoverable.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Sat Oct 19 16:06:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Oct 2002 17:06:43 +0200
Subject: [Python-Dev] Re: Moving to Unicode 3.2
In-Reply-To: <3DB13012.3040604@lemburg.com>
References: <E182ZjA-0002ii-00@usw-pr-cvs1.sourceforge.net>
 <3DB13012.3040604@lemburg.com>
Message-ID: <m3of9qwjr0.fsf_-_@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> I haven't seen any messages about this on python-dev. Did I miss
> something ?

No. For a change like this, I did not think consultation was
necessary.

> The switch from Unicode 3.0 is a big one since 3.2 introduces
> non-BMP character points for the first time.

I disagree; it's a small change. Just look at the patch itself: apart
from the (considerably large) generated data, there were very few
actual changes to source code: Changing a few limits was sufficient.

Since there are no backwards compatibility issues, and no design
choices (apart from the choice of updating the database at all), this
is a straight-forward change.

> I also don't think that it is a good idea to ship the Unicode
> 3.2 database while the code behaves as defined in Unicode 3.0.

Can you please elaborate? What code behaves as defined in Unicode 3.0
that is incompatible with the Unicode 3.2 database?

> And last not least, I'd like to be asked before you make such
> changes.

I find this quite a possessive view, and I would prefer if you bring
up technical arguments instead of procedural ones, but ok...

I was under the impression that I can apply my own professional
judgement when deciding what patches to apply without consultation, in
what cases to ask on python-dev, and when to submit a patch to SF.

Apparently, this impression is wrong. Can you please give precise
instructions what constitutes "such a change"?

Also, should I back this change out?

Regards,
Martin



From guido@python.org  Sat Oct 19 16:42:12 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 19 Oct 2002 11:42:12 -0400
Subject: [Python-Dev] Gone for a week
Message-ID: <200210191542.g9JFgDp28012@pcp02138704pcs.reston01.va.comcast.net>

I'm going to disappear for a week; back on the 28th.  I'll be on a
cruise ship teaching Python and Zope.  (Can you say boondoggle?  It's
spelled www.geekcruises.com. :-)

They've threatened to hook us up with internet access to maybe I won't
be offline completely, but I expect to be checking for email rarely.
The rest of the PL crew is at your service, of course!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Sun Oct 20 02:55:19 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Sat, 19 Oct 2002 21:55:19 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCEFLDMAA.tim.one@comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEALCMAA.gsw@agere.com>

Tim Peters wrote:
> If you've actually got the energy to time these things <wink>, how about
> adapting the int-mul overflow checking code in Python's intobject.c's
> int_mul() function too?

I'll take a look.

> It could be simplified a little if it were specialized to unsigned mult.

You're simply referring to breaking down the operation into "remove
sign", multiply, and "generate sign" stages, right? I'll definitely
keep it in mind.

-Jerry Williams


From tim.one@comcast.net  Sun Oct 20 04:39:08 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 19 Oct 2002 23:39:08 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFGEALCMAA.gsw@agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENEBMAB.tim.one@comcast.net>

[Tim\
>> If you've actually got the energy to time these things <wink>, how
>> about adapting the int-mul overflow checking code in Python's
>> intobject.c's int_mul() function too?

[Gerald S. Williams]
> I'll take a look.

Thanks!

>> It could be simplified a little if it were specialized to unsigned
>> mult.

> You're simply referring to breaking down the operation into "remove
> sign", multiply, and "generate sign" stages, right? I'll definitely
> keep it in mind.

Goodness no -- that kind of cleverness is usually buggy, and usually by
failing to account for that the moral equivalent of -sys.maxint-1 is a fine
product, but sys.maxint+1 isn't.  I was talking specifically about the
based-on-doubles int_mul().  In particular, the slow-path line

		const double absprod = doubleprod >= 0.0 ? doubleprod :
							  -doubleprod;

isn't needed if we know the product is >= 0.



From skip@manatee.mojam.com  Sun Oct 20 13:00:17 2002
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 20 Oct 2002 07:00:17 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200210201200.g9KC0H6L019011@manatee.mojam.com>

Bug/Patch Summary
-----------------

316 open / 2963 total bugs (+4)
92 open / 1733 total patches (-4)

New Bugs
--------

textwrap fails on unicode using defaults (2002-10-13)
	http://python.org/sf/622831
inconsistent results of leading whitespace in textwrap input (2002-10-13)
	http://python.org/sf/622849
urllib.pathname2url(), .url2pathname() (2002-10-14)
	http://python.org/sf/623196
tempfile crashes (2002-10-15)
	http://python.org/sf/623464
__rdiv__ vs new-style classes (2002-10-15)
	http://python.org/sf/623669
Broken link in local documentation. (2002-10-16)
	http://python.org/sf/624024
email 2.4.3 pkg mail header error (2002-10-16)
	http://python.org/sf/624254
sq_concat prevents __radd__ from working (2002-10-17)
	http://python.org/sf/624807
Creation of struct_seq types (2002-10-17)
	http://python.org/sf/624827
help(UNICODE) -- reference page missing (2002-10-17)
	http://python.org/sf/624860
pydoc shows tools/scripts but blows up (2002-10-17)
	http://python.org/sf/624893
Typo at line 351 of email/Charset.py (2002-10-18)
	http://python.org/sf/625509
Errors with recursive objects (2002-10-19)
	http://python.org/sf/625698
Reorganize MacPython resources on OSX (2002-10-19)
	http://python.org/sf/625725
sys.executable in applets (2002-10-19)
	http://python.org/sf/625728
IDE starts in / on OSX (2002-10-19)
	http://python.org/sf/625734
missing names in telnetlib.py (2002-10-19)
	http://python.org/sf/625823

New Patches
-----------

2.2 patches for BSD/OS 5.0 (2002-03-26)
	http://python.org/sf/535335
autoconf-2.5X tweaks for configure.in (2002-10-15)
	http://python.org/sf/623771
HTTP Auth support for xmlrpclib (2002-10-16)
	http://python.org/sf/624180
attributes for urlsplit, urlparse result (2002-10-16)
	http://python.org/sf/624325
sets.BaseSet.isdisjointset(other) (2002-10-18)
	http://python.org/sf/625513

Closed Bugs
-----------

Broken \ref link in documentation (2000-10-18)
	http://python.org/sf/217195
bad links at Ref Guide (2001-11-23)
	http://python.org/sf/484967
re.finditer (2002-07-24)
	http://python.org/sf/585882
list(xrange(sys.maxint / 4)) -> swapping (2002-09-28)
	http://python.org/sf/616019
segmentation fault importing huge source (2002-10-07)
	http://python.org/sf/619789
__ipow__ broken for 'object's (2002-10-08)
	http://python.org/sf/620179
core setup.py fails on Windows (2002-10-08)
	http://python.org/sf/620364
websucker relative-URL errors (2002-10-09)
	http://python.org/sf/620705
SocketServer/socket allow_reuse_address (2002-10-09)
	http://python.org/sf/620783
email: ASCII decoding error on 8bit data (2002-10-10)
	http://python.org/sf/621457
docs do not include spec info from PEPs (2002-10-10)
	http://python.org/sf/621526

Closed Patches
--------------

specifying headers for extensions (2002-03-21)
	http://python.org/sf/533008
opt. timeouts for Queue.put() and .get() (2002-06-22)
	http://python.org/sf/572628
makesockaddr, use addrlen with AF_UNIX (2002-06-27)
	http://python.org/sf/574707
incompatible, but nice strings improveme (2002-07-08)
	http://python.org/sf/578688
smtplib.py patch for macmail esmtp auth (2002-07-17)
	http://python.org/sf/583180
db4 include not found (2002-08-02)
	http://python.org/sf/590377
zipfile.py reads archives with comments (2002-09-19)
	http://python.org/sf/611760
koi8_u codec (2002-09-23)
	http://python.org/sf/613173
tkFileDialog multiple option (2002-10-11)
	http://python.org/sf/621891


From python@rcn.com  Mon Oct 21 04:03:04 2002
From: python@rcn.com (Raymond Hettinger)
Date: Sun, 20 Oct 2002 23:03:04 -0400
Subject: [Python-Dev] __getitem__ in user defined classes
Message-ID: <002001c278ae$60425e00$5bf7a4d8@oemcomputer>

In Py2.3, __getitem__ conveniently supports slices for
builtin sequences: 'abcde'.__getitem__(slice(2,4))

For user defined classes to emulate this behavior, they need
to test the index argument to see whether it is a slice and then
loop over the slice indices like this:

class SquaresToTen:
    """Acts like a list of squares but
       computes only when needed"""

    def __len__(self):
        return 11

    def __getitem__(self, index):
        if isinstance(index, slice):
            return [x**2 for x in range(index.start, index.stop, index.step)]
        else:
            return index**2

print SquaresToTen()[2]
print SquaresToTen()[7:1:-2]


This could be simplified somewhat by making slices iterable so that
the __getitem__ definition looks more like this:

    def __getitem__(self, index):
        if isinstance(index, slice):
            return [x**2 for x in index]
        else:
            return index**2


Raymond Hettinger



From Raymond Hettinger" <python@rcn.com  Mon Oct 21 05:29:00 2002
From: Raymond Hettinger" <python@rcn.com (Raymond Hettinger)
Date: Mon, 21 Oct 2002 00:29:00 -0400
Subject: [Python-Dev] Discussion Draft:  PyDev Summary for week ending 20-Oct-2002
Message-ID: <00f601c278ba$62892340$5bf7a4d8@oemcomputer>

I'm filling in while Brett is on vacation.  To make everyone 
appreciate his formatting and lucidity, I'll skip the 
reStructuredText and write in my own terse style.


http://mail.python.org/pipermail/python-dev/2002-October/029577.html
Python 2.2.2. was released in final form this week.  Oh, yeah!

http://mail.python.org/pipermail/python-dev/2002-October/029527.html
Why does the Window's installer use shortened 8.3 pathnames?
Answer:  It doesn't have to but it less confusing than dealing
with pathnames that can have embedded spaces.

http://mail.python.org/pipermail/python-dev/2002-October/029544.html
PEP 218 was updated to match the sets.py module in CVS for Python 2.3.
The question arose as to whether immutable sets should be implemented
in C as a separate type or whether mutability should be a property.
Answer:  Keep them as different types since they support different
methods.  Next question:  Should there be a separate language syntax
for the two different types.  Answer:  Probably not, leave as an open
question in the PEP.

http://mail.python.org/pipermail/python-dev/2002-October/029542.html
The -U flag was put in to test the Python standard lib for being able 
to handle Unicode seamlessly.  It remained undocumented so that bug
reports wouldn't pile up while the issues were being worked out.
Progress is being made but it is possible the -U flag will never be
fully implemented and released as a documented feature.  

http://mail.python.org/pipermail/python-dev/2002-October/029535.html
Python has more than one place that is vulnerable to 32 bit word 
overflows unless a bunch of tests are put in place to catch unusual 
highly contrived cases that never come up in real world examples.  
Since the fixes would be ugly, wordy, and slow, it was decided to
leave them alone.

http://mail.python.org/pipermail/python-dev/2002-October/029024.html
Tcl 8.0 and 8.1 are ancient history except in the world of Cygwin
which is currently stuck with 8.0.  Accordingly, Python 2.3's support
for the prior versions has been cleaned out and support for 
UCS-4 in _tkinter has been added.

http://mail.python.org/pipermail/python-dev/2002-October/029591.html
Getting mtime, atime, and ctime to return floats instead of ints seemed
like a simple enough change.  Much discussion was held on how to
accommodate the change without breaking anything that relied on
the type.  Though breakage would've been rare (since the only normal
use for the data is to compare it or print it), Guido proposed a solution
(putting the ints in the tuple) that seemed to accommodate all practical
cases.

http://mail.python.org/pipermail/python-dev/2002-October/029679.html
Brian Quinlan teasesd out the design issue behind proxies and 
weakref objects. 






From tim.one@comcast.net  Mon Oct 21 05:56:20 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 21 Oct 2002 00:56:20 -0400
Subject: [Python-Dev] Discussion Draft:  PyDev Summary for week ending
 20-Oct-2002
In-Reply-To: <00f601c278ba$62892340$5bf7a4d8@oemcomputer>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECOBNAB.tim.one@comcast.net>

[Raymond Hettinger]
> I'm filling in while Brett is on vacation.  To make everyone
> appreciate his formatting and lucidity, I'll skip the
> reStructuredText and write in my own terse style.

It has its virtues!  For example, I actually read this to the end <wink>.

> ...
> Python has more than one place that is vulnerable to 32 bit word
> overflows unless a bunch of tests are put in place to catch unusual
> highly contrived cases that never come up in real world examples.
> Since the fixes would be ugly, wordy, and slow, it was decided to
> leave them alone.

That's just what I would do (i.e., leave them alone).  Security Weenies will
eat me for lunch because of it, and I don't think anyone else wants to join
me at the center of the buffet.  That is, this boondoggle had an Air of
Inevitability to it.

Curious:  We're missing a section about your new implementation of the
Mersenne Twister <wink>.  Speaking of which, I haven't been able to get a
copy, but Marsaglia (a Big Name in the field) has another new generator
called "The Monster", supposedly described in a preprint, that's simpler
than the Twister and has a much longer period (on the order of 10**8000; in
contrast, Python's current generator has period ~10**12).  By "simpler", I
mean that if you read the paper by the Twister's authors, you walked away
from it knowing less than when you started <0.6 wink>.



From greg@cosc.canterbury.ac.nz  Mon Oct 21 04:39:53 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 21 Oct 2002 16:39:53 +1300 (NZDT)
Subject: [Python-Dev] __getitem__ in user defined classes
In-Reply-To: <002001c278ae$60425e00$5bf7a4d8@oemcomputer>
Message-ID: <200210210339.g9L3drw15018@kuku.cosc.canterbury.ac.nz>

Raymond Hettinger <python@rcn.com>:

> This could be simplified somewhat by making slices iterable

Hmmm... if slices were iterable, they'd behave a lot
like xrange objects.

Perhaps

  slice == xrange

could be true in some future Python?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From oren-py-d@hishome.net  Mon Oct 21 10:09:34 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Mon, 21 Oct 2002 05:09:34 -0400
Subject: [Python-Dev] __getitem__ in user defined classes
In-Reply-To: <200210210339.g9L3drw15018@kuku.cosc.canterbury.ac.nz>
References: <002001c278ae$60425e00$5bf7a4d8@oemcomputer> <200210210339.g9L3drw15018@kuku.cosc.canterbury.ac.nz>
Message-ID: <20021021090934.GA99584@hishome.net>

On Mon, Oct 21, 2002 at 04:39:53PM +1300, Greg Ewing wrote:
> Raymond Hettinger <python@rcn.com>:
> 
> > This could be simplified somewhat by making slices iterable
> 
> Hmmm... if slices were iterable, they'd behave a lot
> like xrange objects.
> 
> Perhaps
> 
>   slice == xrange
> 
> could be true in some future Python?

http://python.org/sf/575515 , rejected.

	Oren


From mwh@python.net  Mon Oct 21 11:06:46 2002
From: mwh@python.net (Michael Hudson)
Date: 21 Oct 2002 11:06:46 +0100
Subject: [Python-Dev] __getitem__ in user defined classes
In-Reply-To: "Raymond Hettinger"'s message of "Sun, 20 Oct 2002 23:03:04 -0400"
References: <002001c278ae$60425e00$5bf7a4d8@oemcomputer>
Message-ID: <2mlm4sm7gp.fsf@starship.python.net>

"Raymond Hettinger" <python@rcn.com> writes:

> In Py2.3, __getitem__ conveniently supports slices for
> builtin sequences: 'abcde'.__getitem__(slice(2,4))
> 
> For user defined classes to emulate this behavior, they need
> to test the index argument to see whether it is a slice and then
> loop over the slice indices like this:
> 
> class SquaresToTen:
>     """Acts like a list of squares but
>        computes only when needed"""
> 
>     def __len__(self):
>         return 11
> 
>     def __getitem__(self, index):
>         if isinstance(index, slice):
>             return [x**2 for x in range(index.start, index.stop, index.step)]

You can spell that 

        if isinstance(index, slice):
            return [x**2 for x in range(*index.indices(11))]

and, as a bonus, it'll work more often <wink> (consider
"SquaresToTen()[7::-2]", for example).

> This could be simplified somewhat by making slices iterable so that
> the __getitem__ definition looks more like this:
> 
>     def __getitem__(self, index):
>         if isinstance(index, slice):
>             return [x**2 for x in index]
>         else:
>             return index**2

However to make omitted slice places work, you need to pass in the
length of the sequence, so I don't think this can fly.

Cheers,
M.

-- 
  We've had a lot of problems going from glibc 2.0 to glibc 2.1.
  People claim binary compatibility.  Except for functions they
  don't like.                       -- Peter Van Eynde, comp.lang.lisp


From gsw@agere.com  Mon Oct 21 14:33:52 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 21 Oct 2002 09:33:52 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIENEBMAB.tim.one@comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFOEALCMAA.gsw@agere.com>

Tim Peters wrote:
> Goodness no -- that kind of cleverness is usually buggy, and usually by
> failing to account for that the moral equivalent of -sys.maxint-1 is a fine
> product, but sys.maxint+1 isn't.

Agreed. Also, signed values needn't be 2's complement numbers,
and the behavior of signed overflow isn't guaranteed (it could
saturate, for instance). The use of doubles makes sense from a
portability standpoint. Or you could break out the signs and
use unsigned longs with preprocessor (and configure) checks,
being careful about performance impact. :-)

For example, you might need to do something equivalent to this:

#if HAVE_LIMITS_H
#define MAX_POSITIVE_LONG_VALUE (LONG_MAX)
#if -(LONG_MAX) == LONG_MIN
#define MAX_NEGATIVE_LONG_VALUE (LONG_MAX)
#elif (-(LONG_MAX) - 1) == LONG_MIN
#define MAX_NEGATIVE_LONG_VALUE ((LONG_MAX) + 1)
#else
#undef MAX_POSITIVE_LONG_VALUE
...

-Jerry


From tim.one@comcast.net  Mon Oct 21 22:47:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 21 Oct 2002 17:47:02 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOEALCMAA.gsw@agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIELPBNAB.tim.one@comcast.net>

[Gerald S. Williams]
> Agreed. Also, signed values needn't be 2's complement numbers,

Python assumes 2's comp in several places already.

> and the behavior of signed overflow isn't guaranteed (it could
> saturate, for instance).

But it doesn't assume anything about what happens on overflow, for either
doubles or for signed integral types.

> The use of doubles makes sense from a portability standpoint.

I thought you were going to time this <wink>?

> Or you could break out the signs and use unsigned longs with
> preprocessor (and configure) checks, being careful about performance
> impact. :-)

Breaking out the signs means tests and branches.  That's something the
int_mult() code in Python tries to avoid.  Unfortunately, there's a ton of
tests and branches before we ever get to the multiplication, trying to guess
whether this is really a "sequence * integer" case.  But once we get to
actually multiplying two ints, the fast path has only one test+branch (did
it overflow?).



From gsw@agere.com  Mon Oct 21 23:09:29 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 21 Oct 2002 18:09:29 -0400
Subject: [Python-Dev] Memory size overflows
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIELPBNAB.tim.one@comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFKEANCMAA.gsw@agere.com>

Tim Peters wrote:
> Python assumes 2's comp in several places already.

Yeah, I just noticed that.

> I thought you were going to time this <wink>?

I am.

If you assume that, for the vast majority of multiplications, both
multiplicands are in the range -sqrt(sys.maxint):sqrt(sys.maxint),
the following gives a 15% performance improvement for me:

    unsigned long sa = (a < 0L);
    unsigned long sb = (b < 0L);
    unsigned long ua = sa ? -a : a;
    unsigned long ub = sb ? -b : b;

    if ((ua | ub) & OVERFLOW_POSSIBLE_MASK)
    {
        <CURRENT IMPLEMENTATION, AS IS>
    }

If both numbers are evenly distributed from -sys.maxint:sys.maxint,
this yields about a 3% slowdown, though. These numbers are just for
the core multiply path, since I pulled it out in order to compare
algorithms better.

I'm looking into improving the rest, but it's difficult.

-Jerry


From gward@python.net  Tue Oct 22 20:24:39 2002
From: gward@python.net (Greg Ward)
Date: Tue, 22 Oct 2002 15:24:39 -0400
Subject: [Python-Dev] textwrap and unicode
Message-ID: <20021022192439.GA18528@cthulhu.gerg.ca>

Well, my ignorance of Unicode has finally bitten me -- someone filed a
bug (#622831) against textwrap.py because it crashes when it attempts to
wrap a Unicode string.

Here are the problems that I am aware of:

  * textwrap assumes "whitespace" means "the characters in
    string.whitespace"

  * textwrap assumes "lowercase letter" means "the characters in
    string.lowercase" (heck, this only works in English)

Can someone tell me what the proper way to do this is?  Or just point me
at the relevant documentation?  I've scoured the online docs and *Python
Essential Reference*, and I know more about the codes and unicodedata
modules than I did before.  But I still don't know how to replace all
whitespace with space, or detect words that end with a lowercase letter.

Thanks --

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
I brought my BOWLING BALL -- and some DRUGS!!


From gsw@agere.com  Tue Oct 22 20:48:48 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 22 Oct 2002 15:48:48 -0400
Subject: [Python-Dev] Re: Memory size overflows
In-Reply-To: <20021022160006.7706.54880.Mailman@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFCEAPCMAA.gsw@agere.com>

I didn't find any way to improve the actual overflow check,
although if you entirely replace the "fast path" check with
checks involving unsigned masking, you get some performance
improvement. For a wide variety of input patterns, I get
about an 18% speedup versus the core long multiply code,
when modified as shown below:

#define UL_LO_HI_BIT (((unsigned long)1) << (sizeof(unsigned long) * 4U))
#define UL_LO_MASK ((UL_LO_HI_BIT) - 1)
#define UL_HI_MASK (~(UL_LO_MASK))
#define UL_HI_LO_BIT (((unsigned long)1) << ((sizeof(unsigned long) * 4U)-1))
#define UL_OVERFLOW_IMPOSSIBLE_MASK ((UL_HI_LO_BIT) - 1)
#define UL_OVERFLOW_POSSIBLE_MASK (~(UL_OVERFLOW_IMPOSSIBLE_MASK))

long
core_int_mul(long a, long b)
{
    long longprod = a * b;
    unsigned long ma = a & UL_HI_MASK;

    if (ma == ((a < 0) ? UL_HI_MASK : 0))
    {
        unsigned long mb = b & UL_OVERFLOW_POSSIBLE_MASK;

        if (mb == ((b < 0) ? UL_OVERFLOW_POSSIBLE_MASK : 0))
        {
            return longprod;
        }
    }
    {
        double doubleprod = (double)a * (double)b;
        double doubled_longprod = (double)longprod;
        double diff = doubled_longprod - doubleprod;
        double absdiff = (diff >= 0.0) ? diff : -diff;
        double absprod = (doubleprod >= 0.0) ? doubleprod : -doubleprod;

        /* absdiff/absprod <= 1/32 iff
           32 * absdiff <= absprod -- 5 good bits is "close enough" */
        if (32.0 * absdiff <= absprod)
        {
            return longprod;
        }
        else
        {
            SIGNAL_AN_ERROR;
        }
    }
}

This version suffers no apparent degradation versus the
existing implementation when fed sets of multiplicands
evenly distributed over range(-sys.maxint, sys.maxint),
and almost always shows an improvement.

Shall I submit a patch?

-Jerry


From zooko@zooko.com  Tue Oct 22 20:51:36 2002
From: zooko@zooko.com (Zooko)
Date: Tue, 22 Oct 2002 15:51:36 -0400
Subject: [Python-Dev] pymalloc.supp (a valgrind suppressions file)
Message-ID: <E18453x-0001LJ-00@localhost>

# add the cmdline arg `--suppressions=thisfile' when invoking valgrind

  {
    PyObject_Free does tricky inside-a-page peeking, but it's really okay.
    Cond
    fun:PyObject_Free
  }

  {
    PyObject_Free does tricky inside-a-page peeking, but it's really okay.
    Value4
    fun:PyObject_Free
  }

  {
    PyObject_Free does tricky inside-a-page peeking, but it's really okay.
    Addr4
    fun:PyObject_Free
  }

  {
    PyObject_Realloc does tricky inside-a-page peeking, but it's really okay.
    Cond
    fun:PyObject_Realloc
  }

  {
    PyObject_Realloc does tricky inside-a-page peeking, but it's really okay.
    Value4
    fun:PyObject_Realloc
  }

  {
    PyObject_Realloc does tricky inside-a-page peeking, but it's really okay.
    Addr4
    fun:PyObject_Realloc
  }



From martin@v.loewis.de  Tue Oct 22 20:58:07 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 22 Oct 2002 21:58:07 +0200
Subject: [Python-Dev] textwrap and unicode
In-Reply-To: <20021022192439.GA18528@cthulhu.gerg.ca>
References: <20021022192439.GA18528@cthulhu.gerg.ca>
Message-ID: <m3lm4q1c1c.fsf@mira.informatik.hu-berlin.de>

Greg Ward <gward@python.net> writes:

> Here are the problems that I am aware of:
> 
>   * textwrap assumes "whitespace" means "the characters in
>     string.whitespace"

I don't know how precisely you want to formulate the property. If x is
a Unicode letter, then x.isspace() tells you whether it is a space
character (this property holds for all characters of the Zs category,
and all characters that have a bidirectionality of WS, B, or S).

>   * textwrap assumes "lowercase letter" means "the characters in
>     string.lowercase" (heck, this only works in English)

Works the same way: x.islower() tells you whether a character is
lower-case (meaning it is in the Ll category).

HTH,
Martin


From mal@lemburg.com  Tue Oct 22 21:01:32 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 22 Oct 2002 22:01:32 +0200
Subject: [Python-Dev] textwrap and unicode
References: <20021022192439.GA18528@cthulhu.gerg.ca>
Message-ID: <3DB5AE9C.1030607@lemburg.com>

Greg Ward wrote:
> Well, my ignorance of Unicode has finally bitten me -- someone filed a
> bug (#622831) against textwrap.py because it crashes when it attempts to
> wrap a Unicode string.
> 
> Here are the problems that I am aware of:
> 
>   * textwrap assumes "whitespace" means "the characters in
>     string.whitespace"

It should use u.isspace() for this.

You might also want to consider u.splitlines() for line breaking,
since Unicode has a lot more line breaking characters than
ASCII (which u.splitlines() knows about).

>   * textwrap assumes "lowercase letter" means "the characters in
>     string.lowercase" (heck, this only works in English)

u.lower() will do the right thing for Unicode.

> Can someone tell me what the proper way to do this is?  Or just point me
> at the relevant documentation?  I've scoured the online docs and *Python
> Essential Reference*, and I know more about the codes and unicodedata
> modules than I did before.  But I still don't know how to replace all
> whitespace with space, or detect words that end with a lowercase letter.

Hope that helps,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From fredrik@pythonware.com  Tue Oct 22 21:04:25 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 22 Oct 2002 22:04:25 +0200
Subject: [Python-Dev] textwrap and unicode
References: <20021022192439.GA18528@cthulhu.gerg.ca>
Message-ID: <00a201c27a06$45299260$ced241d5@hagrid>

greg wrote:
> But I still don't know how to replace all whitespace with
> space

string.join(phrase.split(), " ")

or

re.sub("(?u)\s+", " ", phrase)

not sure which one's faster; I suggest benchmarking.

(if you want to preserve leading/trailing space with the split
approach, use isspace on the start/end of phrase)

> or detect words that end with a lowercase letter.

word[-1].islower()

</F>



From gward@python.net  Tue Oct 22 21:24:08 2002
From: gward@python.net (Greg Ward)
Date: Tue, 22 Oct 2002 16:24:08 -0400
Subject: [Python-Dev] textwrap and unicode
In-Reply-To: <m3lm4q1c1c.fsf@mira.informatik.hu-berlin.de>
References: <20021022192439.GA18528@cthulhu.gerg.ca> <m3lm4q1c1c.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20021022202408.GA18718@cthulhu.gerg.ca>

On 22 October 2002, Martin v. Loewis said:
> I don't know how precisely you want to formulate the property. If x is
> a Unicode letter, then x.isspace() tells you whether it is a space
> character (this property holds for all characters of the Zs category,
> and all characters that have a bidirectionality of WS, B, or S).

OK, then it's an implementation problem rather than a "you can't get
there from here" problem.  Good.  The reason I need a list of
"whitespace chars" is to convert all whitespace to spaces; I use
string.maketrans() and s.translate() to do this efficiently:

class TextWrapper:

    [...]

    whitespace_trans = string.maketrans(string.whitespace,
                                        ' ' * len(string.whitespace))
    [...]

    def _munge_whitespace(self, text):
        """_munge_whitespace(text : string) -> string

        Munge whitespace in text: expand tabs and convert all other
        whitespace characters to spaces.  Eg. " foo\tbar\n\nbaz"
        becomes " foo    bar  baz".
        """
        if self.expand_tabs:
            text = text.expandtabs()
        if self.replace_whitespace:
            text = text.translate(self.whitespace_trans)
        return text

(The rationale: having tabs and newlines in a paragraph about to be
wrapped doesn't make any sense to me.)

Ahh, OK, I'm starting to see the problem: there's nothing wrong with the
translate() method of strings or unicode strings, but string.maketrans()
doesn't generate a mapping that u''.translate() likes.  Hmmmm.

Right, now I've RTFD'd (read the fine docstring) for u''.translate().
Here's what I've got now:

    whitespace_trans = string.maketrans(string.whitespace,
                                        ' ' * len(string.whitespace))

    unicode_whitespace_trans = {}
    for c in string.whitespace:
        unicode_whitespace_trans[ord(unicode(c))] = ord(u' ')
    [...]
    def _munge_whitespace (self, text):
        [...]
        if self.replace_whitespace:
            if isinstance(text, str):
                text = text.translate(self.whitespace_trans)
            elif isinstance(text, unicode):
                text = text.translate(self.unicode_whitespace_trans)

That's ugly as hell, but it works.  Is there a cleaner way?

The other bit of ASCII/English prejudice hardcoded into textwrap.py is
this regex:

    sentence_end_re = re.compile(r'[%s]'              # lowercase letter
                                 r'[\.\!\?]'          # sentence-ending punct.
                                 r'[\"\']?'           # optional end-of-quote
                                 % string.lowercase)

You may recall this from the kerfuffle over whether there should be two
spaces after a sentence in fixed-width fonts.  The feature is there, and
off by default, in TextWrapper.  I'm not so concerned about this -- I
mean, this doesn't even work with German or French, never mind Hebrew or
Chinese or Hindi.  Apart from the narrow definition of "lowercase
letter", it has English punctuation conventions hardcoded into it.  But
still, it seems *awfully* dumb in this day and age to hardcode
string.lowercase into a regex that's meant to detect "lowercase
letters".  But I couldn't find a better way to do it when I wrote this
code last spring.  Is there one?

Thanks!

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
OUR PLAN HAS FAILED STOP JOHN DENVER IS NOT TRULY DEAD STOP
HE LIVES ON IN HIS MUSIC STOP PLEASE ADVISE FULL STOP


From tim.one@comcast.net  Tue Oct 22 21:43:46 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 22 Oct 2002 16:43:46 -0400
Subject: [Python-Dev] Re: Memory size overflows
In-Reply-To: <GBEGLOMMCLDACBPKDIHFCEAPCMAA.gsw@agere.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEHHDMAA.tim.one@comcast.net>

[Gerald S. Williams]
> I didn't find any way to improve the actual overflow check,
> although if you entirely replace the "fast path" check with
> checks involving unsigned masking, you get some performance
> improvement. For a wide variety of input patterns, I get
> about an 18% speedup versus the core long multiply code,
> when modified as shown below:

Which platform?  Which compiler?  What was your test driver?  Was this
timing the mult code in isolation, or timing Python-level multiplies?
Claims of small speedups are notoriously platform- and test-dependent.  If
it's a mixed bag across platforms, the risk of introducing a new bug would
favor leaving things alone.  In the absence of a clear correctness proof, a
Python simulation program demonstrating correctness exhaustively in small
bases would also be helpful.

> ...
> Shall I submit a patch?

Sure, but also submit your timing harness so that people can measure the
effects cross-platform and cross-compiler.



From martin@v.loewis.de  Tue Oct 22 22:17:52 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 22 Oct 2002 23:17:52 +0200
Subject: [Python-Dev] textwrap and unicode
In-Reply-To: <20021022202408.GA18718@cthulhu.gerg.ca>
References: <20021022192439.GA18528@cthulhu.gerg.ca>
 <m3lm4q1c1c.fsf@mira.informatik.hu-berlin.de>
 <20021022202408.GA18718@cthulhu.gerg.ca>
Message-ID: <m3hefe18cf.fsf@mira.informatik.hu-berlin.de>

Greg Ward <gward@python.net> writes:

>         if self.replace_whitespace:
>             text = text.translate(self.whitespace_trans)
>         return text
> 
> (The rationale: having tabs and newlines in a paragraph about to be
> wrapped doesn't make any sense to me.)

Is it then really necessary to replace each of these characters, or
would it be acceptable to replace sequences of them, as Fredrik
proposes (.split,.join)?

>     for c in string.whitespace:
>         unicode_whitespace_trans[ord(unicode(c))] = ord(u' ')

There are conceptually 5 times as many space characters in Unicode
(NO-BREAK SPACE, THREE-PER-EM SPACE, OGHAM SPACE MARK, and whatnot),
but it is probably safe to ignore them. The complete fragment would
read

for c in range(sys.maxunicode):
  if unichr(c).isspace():
    unicode_whitespace_trans[c] = u' '
(which is somewhat time-consuming, so you could hard-code a larger
 list if you wanted to)

> That's ugly as hell, but it works.  Is there a cleaner way?

You may want to time re.sub, perhaps to find that the speed decrease
is acceptable:

space = re.compile("\s")

   text = space.sub(" ", text)

> The other bit of ASCII/English prejudice hardcoded into textwrap.py is
> this regex:
> 
>     sentence_end_re = re.compile(r'[%s]'              # lowercase letter
>                                  r'[\.\!\?]'          # sentence-ending punct.
>                                  r'[\"\']?'           # optional end-of-quote
>                                  % string.lowercase)

For the issue at hand: this code does "work" with Unicode, right?
I.e. it will give some result, even if confronted with funny characters?
If so, I think you can ignore this for the moment.

> But I couldn't find a better way to do it when I wrote this code
> last spring.  Is there one?

I believe the right approach is to support more classes in
SRE. This one would be covered if there was a [:lower:] class.

Regards,
Martin


From Jack.Jansen@oratrix.com  Tue Oct 22 22:46:15 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 22 Oct 2002 23:46:15 +0200
Subject: [Python-Dev] Precompiled headers
Message-ID: <B06FA69E-E607-11D6-802C-003065517236@oratrix.com>

Gcc on MacOSX has the ability to use precompiled headers, and I 
was toying with the idea of giving this a try for Python.h. Has 
anyone tried this, and is it worth the hassle?
Also, are precompiled headers an Apple addition to gcc, or are 
they available on all platforms?

I'm interested in trying this because when I started using 
precompiled headers for MacPython with the CodeWarrior compiler 
the build time for a complete Python tree want down by about 
70%, so that's definitely worth a little effort...
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From barry@python.org  Tue Oct 22 23:09:42 2002
From: barry@python.org (Barry A. Warsaw)
Date: Tue, 22 Oct 2002 18:09:42 -0400
Subject: [Python-Dev] Precompiled headers
References: <B06FA69E-E607-11D6-802C-003065517236@oratrix.com>
Message-ID: <15797.52390.563124.805066@gargle.gargle.HOWL>

>>>>> "JJ" == Jack Jansen <Jack.Jansen@oratrix.com> writes:

    JJ> Gcc on MacOSX has the ability to use precompiled headers, and
    JJ> I was toying with the idea of giving this a try for
    JJ> Python.h. Has anyone tried this, and is it worth the hassle?
    JJ> Also, are precompiled headers an Apple addition to gcc, or are
    JJ> they available on all platforms?

I don't know, but didn't the old NeXTSTEP Objective-C compilers (also
based off of gcc) support pre-compiled headers?

-Barry


From martin@v.loewis.de  Tue Oct 22 23:54:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Oct 2002 00:54:11 +0200
Subject: [Python-Dev] Precompiled headers
In-Reply-To: <B06FA69E-E607-11D6-802C-003065517236@oratrix.com>
References: <B06FA69E-E607-11D6-802C-003065517236@oratrix.com>
Message-ID: <m3vg3uytik.fsf@mira.informatik.hu-berlin.de>

Jack Jansen <Jack.Jansen@oratrix.com> writes:

> Also, are precompiled headers an Apple addition to gcc, or are they
> available on all platforms?

It's an Apple addition. Apple has a long-standing plan to contribute
this back, but there are several things in the way; mainly that it is
a large ugly hack.

> I'm interested in trying this because when I started using precompiled
> headers for MacPython with the CodeWarrior compiler the build time for
> a complete Python tree want down by about 70%, so that's definitely
> worth a little effort...

I find the compile times of the Python tree on my Pentium 930MHz
system to be accceptable, so I'd personally rather avoid a feature
with the potential of producing incorrect binaries.

Regards,
Martin



From Jack.Jansen@oratrix.com  Wed Oct 23 01:42:29 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Wed, 23 Oct 2002 02:42:29 +0200
Subject: [Python-Dev] Precompiled headers
In-Reply-To: <m3vg3uytik.fsf@mira.informatik.hu-berlin.de>
Message-ID: <4F2AEB2A-E620-11D6-802C-003065517236@oratrix.com>

On woensdag, oktober 23, 2002, at 12:54 , Martin v. Loewis wrote:
> I find the compile times of the Python tree on my Pentium 930MHz
> system to be accceptable, so I'd personally rather avoid a feature
> with the potential of producing incorrect binaries.

This last sentence sounds like you know something that I don't, 
could you explain? Precompiled headers keep track of the 
preprocessor symbols they depend on, and their value at 
precompile time, so I've always thought that this (together with 
correct Makefile dependencies) would be safe...
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From python@rcn.com  Wed Oct 23 01:48:19 2002
From: python@rcn.com (Raymond Hettinger)
Date: Tue, 22 Oct 2002 20:48:19 -0400
Subject: [Python-Dev] textwrap and unicode
References: <20021022192439.GA18528@cthulhu.gerg.ca>
Message-ID: <009901c27a2d$e263bf20$45f6a4d8@oemcomputer>

> Can someone tell me what the proper way to do this is?  Or just point me
> at the relevant documentation?  I've scoured the online docs and *Python
> Essential Reference*, and I know more about the codes and unicodedata
> modules than I did before.  But I still don't know how to replace all
> whitespace with space, or detect words that end with a lowercase letter.

Hang tight.
I'm writing isspace() and islower() for UniCode.
Then your fix will be much simpler and will
leave the Uni-magic all in one place.


Raymond Hettinger


From Anthony Baxter <anthony@interlink.com.au>  Wed Oct 23 02:50:32 2002
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Wed, 23 Oct 2002 11:50:32 +1000
Subject: [Python-Dev] Precompiled headers
In-Reply-To: <m3vg3uytik.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210230150.g9N1oXn25708@localhost.localdomain>

>>> Martin v. Loewis wrote
> Jack Jansen <Jack.Jansen@oratrix.com> writes:
> 
> > Also, are precompiled headers an Apple addition to gcc, or are they
> > available on all platforms?
> 
> It's an Apple addition. Apple has a long-standing plan to contribute
> this back, but there are several things in the way; mainly that it is
> a large ugly hack.

If it's anything like the way the old Lucid C++ compiler did it, it'll
be just coredumping the preprocessor's state. We always found that it
wasn't worthwhile unless you were dealing with something like OI's 
1.something megabytes of headers...

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From bfordham@socialistsushi.com  Wed Oct 23 03:55:54 2002
From: bfordham@socialistsushi.com (Bryan L. Fordham)
Date: Tue, 22 Oct 2002 22:55:54 -0400 (EDT)
Subject: [Python-Dev] tamper-evident logs
Message-ID: <20021022225332.P88383-100000@the.whole.net>

here's a question for y'all:

I've written a simple http proxy that logs the domains visited.  The idea
is that it keeps a list of the unique domains for each day.  No problem.
The problem is that I want it to be obvious if someone has deleted
something from the logs.

This is a little bit o' software meant to run on the user's box.  It does
not have to be impossible to tamper with it, nor do I need to be able to
say what was removed; just "hey, someone futzed with the log!"

Any ideas would be greatly appreciated.

--B




From fredrik@pythonware.com  Wed Oct 23 06:58:31 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 23 Oct 2002 07:58:31 +0200
Subject: [Python-Dev] tamper-evident logs
References: <20021022225332.P88383-100000@the.whole.net>
Message-ID: <01cb01c27a59$39639220$ced241d5@hagrid>

Bryan L. Fordham wrote:

> I've written a simple http proxy that logs the domains visited.  The idea
> is that it keeps a list of the unique domains for each day.  No problem.
> The problem is that I want it to be obvious if someone has deleted
> something from the logs.
> 
> This is a little bit o' software meant to run on the user's box.  It does
> not have to be impossible to tamper with it, nor do I need to be able to
> say what was removed; just "hey, someone futzed with the log!"
> 
> Any ideas would be greatly appreciated.

This mailing list is for development of Python itself, not for general
python questions.  I suggest posting to comp.lang.python, or maybe
to a security forum.  

</F>



From martin@v.loewis.de  Wed Oct 23 07:16:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Oct 2002 08:16:54 +0200
Subject: [Python-Dev] Precompiled headers
In-Reply-To: <4F2AEB2A-E620-11D6-802C-003065517236@oratrix.com>
References: <4F2AEB2A-E620-11D6-802C-003065517236@oratrix.com>
Message-ID: <m3smyx65nt.fsf@mira.informatik.hu-berlin.de>

Jack Jansen <Jack.Jansen@oratrix.com> writes:

> On woensdag, oktober 23, 2002, at 12:54 , Martin v. Loewis wrote:
> > I find the compile times of the Python tree on my Pentium 930MHz
> > system to be accceptable, so I'd personally rather avoid a feature
> > with the potential of producing incorrect binaries.
> 
> This last sentence sounds like you know something that I don't, could
> you explain? Precompiled headers keep track of the preprocessor
> symbols they depend on, and their value at precompile time, so I've
> always thought that this (together with correct Makefile dependencies)
> would be safe...

... assuming it is correctly implemented. All implementations of
precompiled headers I've used so far had certain limitations
(sometimes documented, sometimes undocumented), which would cause
users to throw away the precompiled stuff from time to time to make
sure that the compiler would really pick up their source code.

I don't know the Apple gcc implementation of precompiled headers,
though - it might be flawless.

Regards,
Martin


From martin@v.loewis.de  Wed Oct 23 07:19:08 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Oct 2002 08:19:08 +0200
Subject: [Python-Dev] textwrap and unicode
In-Reply-To: <009901c27a2d$e263bf20$45f6a4d8@oemcomputer>
References: <20021022192439.GA18528@cthulhu.gerg.ca>
 <009901c27a2d$e263bf20$45f6a4d8@oemcomputer>
Message-ID: <m3of9l65k3.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <python@rcn.com> writes:

> I'm writing isspace() and islower() for UniCode.

Ah, the time machine!

Regards,
Martin


From mal@lemburg.com  Wed Oct 23 09:12:48 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 23 Oct 2002 10:12:48 +0200
Subject: [Python-Dev] textwrap and unicode
References: <20021022192439.GA18528@cthulhu.gerg.ca>	<m3lm4q1c1c.fsf@mira.informatik.hu-berlin.de> <20021022202408.GA18718@cthulhu.gerg.ca>
Message-ID: <3DB65A00.8010000@lemburg.com>

Greg Ward wrote:
> On 22 October 2002, Martin v. Loewis said:
> 
> OK, then it's an implementation problem rather than a "you can't get
> there from here" problem.  Good.  The reason I need a list of
> "whitespace chars" is to convert all whitespace to spaces; I use
> string.maketrans() and s.translate() to do this efficiently:

Use the trick Fredrik posted: u' '.join(x.split()) (.split() defaults
to splitting on whitespace, Unicode whitespace if x is Unicode).

> Ahh, OK, I'm starting to see the problem: there's nothing wrong with the
> translate() method of strings or unicode strings, but string.maketrans()
> doesn't generate a mapping that u''.translate() likes.  Hmmmm.

Unicode uses a different API for this since it wouldn't make
sense to pass a sys.maxunicode character Unicode string to translate
just to map a few characters.

> The other bit of ASCII/English prejudice hardcoded into textwrap.py is
> this regex:
> 
>     sentence_end_re = re.compile(r'[%s]'              # lowercase letter
>                                  r'[\.\!\?]'          # sentence-ending punct.
>                                  r'[\"\']?'           # optional end-of-quote
>                                  % string.lowercase)
> 
> You may recall this from the kerfuffle over whether there should be two
> spaces after a sentence in fixed-width fonts.  The feature is there, and
> off by default, in TextWrapper.  I'm not so concerned about this -- I
> mean, this doesn't even work with German or French, never mind Hebrew or
> Chinese or Hindi.  Apart from the narrow definition of "lowercase
> letter", it has English punctuation conventions hardcoded into it.  But
> still, it seems *awfully* dumb in this day and age to hardcode
> string.lowercase into a regex that's meant to detect "lowercase
> letters".  But I couldn't find a better way to do it when I wrote this
> code last spring.  Is there one?

There are far too many lowercase characters in Unicode to make
this approach usable. It would be better if there were a
way to use Unicode character categories in the re sets. Since
that's not available, why not search for all potential sentence
ends and then try all of the using .islower() in a for-loop ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From bfordham@socialistsushi.com  Wed Oct 23 14:07:44 2002
From: bfordham@socialistsushi.com (Bryan L. Fordham)
Date: Wed, 23 Oct 2002 09:07:44 -0400
Subject: [Python-Dev] tamper-evident logs
References: <20021022225332.P88383-100000@the.whole.net> <01cb01c27a59$39639220$ced241d5@hagrid>
Message-ID: <3DB69F20.1070900@socialistsushi.com>

  Fredrik Lundh wrote

>This mailing list is for development of Python itself, not for general
>python questions.  I suggest posting to comp.lang.python, or maybe
>to a security forum.  
>  
>
yeah yeah yeah.... 8)
Seriously, I just clicked the wrong address when typing the email.  Sorry.

Please don't send anyone for me in the middle of the night (or the 
middle of the day, for that matter)...

My apologies.

--B



From radix@twistedmatrix.com  Wed Oct 23 18:32:47 2002
From: radix@twistedmatrix.com (Christopher Armstrong)
Date: Wed, 23 Oct 2002 19:32:47 +0200
Subject: [Python-Dev] src/Mac/Tools/IDE/PyFontify.py
Message-ID: <20021023173247.GU12550@twistedmatrix.com>

Hi!

Python2.3 contains, inside the src/Mac/Tools/IDE/ directory a
perfectly portable module called "PyFontify". Can this be added to the
Python standard library somewhere under site-packages/? Twisted Matrix
Labs distributes it with Twisted manually (with the copyright intact,
of course), but this isn't very nice.

-- 
			Christopher Armstrong
		    << radix@twistedmatrix.com >>
	     http://twistedmatrix.com/users/radix.twistd/


From fredrik@pythonware.com  Wed Oct 23 21:03:00 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 23 Oct 2002 22:03:00 +0200
Subject: [Python-Dev] src/Mac/Tools/IDE/PyFontify.py
References: <20021023173247.GU12550@twistedmatrix.com>
Message-ID: <00c701c27acf$322d6550$ced241d5@hagrid>

Christopher Armstrong wrote:
> Python2.3 contains, inside the src/Mac/Tools/IDE/ directory a
> perfectly portable module called "PyFontify". Can this be added to the
> Python standard library somewhere under site-packages/? Twisted Matrix
> Labs distributes it with Twisted manually (with the copyright intact,
> of course), but this isn't very nice.

any reason you cannot just use the standard "tokenize" module?

</F>



From just@letterror.com  Wed Oct 23 21:08:51 2002
From: just@letterror.com (Just van Rossum)
Date: Wed, 23 Oct 2002 22:08:51 +0200
Subject: [Python-Dev] src/Mac/Tools/IDE/PyFontify.py
In-Reply-To: <00c701c27acf$322d6550$ced241d5@hagrid>
Message-ID: <r01050400-1021-42AA0C41E6C311D6B44F003065D5E7E4@[10.0.0.23]>

Fredrik Lundh wrote:

> any reason you cannot just use the standard "tokenize" module?

Hear hear. PyFontify.py is dead. And it sucks.

Just


From pobrien@orbtech.com  Wed Oct 23 23:22:28 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Wed, 23 Oct 2002 17:22:28 -0500
Subject: [Python-Dev] OT: Pickling article
Message-ID: <200210231722.28376.pobrien@orbtech.com>

I recently finished an article for IBM's developerWorks website on Python=
=20
Persistence Management. I basically cover quite a bit of material on=20
pickling/unpickling. I'm looking for a couple of pickling experts to crit=
ique=20
the article before it gets published. If you'd like to do so, please cont=
act=20
me offlist at pobrien@orbtech.com. Thanks.

--=20
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------



From mal@lemburg.com  Thu Oct 24 09:50:42 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 24 Oct 2002 10:50:42 +0200
Subject: [Python-Dev] Re: Moving to Unicode 3.2
References: <E182ZjA-0002ii-00@usw-pr-cvs1.sourceforge.net>	<3DB13012.3040604@lemburg.com> <m3of9qwjr0.fsf_-_@mira.informatik.hu-berlin.de>
Message-ID: <3DB7B462.8080702@lemburg.com>

Martin v. Loewis wrote:
 > "M.-A. Lemburg" <mal@lemburg.com> writes:
 >
 >
 >>I haven't seen any messages about this on python-dev. Did I miss
 >>something ?
 >
 > No. For a change like this, I did not think consultation was
 > necessary.

I saw that :-)

 >>The switch from Unicode 3.0 is a big one since 3.2 introduces
 >>non-BMP character points for the first time.
 >
 > I disagree; it's a small change. Just look at the patch itself: apart
 > from the (considerably large) generated data, there were very few
 > actual changes to source code: Changing a few limits was sufficient.

Which underlines Fredrik's good design of the Unicode database.

Still, the changes from Unicode 3.0 to 3.2 are significant (to the
few users who actually make use of the database):

	http://www.unicode.org/unicode/reports/tr28/

Looking at a diff of the 3.0 and the 3.2 data files you can
find quite a few additions, changes in categories and several
removals of numeric interpretations of code points. Most
important is, of course, that 3.2 actually assing code points
outside the 16-bit Unicode range.

What I'm really concerned about is that Python is moving on the
dev edge here while most other technologies are still using
Unicode 2.0. It's nice to be able to use Python as reference
implementation for Unicode, but the interoperability between
Python and Java/Windows suffers from this.

 > Since there are no backwards compatibility issues, and no design
 > choices (apart from the choice of updating the database at all), this
 > is a straight-forward change.
 >
 >
 >>I also don't think that it is a good idea to ship the Unicode
 >>3.2 database while the code behaves as defined in Unicode 3.0.
 >
 > Can you please elaborate? What code behaves as defined in Unicode 3.0
 > that is incompatible with the Unicode 3.2 database?

Well, I should have written: ... while the code does not even
fully implement Unicode 3.0.

Fortunately, you have already started working in that
direction (adding normalization) and I'd like to thank
you for your efforts.

 >>And last not least, I'd like to be asked before you make such
 >>changes.
 >
 >
 > I find this quite a possessive view, and I would prefer if you bring
 > up technical arguments instead of procedural ones, but ok...

I am not trying to possess anything here. This is about managing
the Unicode code base and I'm still under the impression that
I'm the one in charge here.

 > I was under the impression that I can apply my own professional
 > judgement when deciding what patches to apply without consultation, in
 > what cases to ask on python-dev, and when to submit a patch to SF.
  >
 > Apparently, this impression is wrong. Can you please give precise
 > instructions what constitutes "such a change"?

I consider changes in the design or additions that affect
the design such a change.

 > Also, should I back this change out?

No, let's first find out what the consequences of this change
are and then decide whether it's a good idea or not.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/




From martin@v.loewis.de  Thu Oct 24 13:34:25 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 24 Oct 2002 14:34:25 +0200
Subject: [Python-Dev] Re: Moving to Unicode 3.2
In-Reply-To: <3DB7B462.8080702@lemburg.com>
References: <E182ZjA-0002ii-00@usw-pr-cvs1.sourceforge.net>
 <3DB13012.3040604@lemburg.com>
 <m3of9qwjr0.fsf_-_@mira.informatik.hu-berlin.de>
 <3DB7B462.8080702@lemburg.com>
Message-ID: <m3wuo810dq.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Still, the changes from Unicode 3.0 to 3.2 are significant (to the
> few users who actually make use of the database):
> 
> 	http://www.unicode.org/unicode/reports/tr28/
> 
> Looking at a diff of the 3.0 and the 3.2 data files you can
> find quite a few additions, changes in categories and several
> removals of numeric interpretations of code points. Most
> important is, of course, that 3.2 actually assing code points
> outside the 16-bit Unicode range.

I still can't see why adding code points beyond the BMP is most
important part of this change. If anything, the changes to categories
might affect applications. In all these cases, I'll assume that the
more recent database is more correct, so any change in behaviour is
likely for the better.

> What I'm really concerned about is that Python is moving on the dev
> edge here while most other technologies are still using Unicode
> 2.0. It's nice to be able to use Python as reference implementation
> for Unicode, but the interoperability between Python and
> Java/Windows suffers from this.

What do you mean "most other technologies"? Windows 2000+ support
UTF-16 at the API quite happily, OpenType supports UCS-4, XML is
defined in terms of Unicode-including-extended-planes, Unicode
normalization (mandated by all W3C recommendations) is defined in
terms of Unicode 3.1, the character database of JDK 1.4 is based on
Unicode 3.0. I don't think there are that many technologies which
still use Unicode 2.0 (explicitly: most applications don't actually
care whether a code point is assigned or not).

> Well, I should have written: ... while the code does not even
> fully implement Unicode 3.0.
> 
> Fortunately, you have already started working in that
> direction (adding normalization) and I'd like to thank
> you for your efforts.

While looking at normalization, I noticed an aspect that still
confuses me:

When new characters are added to Unicode, you can get additional
canonical decompositions. Applying the composition algorithm
mechanically, that could cause certain normal forms to change
depending on the Unicode version. Therefore, TR#15 defines a
composition version of Unicode to be used in normalization, see

http://www.unicode.org/unicode/reports/tr15/#Versioning

This version is defined to be Unicode 3.1.0. 

In turn, it seems that algorithms that perform normalization based on
older databases are incorrect (algorithms based on newer databases
must take the exclusion tables into account, which my patch does)
[this is the confusing part: how wrong are normalization algorithms
 based on, say, Unicode 3.0?]

So for this specific aspect, updating the Unicode database was really
necessary.

>  > Apparently, this impression is wrong. Can you please give precise
>  > instructions what constitutes "such a change"?
> 
> I consider changes in the design or additions that affect
> the design such a change.

That is still unclear. What's an addition? I did not really add
anything - I changed what was already there. Nor do I think I changed
the design.

Regards,
Martin



From radix@twistedmatrix.com  Thu Oct 24 18:54:21 2002
From: radix@twistedmatrix.com (Christopher Armstrong)
Date: Thu, 24 Oct 2002 19:54:21 +0200
Subject: [Python-Dev] src/Mac/Tools/IDE/PyFontify.py
In-Reply-To: <r01050400-1021-42AA0C41E6C311D6B44F003065D5E7E4@[10.0.0.23]>
References: <00c701c27acf$322d6550$ced241d5@hagrid> <r01050400-1021-42AA0C41E6C311D6B44F003065D5E7E4@[10.0.0.23]>
Message-ID: <20021024175420.GB631@twistedmatrix.com>

On Wed, Oct 23, 2002 at 10:08:51PM +0200, Just van Rossum wrote:
> Fredrik Lundh wrote:
> 
> > any reason you cannot just use the standard "tokenize" module?
> 
> Hear hear. PyFontify.py is dead. And it sucks.

Well, the reason we used it was that py2html used it. :-) Anyway, we
searched for alternatives, and only came across Norvig's py2html which
is very broken, so Moshe Zadka wrote a new htmlizer in half an hour
today, based on tokenizer. Hooray NIH! twisted.python.htmlizer. ;)

-- 
			Christopher Armstrong
		    << radix@twistedmatrix.com >>
	     http://twistedmatrix.com/users/radix.twistd/


From dave@boost-consulting.com  Thu Oct 24 23:41:04 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 24 Oct 2002 18:41:04 -0400
Subject: [Python-Dev] pyconfig.h and exec-prefix
Message-ID: <uu1jbsbnj.fsf@boost-consulting.com>

This issue came up on the c++-sig. I'm not qualified to answer any of
the questions at the bottom, and hoped that this forum could be of
some help.

From: "Paul F. Kunz" <Paul_Kunz@SLAC.Stanford.EDU>
Subject: Re: [C++-sig] Boost V1 build on Solaris
To: c++-sig@python.org
Date: Wed, 23 Oct 2002 17:36:58 -0700
Reply-To: c++-sig@python.org

   One final question, which is not specific to boost::python but
general question on building Python modules.   What my Solaris
administrators have done is to configure Python with 

>  ./configure
 --prefix=/afs/.slac.stanford.edu/package/python/common/2.0
 --exec-prefix=/afs/.slac.stanford.edu/package/python/sun4x_55/2.0

They also did a build fron Linux with the appropriate `exec-prefix'.
What apparently this has done is to put all but one of the Python
include files in the directory `common/2.0/include'.  But Python.h
includes `config.h' or `pyconfig.h' (depending on the version) which
gets installed in `sun4x_55/2.0/include'.  This is appropriate since
this file is machine/OS dependent.

   Now the problem is that boost::python, and other packages, gives you
one variable, like PYTHON_INCLUDES,  for its build.   When it is set to find
`Python.h' it will not find `pyconfig.h'.   

   What is the best way to handle this situation?   Is it a bug in
Python's configure script?   Bug in boost::python and others?   Or am
I missing something obvious?

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html



From skip@pobox.com  Fri Oct 25 00:11:11 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 24 Oct 2002 18:11:11 -0500
Subject: [Python-Dev] pyconfig.h and exec-prefix
In-Reply-To: <uu1jbsbnj.fsf@boost-consulting.com>
References: <uu1jbsbnj.fsf@boost-consulting.com>
Message-ID: <15800.32271.195128.852080@montanaro.dyndns.org>

    >> Now the problem is that boost::python, and other packages, gives you
    >> one variable, like PYTHON_INCLUDES, for its build.  When it is set to
    >> find `Python.h' it will not find `pyconfig.h'.

Seems to me like boost::python needs to be able to point at both
directories.  Maybe PYTHON_INCLUDES could be a colon-separated list of
directories which gets broken up into multiple -I<dir> flags.  I suppose
that sort of change is a bit late for v1?

Skip


From dave@boost-consulting.com  Fri Oct 25 00:01:48 2002
From: dave@boost-consulting.com (David Abrahams)
Date: 24 Oct 2002 19:01:48 -0400
Subject: [Python-Dev] pyconfig.h and exec-prefix
In-Reply-To: <15800.32271.195128.852080@montanaro.dyndns.org>
References: <uu1jbsbnj.fsf@boost-consulting.com>
 <15800.32271.195128.852080@montanaro.dyndns.org>
Message-ID: <uadl3saoz.fsf@boost-consulting.com>

Skip Montanaro <skip@pobox.com> writes:

>     >> Now the problem is that boost::python, and other packages, gives you
>     >> one variable, like PYTHON_INCLUDES, for its build.  When it is set to
>     >> find `Python.h' it will not find `pyconfig.h'.
> 
> Seems to me like boost::python needs to be able to point at both
> directories.  Maybe PYTHON_INCLUDES could be a colon-separated list of
> directories which gets broken up into multiple -I<dir> flags.  I suppose
> that sort of change is a bit late for v1?

Considering that v2 was released a few weeks ago, that sounds about
right ;-)

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html



From skip@pobox.com  Fri Oct 25 00:59:27 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 24 Oct 2002 18:59:27 -0500
Subject: [Python-Dev] pyconfig.h and exec-prefix
In-Reply-To: <uadl3saoz.fsf@boost-consulting.com>
References: <uu1jbsbnj.fsf@boost-consulting.com>
 <15800.32271.195128.852080@montanaro.dyndns.org>
 <uadl3saoz.fsf@boost-consulting.com>
Message-ID: <15800.35167.82866.831754@montanaro.dyndns.org>

    >> Seems to me like boost::python needs to be able to point at both
    >> directories.  Maybe PYTHON_INCLUDES could be a colon-separated list
    >> of directories which gets broken up into multiple -I<dir> flags.  I
    >> suppose that sort of change is a bit late for v1?

    David> Considering that v2 was released a few weeks ago, that sounds
    David> about right ;-)

How about

    OTHER=<the other include directory
    make -n \
    | sed -e "s/-I$PYTHON_INCLUDES/-I$PYTHON_INCLUDES -I$OTHER/" \
    | sh -x

?  Depending on how the boost::python directory tree is organized, it just
might work.

Skip


From geertj@boskant.nl  Fri Oct 25 17:26:51 2002
From: geertj@boskant.nl (Geert Jansen)
Date: Fri, 25 Oct 2002 18:26:51 +0200
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
Message-ID: <20021025162651.GA9056@vader.boskant.nl>

Dear Python developers,

I have a question about "dict" subclasses and eval() that I would kindly
like to ask.

The problem I have is the following: For a web content system in Python, I
am using Python's eval() function to run code embedded in templates.
Currently, the code failes with a NameError if a variable is referenced
inside the code that is not defined in the locals or globals dictionary.
Unfortunately, this is a rather common situation: one often uses variables
(like error codes) that aren't set all the time. Intializing these variables
is a tedious job so a default value would suit me nicely.

I created a derived class of the standard `dict' that fills in a default
value when a key is not found. This is exactly the same as Guido describes
in his "descintro" paper. I tried to use this dictionary as the "globals"
parameter with eval(). As Guido already describes in his paper, this doesn't
work. Quoting Guido: 

"The interpreter uses an internal function to access the dictionary, which
bypasses our __getitem__() override. I admit that this can be a problem
(although it is only a problem in this context, when a dict subclass is used
as a locals/globals dictionary); it remains to be seen if I can fix this
without compromising performance in the common case."

Is there a solution to this problem in sight? Or altenately, is there a way
I can find out which variables are used inside a compiled code block so I
can initialize non-specified variables? I have a vague memory that the
nested scopes feature has to determine at compile time which variables are
being used in a code block.

Thanks for your time!

Greetings,
Geert Jansen


From martin@v.loewis.de  Fri Oct 25 17:50:02 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 25 Oct 2002 18:50:02 +0200
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
In-Reply-To: <20021025162651.GA9056@vader.boskant.nl>
References: <20021025162651.GA9056@vader.boskant.nl>
Message-ID: <m3iszqihtx.fsf@mira.informatik.hu-berlin.de>

Geert Jansen <geertj@boskant.nl> writes:

> Is there a solution to this problem in sight? 

No, although contributions are welcome.

> Or altenately, is there a way I can find out which variables are
> used inside a compiled code block so I can initialize non-specified
> variables? I have a vague memory that the nested scopes feature has
> to determine at compile time which variables are being used in a
> code block.

Sure: you can parse the code, build an AST tuple (or list), and
traverse that.

This, of course, is off-topic for python-dev.

Regards,
Martin


From nas@python.ca  Fri Oct 25 18:01:29 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 25 Oct 2002 10:01:29 -0700
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
In-Reply-To: <m3iszqihtx.fsf@mira.informatik.hu-berlin.de>
References: <20021025162651.GA9056@vader.boskant.nl> <m3iszqihtx.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20021025170129.GA18253@glacier.arctrix.com>

Martin v. Loewis wrote:
> Geert Jansen <geertj@boskant.nl> writes:
> > Or altenately, is there a way I can find out which variables are
> > used inside a compiled code block so I can initialize non-specified
> > variables? I have a vague memory that the nested scopes feature has
> > to determine at compile time which variables are being used in a
> > code block.
> 
> Sure: you can parse the code, build an AST tuple (or list), and
> traverse that.

You might want to look at the ptl_compile module in Quixote for some
ideas.  The AST from the 'compiler' package is much nicer than the parse
tree from the 'parser' module.

  Neil


From gsw@agere.com  Fri Oct 25 18:07:14 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 25 Oct 2002 13:07:14 -0400
Subject: [Python-Dev] Re: Memory size overflows
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCEHHDMAA.tim.one@comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFCEBDCMAA.gsw@agere.com>

I wrote:
> Shall I submit a patch?

Tim Peters wrote:
> Sure, but also submit your timing harness so that people can measure the
> effects cross-platform and cross-compiler.

Of course. I first wanted to see if there was any interest.

Integrated into Python, the speedup was much less: only 0.1%
overall for this million-multiply script:

 import time

 start = time.time()
 a = -1000
 while a <= 1000:
     b = 1000
     while b <= 1000:
         b += 1
     a += 1
 end = time.time()

 base = end - start

 start = time.time()
 a = -1000
 while a <= 1000:
     b = -1000
     while b <= 1000:
         b += 1
         c = a * b
     a += 1
 end = time.time()

 print end - start - base

The difference in speedup is an interesting data point on
Python overhead in general (it took just over 6 seconds to
do 1,000,000 multiplies on my system).

> Which platform?  Which compiler?  What was your test driver?  Was this
> timing the mult code in isolation, or timing Python-level multiplies?
> Claims of small speedups are notoriously platform- and test-dependent.  If
> it's a mixed bag across platforms, the risk of introducing a new bug would
> favor leaving things alone.  In the absence of a clear correctness proof, a
> Python simulation program demonstrating correctness exhaustively in small
> bases would also be helpful.

I doubt you're still interested, but I used Cygwin on a
P3/833 Thinkpad running Win2K (GCC compiler). My original
results were on just the multiply code in isolation--I
wanted to see if there was interest before continuing.

Since I have them, I attached my initial testbed (new.c
is the final version), and here is an exhaustive test
for the "cannot_overflow" check used in that version:

 from __future__ import generators

 def cannot_overflow(a,b,m):
     M_MASK = (1 << m) - 1
     HI_LO_BIT = 1 << ((m/2) - 1)
     OVL_IMPOSSIBLE_MASK = HI_LO_BIT - 1
     OVL_POSSIBLE_MASK = M_MASK ^ OVL_IMPOSSIBLE_MASK
     if a < 0:
         check = OVL_POSSIBLE_MASK
     else:
         check = 0
     if (a & OVL_POSSIBLE_MASK) == check:
         if b < 0:
             check = OVL_POSSIBLE_MASK
         else:
             check = 0
         if (b & OVL_POSSIBLE_MASK) == check:
             return True
     return False

 def overflowed(a,b,m):
     VALID_MASK = (2**(m - 1)) - 1
     INVALID_MASK = (2**(2*m) - 1) ^ VALID_MASK
     prod = a * b
     highbits = prod & INVALID_MASK
     return (highbits != 0) and (highbits != INVALID_MASK)

 def checkall(m):
     for i in range(-(2**(m-1)),2**(m-1)):
         for j in range(-(2**(m-1)),2**(m-1)):
             if cannot_overflow(i,j,m) and overflowed(i,j,m):
                 yield (i,j)

 if __name__ == "__main__":
     for m in range(2,11,2):
         print "M ==", m
         for i in checkall(m):
             print i
         print "END OF LIST"

You can compare the "cannot_overflow" code to the code that
would be added to int_mul. For the sake of minimizing impact,
I limited the changes to defining UL_OVERFLOW_POSSIBLE_MASK
as the equivalent of ~(2**(m/2-1) - 1) and this code, added
immediately after converting the arguments into longs:

        {
            unsigned long ma = a & UL_OVERFLOW_POSSIBLE_MASK;
            if (ma == ((a < 0) ? UL_OVERFLOW_POSSIBLE_MASK : 0))
            {
                unsigned long mb = b & UL_OVERFLOW_POSSIBLE_MASK;
                if (mb == ((b < 0) ? UL_OVERFLOW_POSSIBLE_MASK : 0))
                {
                    return PyInt_FromLong(a*b);
                }
            }
        }

This seems pretty safe, but the payout seems too small
to warrant taking this any further. Unless someone asks
me to do so, I'm planning to drop it.

-Jerry


From alex_c@MIT.EDU  Sat Oct 26 21:34:39 2002
From: alex_c@MIT.EDU (Alex Coventry)
Date: Sat, 26 Oct 2002 16:34:39 -0400 (EDT)
Subject: [Python-Dev] "conflicting types" in python CVS build
Message-ID: <200210262034.QAA15149@multics.mit.edu>

hi.  i built a fresh checkout of python from CVS, and got this error:

Parser/pgenmain.c:169: conflicting types for PyOS_Readline'
Include/pythonrun.h:116: previous declaration of PyOS_Readline'
make: *** [Parser/pgenmain.o] Error 1

cvs annoate says the latest prototype for PyOS_Readline is in
pythonrun.h.  Commenting that line out results in a complete build that
passes the test suite.

hope this helps.
alex.


From martin@v.loewis.de  Sun Oct 27 09:23:29 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Oct 2002 10:23:29 +0100
Subject: [Python-Dev] "conflicting types" in python CVS build
In-Reply-To: <200210262034.QAA15149@multics.mit.edu>
References: <200210262034.QAA15149@multics.mit.edu>
Message-ID: <m33cqsutf2.fsf@mira.informatik.hu-berlin.de>

Alex Coventry <alex_c@MIT.EDU> writes:

> hi.  i built a fresh checkout of python from CVS, and got this error:
> 
> Parser/pgenmain.c:169: conflicting types for PyOS_Readline'
> Include/pythonrun.h:116: previous declaration of PyOS_Readline'
> make: *** [Parser/pgenmain.o] Error 1
> 
> cvs annoate says the latest prototype for PyOS_Readline is in
> pythonrun.h.  Commenting that line out results in a complete build that
> passes the test suite.
> 
> hope this helps.

Not really. Can you please copy line 169 of pgenmain.c? I can't see a
problem.

Regards,
Martin



From skip@manatee.mojam.com  Sun Oct 27 13:00:39 2002
From: skip@manatee.mojam.com (Skip Montanaro)
Date: Sun, 27 Oct 2002 07:00:39 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200210271300.g9RD0dev003909@manatee.mojam.com>

Bug/Patch Summary
-----------------

318 open / 2981 total bugs (no change)
96 open / 1743 total patches (+4)

New Bugs
--------

email incompatibility upgrading to 2.2.2 (2002-10-20)
	http://python.org/sf/626119
missing DECREF's in embedding example (2002-10-21)
	http://python.org/sf/626275
Support RFC 2111 in email package (2002-10-21)
	http://python.org/sf/626452
urllib2 doesn't do HTTP-EQUIV & Refresh (2002-10-21)
	http://python.org/sf/626543
strptime() always returns 0 in dst field (2002-10-21)
	http://python.org/sf/626570
Build error using make VPATH feature (2002-10-22)
	http://python.org/sf/626926
canvas.create_box() crashes Tk thread (2002-10-22)
	http://python.org/sf/626936
KeyPress bindings don't work (2002-10-23)
	http://python.org/sf/627798
curses library not found during make (2002-10-23)
	http://python.org/sf/627864
Set constructor fails with NameError (2002-10-24)
	http://python.org/sf/628246
pydoc.Helper.topics not based on docs (2002-10-24)
	http://python.org/sf/628258
Deprecate PyNumber_Check (2002-10-25)
	http://python.org/sf/628842
pickle won't dump instances after reload (2002-10-25)
	http://python.org/sf/628925
Race condition in asyncore poll (2002-10-26)
	http://python.org/sf/629097
build does not respect --prefix (2002-10-27)
	http://python.org/sf/629345

New Patches
-----------

db4 include not found (2002-08-02)
	http://python.org/sf/590377
Support Unicode normalization (2002-10-21)
	http://python.org/sf/626485
Support Hangul Syllable names (2002-10-21)
	http://python.org/sf/626548
Bytecode copy bug in freeze (2002-10-24)
	http://python.org/sf/627900
xmlrpclib: Optional 'nil' support  (2002-10-24)
	http://python.org/sf/628208
Experimental Inno Setup Win32 installer (2002-10-24)
	http://python.org/sf/628301
Detect BLT on more systems (2002-10-26)
	http://python.org/sf/629126
install lib-dynload .so files mode 555 (2002-10-26)
	http://python.org/sf/629278

Closed Bugs
-----------

mac module documentation inaccuracy. (2002-01-17)
	http://python.org/sf/505150
ConfigParser bug/limitation (2002-02-01)
	http://python.org/sf/511737
ConfigParser cleanup, items() [needs tests] (2002-04-17)
	http://python.org/sf/545096
Nested anchors in Ref Man index.html (2002-04-20)
	http://python.org/sf/546579
Wrong description for PyErr_Restore (2002-07-03)
	http://python.org/sf/577000
PyString_AsString underdocumented (2002-09-08)
	http://python.org/sf/606463
packages doc in the tutorial (2002-09-29)
	http://python.org/sf/616211
PythonLauncher Dialogs do not work (2002-10-06)
	http://python.org/sf/619245
test_builtin hangs on MacOSX 10.2.1 (2002-10-06)
	http://python.org/sf/619400
sizeof(struct stat) changes (2002-10-10)
	http://python.org/sf/621523
urllib.pathname2url(), .url2pathname() (2002-10-14)
	http://python.org/sf/623196
Typo at line 351 of email/Charset.py (2002-10-18)
	http://python.org/sf/625509
IDE starts in / on OSX (2002-10-19)
	http://python.org/sf/625734

Closed Patches
--------------

readline /dev/tty problem (2002-02-04)
	http://python.org/sf/512981
Installation database patch (2002-05-29)
	http://python.org/sf/562100
add unescape method to xml.sax.saxutils (2002-09-23)
	http://python.org/sf/613256
autoconf-2.5X tweaks for configure.in (2002-10-15)
	http://python.org/sf/623771


From alex_c@MIT.EDU  Sun Oct 27 13:47:28 2002
From: alex_c@MIT.EDU (Alex Coventry)
Date: Sun, 27 Oct 2002 08:47:28 -0500 (EST)
Subject: [Python-Dev] "conflicting types" in python CVS build
Message-ID: <200210271347.IAA25086@multics.mit.edu>

> > hope this helps.
> 
> Not really. Can you please copy line 169 of pgenmain.c? I can't see a
> problem.

at the time, lines 167-169 were

char *
PyOS_Readline(char *prompt)
{

which conflicted with the prototype in pythonrun.h.  but it looks like a
change which fixed that was checked in this morning.  sorry for the
confusion.

alex.


From skip@pobox.com  Sun Oct 27 14:20:05 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 27 Oct 2002 08:20:05 -0600
Subject: [Python-Dev] "conflicting types" in python CVS build
In-Reply-To: <200210271347.IAA25086@multics.mit.edu>
References: <200210271347.IAA25086@multics.mit.edu>
Message-ID: <15803.62997.75987.884926@montanaro.dyndns.org>

    Alex> at the time, lines 167-169 were

    Alex> char *
    Alex> PyOS_Readline(char *prompt)
    Alex> {

    Alex> which conflicted with the prototype in pythonrun.h.  but it looks
    Alex> like a change which fixed that was checked in this morning.  sorry
    Alex> for the confusion.

No confusion on your part.  I encountered the same problem and checked in
the change last night.  This was the checkin message:

    Change PyOS_Readline declaration to match the recent change to
    myreadline.c (see Patch 512981).  I changed stdin to sys_stdin in the
    body of the function, but did not change stderr to sys_stdout, though I
    suspect that may be the correct course.  I don't know the code involved
    well enough to judge.

Skip


From guido@python.org  Mon Oct 28 03:14:40 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 27 Oct 2002 22:14:40 -0500
Subject: [Python-Dev] __getitem__ in user defined classes
In-Reply-To: Your message of "Mon, 21 Oct 2002 05:09:34 EDT."
 <20021021090934.GA99584@hishome.net>
References: <002001c278ae$60425e00$5bf7a4d8@oemcomputer> <200210210339.g9L3drw15018@kuku.cosc.canterbury.ac.nz>
 <20021021090934.GA99584@hishome.net>
Message-ID: <200210280314.g9S3EeR16822@pcp02138704pcs.reston01.va.comcast.net>

> > > This could be simplified somewhat by making slices iterable
> > 
> > Hmmm... if slices were iterable, they'd behave a lot
> > like xrange objects.
> > 
> > Perhaps
> > 
> >   slice == xrange
> > 
> > could be true in some future Python?
> 
> http://python.org/sf/575515 , rejected.

Note that slice() isn't limited to integers -- that's up to whoever
interprets the slice instance.  But iteration really should only
support ints.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python@rcn.com  Mon Oct 28 13:43:59 2002
From: python@rcn.com (Raymond Hettinger)
Date: Mon, 28 Oct 2002 08:43:59 -0500
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
References: <20021025162651.GA9056@vader.boskant.nl>
Message-ID: <003901c27e88$12271b40$d3b53bd0@oemcomputer>

> I created a derived class of the standard `dict' that fills in a default
> value when a key is not found. This is exactly the same as Guido describes
> in his "descintro" paper. I tried to use this dictionary as the "globals"
> parameter with eval(). As Guido already describes in his paper, this doesn't
> work. 

I've wanted this for years and think it would be an important
improvement.  Unfortunately, it appears to be a problem without 
a simple solution.


Raymond Hettinger


From guido@python.org  Mon Oct 28 14:14:53 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 28 Oct 2002 09:14:53 -0500
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
In-Reply-To: Your message of "Mon, 28 Oct 2002 08:43:59 EST."
 <003901c27e88$12271b40$d3b53bd0@oemcomputer>
References: <20021025162651.GA9056@vader.boskant.nl>
 <003901c27e88$12271b40$d3b53bd0@oemcomputer>
Message-ID: <200210281414.g9SEErd18664@pcp02138704pcs.reston01.va.comcast.net>

> > I created a derived class of the standard `dict' that fills in a
> > default value when a key is not found. This is exactly the same as
> > Guido describes in his "descintro" paper. I tried to use this
> > dictionary as the "globals" parameter with eval(). As Guido
> > already describes in his paper, this doesn't work.
> 
> I've wanted this for years and think it would be an important
> improvement.  Unfortunately, it appears to be a problem without 
> a simple solution.

The solution appears simple (use PyObject_GetItem etc. instead of
PyDict_GetItem) but would cause serious performance hits: the dict API
and its implementatio are highly optimized for this; PyObject_GetItem
would add several levels of function calls, *plus* more reference
count handling and exception handling.  Oren Tirosh's global/builtin
lookup speedup would be impossible.  (Oren, how's that coming?)

In addition, I'm concerned that this would become a popular tool to
hack Python's semantics in all sorts of ways.  While hacking the
semantics is essential for some situations, in most cases this causes
more problems than it solves because you can't use standard tools like
PyChecker when you change something as fundamental as how variable
references are resolved.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From geertj@boskant.nl  Mon Oct 28 14:44:43 2002
From: geertj@boskant.nl (Geert Jansen)
Date: Mon, 28 Oct 2002 15:44:43 +0100
Subject: [Python-Dev] Using defaultdict as globals/locals for eval()
In-Reply-To: <200210281414.g9SEErd18664@pcp02138704pcs.reston01.va.comcast.net>
References: <20021025162651.GA9056@vader.boskant.nl> <003901c27e88$12271b40$d3b53bd0@oemcomputer> <200210281414.g9SEErd18664@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021028144443.GA21911@vader.boskant.nl>

On Mon, Oct 28, 2002 at 09:14:53AM -0500, Guido van Rossum wrote:
> > > I created a derived class of the standard `dict' that fills in a
> > > default value when a key is not found. This is exactly the same as
> > > Guido describes in his "descintro" paper. I tried to use this
> > > dictionary as the "globals" parameter with eval(). As Guido
> > > already describes in his paper, this doesn't work.
> > 
> > I've wanted this for years and think it would be an important
> > improvement.  Unfortunately, it appears to be a problem without 
> > a simple solution.
> 
> The solution appears simple (use PyObject_GetItem etc. instead of
> PyDict_GetItem) but would cause serious performance hits: the dict API
> and its implementatio are highly optimized for this; PyObject_GetItem
> would add several levels of function calls, *plus* more reference
> count handling and exception handling.

Maybe eval() could check whether the "locals" and "globals" are plain
dictionaries, or dictionary derived objects. This check would have to
be done once, just after eval() is called. Depending on the result,
PyDict_GetItem or PyObject_GetItem is used further on.

Hmm, this sounds too trivial. I bet there are complications...

Geert


From mgilfix@eecs.tufts.edu  Tue Oct 29 00:12:55 2002
From: mgilfix@eecs.tufts.edu (Michael Gilfix)
Date: Mon, 28 Oct 2002 19:12:55 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021029001255.GB12789@eecs.tufts.edu>

  Just been lurkin' :) Possibly lost means that Valgrind didn't see
any freeing for a specific point after a malloc. In valgrind terms,
you probably forgot to free a pointer and lost all points that could
reference that memory. In other words, a leak :) Usually 75% of those
errors can be solved locally within the function, although I haven't
looked at the specifics, but that 25% can get nasty.

                        -- Mike

On Thu, Oct 17 @ 20:38, Guido van Rossum wrote:
> There are a few other records mentioning GC_Resize, but this one is
> the biggest.  Could it be that the free frame list is botched?  OTOH,
> what does "possibly lost" really mean?

-- 
Michael Gilfix
mgilfix@eecs.tufts.edu

For my gpg public key:
http://www.eecs.tufts.edu/~mgilfix/contact.html


From guido@python.org  Tue Oct 29 00:37:55 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 28 Oct 2002 19:37:55 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: Your message of "Mon, 28 Oct 2002 19:12:55 EST."
 <20021029001255.GB12789@eecs.tufts.edu>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
 <20021029001255.GB12789@eecs.tufts.edu>
Message-ID: <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net>

>   Just been lurkin' :) Possibly lost means that Valgrind didn't see
> any freeing for a specific point after a malloc. In valgrind terms,
> you probably forgot to free a pointer and lost all points that could
> reference that memory. In other words, a leak :) Usually 75% of those
> errors can be solved locally within the function, although I haven't
> looked at the specifics, but that 25% can get nasty.

Booh.

That statistic doesn't apply to Python -- there are no "simple" leaks
left in Python's C code, the remaining ones are all nasty.

Also, valgrind carefully distinguishes between "definitely lost" and
"possibly lost".  What's the difference?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mgilfix@eecs.tufts.edu  Tue Oct 29 01:00:29 2002
From: mgilfix@eecs.tufts.edu (Michael Gilfix)
Date: Mon, 28 Oct 2002 20:00:29 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net> <20021029001255.GB12789@eecs.tufts.edu> <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021029010029.GC12789@eecs.tufts.edu>

  This should help shed some light on the situation:

Quoth the docs:

"""
For each such block, Valgrind scans the entire address space of the
process, looking for pointers to the block. One of three situations
may result:

  * A pointer to the start of the block is found. This usually
  indicates programming sloppiness; since the block is still pointed
  at, the programmer could, at least in principle, free'd it before
  program exit.

  * A pointer to the interior of the block is found. The pointer
  might originally have pointed to the start and have been moved
  along, or it might be entirely unrelated. Valgrind deems such a
  block as "dubious", that is, possibly leaked, because it's unclear
  whether or not a pointer to it still exists.

  * The worst outcome is that no pointer to the block can be
  found. The block is classified as "leaked", because the programmer
  could not possibly have free'd it at program exit, since no pointer
  to it exists. This might be a symptom of having lost the pointer at
  some earlier point in the program.
"""

  Possibly is the second case and definitely lost is the third case.
The definitely lost, in my experience, tends to mean you just forgot
to free a pointer. The possibly lost usually means that some memory
rot occurred, where it's not clear which pointer is causing the mem
leak.

                       -- Mike

On Mon, Oct 28 @ 19:37, Guido van Rossum wrote:
> >   Just been lurkin' :) Possibly lost means that Valgrind didn't see
> > any freeing for a specific point after a malloc. In valgrind terms,
> > you probably forgot to free a pointer and lost all points that could
> > reference that memory. In other words, a leak :) Usually 75% of those
> > errors can be solved locally within the function, although I haven't
> > looked at the specifics, but that 25% can get nasty.
> 
> Booh.
> 
> That statistic doesn't apply to Python -- there are no "simple" leaks
> left in Python's C code, the remaining ones are all nasty.
>
> Also, valgrind carefully distinguishes between "definitely lost" and
> "possibly lost".  What's the difference?
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

-- 
Michael Gilfix
mgilfix@eecs.tufts.edu

For my gpg public key:
http://www.eecs.tufts.edu/~mgilfix/contact.html


From guido@python.org  Tue Oct 29 01:15:44 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 28 Oct 2002 20:15:44 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: Your message of "Mon, 28 Oct 2002 20:00:29 EST."
 <20021029010029.GC12789@eecs.tufts.edu>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net> <20021029001255.GB12789@eecs.tufts.edu> <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net>
 <20021029010029.GC12789@eecs.tufts.edu>
Message-ID: <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net>

>   This should help shed some light on the situation:

Thanks; this indeed helps.

> Quoth the docs:
> 
> """
> For each such block, Valgrind scans the entire address space of the
> process, looking for pointers to the block. One of three situations
> may result:
> 
>   * A pointer to the start of the block is found. This usually
>   indicates programming sloppiness; since the block is still pointed
>   at, the programmer could, at least in principle, free'd it before
>   program exit.

Booh again.  Lots of globals get initialized with pointers to
malloc'ed blocks that are never freed.  There are never called "leaks"
in other leak detectors, just "alive at exit".  I think valgrind
actually doesn't call these leaks either.

>   * A pointer to the interior of the block is found. The pointer
>   might originally have pointed to the start and have been moved
>   along, or it might be entirely unrelated. Valgrind deems such a
>   block as "dubious", that is, possibly leaked, because it's unclear
>   whether or not a pointer to it still exists.

Aha!  This may be the case.  When an object has a GC header, all
pointers to the object point to an address 12 bytes in the block,
which is where the "object" lay-out begins.  Normally, there should be
at least one pointer to the start of the block from one of the GC
chains, but objects don't have to be in a chain at all.

(I wonder if pymalloc adds to the confusion, since its arenas count as
a single block to malloc and hence to valgrind, but are internally cut
up into many objects.)

>   * The worst outcome is that no pointer to the block can be
>   found. The block is classified as "leaked", because the programmer
>   could not possibly have free'd it at program exit, since no pointer
>   to it exists. This might be a symptom of having lost the pointer at
>   some earlier point in the program.

This is a true leak.

> """
> 
>   Possibly is the second case and definitely lost is the third case.
> The definitely lost, in my experience, tends to mean you just forgot
> to free a pointer. The possibly lost usually means that some memory
> rot occurred, where it's not clear which pointer is causing the mem
> leak.

How much Python extension coding (in C) have you done?  In Python, it
almost never is a matter of forgetting to free() -- it's usually a
matter of forgetting to DECREF, and sometimes a matter of doing an
unnecessary INCREF.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Tue Oct 29 01:36:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 28 Oct 2002 20:36:50 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDKCCAB.tim.one@comcast.net>

[Guido]
> ...
> (I wonder if pymalloc adds to the confusion, since its arenas count as
> a single block to malloc and hence to valgrind, but are internally cut
> up into many objects.)

For each arena, the address returned by malloc() is stored in the
file-static arenas[] vector.  So if a diagnostic program can find that
vector, it can find the base address of every arena gotten from malloc.

In a debug build, though, pymalloc adds pad bytes to both ends of each
request (and whether handled by pymalloc or by malloc!), and returns the
address of the byte beyond the leading pad byte.  This can leave any number
of system-malloc blocks with no direct pointer to their start.  That's
specific to the debug build, which forces all Python mem API calls to go
thru pymalloc (the release build only directs PyObject_{Malloc,etc}() calls
to pymalloc).



From mgilfix@eecs.tufts.edu  Tue Oct 29 01:58:45 2002
From: mgilfix@eecs.tufts.edu (Michael Gilfix)
Date: Mon, 28 Oct 2002 20:58:45 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net> <20021029001255.GB12789@eecs.tufts.edu> <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net> <20021029010029.GC12789@eecs.tufts.edu> <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021029015845.GD12789@eecs.tufts.edu>

On Mon, Oct 28 @ 20:15, Guido van Rossum wrote:
> >   * A pointer to the start of the block is found. This usually
> >   indicates programming sloppiness; since the block is still pointed
> >   at, the programmer could, at least in principle, free'd it before
> >   program exit.
> 
> Booh again.  Lots of globals get initialized with pointers to
> malloc'ed blocks that are never freed.  There are never called "leaks"
> in other leak detectors, just "alive at exit".  I think valgrind
> actually doesn't call these leaks either.

  I don't think valgrind reports this first case. Only the second and
third, from what I gathered in the docs.

> >   Possibly is the second case and definitely lost is the third case.
> > The definitely lost, in my experience, tends to mean you just forgot
> > to free a pointer. The possibly lost usually means that some memory
> > rot occurred, where it's not clear which pointer is causing the mem
> > leak.
> 
> How much Python extension coding (in C) have you done?  In Python, it
> almost never is a matter of forgetting to free() -- it's usually a
> matter of forgetting to DECREF, and sometimes a matter of doing an
> unnecessary INCREF.

  I've done a fair bit, but to explain valgrind things, I figured it was
best to talk the valgrind talk. Either way, it comes to the same thing.
Glad that helped.

                      -- Mike

-- 
Michael Gilfix
mgilfix@eecs.tufts.edu

For my gpg public key:
http://www.eecs.tufts.edu/~mgilfix/contact.html


From jepler@unpythonic.net  Tue Oct 29 02:39:55 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Mon, 28 Oct 2002 20:39:55 -0600
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
Message-ID: <20021028203952.A1460@unpythonic.net>

--PEIAKu/WMn1b1Hv9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Using dict subclasses in eval() and exec
----------------------------------------

With the attached patch, you can now use a dict subclass which implements
__getitem__ as the global or builtin namespace for eval() and exec.

If you do not use the feature, the performance impact is low (<2%).
The fast case adds only one test+branch to each eval_code() call (to
set 'fastglobals' and 'slowbuiltins'), each lookup in globals (to
check 'fastglobals'), and each lookup in builtins (to check 'slowbuiltins').

If you do use the feature, the performance impact is unfortunately
quite substantial (400%).

If you use subclasses of dict with eval()/exec, but do not define __getitem__,
the performance penalty is modest (~10%).

The smaller penalty of using a dict subclass without __getitem__ could
probably be erased if the 'fastglobals' test can check that __getitem__
is not defined

Included are a patch and a test suite.  Note that the patch causes a
failure in test_descr which is testing that __getitem__ is *not* called on
a dictionary subclass.

Pystone timings on a 1GHz Duron system running RedHat Linux 8.0 (gcc
3.2) follow.

    CVS	    +patch   +subclass	+__getattr__
1   2.74    2.76     3          14.05
2   2.74    2.78     3.02       14.07
3   2.74    2.78     3.03       14.07
4   2.76    2.78     3.04       14.08
5   2.76    2.79     3.04       14.1
avg 2.748   2.778    3.026      14.074
%   100     101.1    110.1      512.2

* 2.3a0 CVS (28 Oct 2002)
Pystone(1.1) time for 50000 passes = 2.74
This machine benchmarks at 18248.2 pystones/second
Pystone(1.1) time for 50000 passes = 2.74
This machine benchmarks at 18248.2 pystones/second
Pystone(1.1) time for 50000 passes = 2.74
This machine benchmarks at 18248.2 pystones/second
Pystone(1.1) time for 50000 passes = 2.76
This machine benchmarks at 18115.9 pystones/second
Pystone(1.1) time for 50000 passes = 2.76
This machine benchmarks at 18115.9 pystones/second

* 2.3a0 CVS + patch
Pystone(1.1) time for 50000 passes = 2.78
This machine benchmarks at 17985.6 pystones/second
Pystone(1.1) time for 50000 passes = 2.78
This machine benchmarks at 17985.6 pystones/second
Pystone(1.1) time for 50000 passes = 2.79
This machine benchmarks at 17921.1 pystones/second
Pystone(1.1) time for 50000 passes = 2.79
This machine benchmarks at 17921.1 pystones/second
Pystone(1.1) time for 50000 passes = 2.76
This machine benchmarks at 18115.9 pystones/second

* 2.3a0 CVS + patch + dict subclass w/o __getitem__
(commandline:
./python -c 'class D(dict): pass                          
d = D(globals()); execfile("Lib/test/pystone.py", d)')

Pystone(1.1) time for 50000 passes = 3.03
This machine benchmarks at 16501.7 pystones/second
Pystone(1.1) time for 50000 passes = 3.04
This machine benchmarks at 16447.4 pystones/second
Pystone(1.1) time for 50000 passes = 3
This machine benchmarks at 16666.7 pystones/second
Pystone(1.1) time for 50000 passes = 3.04
This machine benchmarks at 16447.4 pystones/second
Pystone(1.1) time for 50000 passes = 3.02
This machine benchmarks at 16556.3 pystones/second

* 2.3a0 CVS + patch + dict subclass w/__getitem__
(commandline:
./python -c 'class D(dict): __getitem__ = lambda s, i: dict.__getitem__(s, i)
d = D(globals()); execfile("Lib/test/pystone.py", d)')

Pystone(1.1) time for 50000 passes = 14.05
This machine benchmarks at 3558.72 pystones/second
Pystone(1.1) time for 50000 passes = 14.08
This machine benchmarks at 3551.14 pystones/second
Pystone(1.1) time for 50000 passes = 14.07
This machine benchmarks at 3553.66 pystones/second
Pystone(1.1) time for 50000 passes = 14.07
This machine benchmarks at 3553.66 pystones/second
Pystone(1.1) time for 50000 passes = 14.1
This machine benchmarks at 3546.1 pystones/second


--PEIAKu/WMn1b1Hv9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="ceval-userdict.patch"

Index: Python/ceval.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v
retrieving revision 2.338
diff -u -r2.338 ceval.c
--- Python/ceval.c	3 Oct 2002 09:53:11 -0000	2.338
+++ Python/ceval.c	29 Oct 2002 00:22:42 -0000
@@ -529,6 +529,8 @@
 	char *filename;
 #endif
 
+	int fastglobals, slowbuiltins;
+
 /* Tuple access macros */
 
 #ifndef Py_DEBUG
@@ -603,6 +605,9 @@
 	names = co->co_names;
 	consts = co->co_consts;
 	fastlocals = f->f_localsplus;
+#define PyDict_CheckExact(d) ((d)->ob_type == &PyDict_Type)
+	fastglobals = PyDict_CheckExact(f->f_globals);
+	slowbuiltins = !PyDict_CheckExact(f->f_builtins);
 	freevars = f->f_localsplus + f->f_nlocals;
 	_PyCode_GETCODEPTR(co, &first_instr);
 	/* An explanation is in order for the next line.
@@ -1596,7 +1601,7 @@
 					     PyObject_REPR(w));
 				break;
 			}
-			err = PyDict_SetItem(x, w, v);
+			err = (fastglobals ? PyDict_SetItem : PyObject_SetItem)(x, w, v);
 			Py_DECREF(v);
 			break;
 
@@ -1675,7 +1680,7 @@
 		case STORE_GLOBAL:
 			w = GETITEM(names, oparg);
 			v = POP();
-			err = PyDict_SetItem(f->f_globals, w, v);
+			err = (fastglobals ? PyDict_SetItem : PyObject_SetItem)(f->f_globals, w, v);
 			Py_DECREF(v);
 			break;
 
@@ -1696,9 +1701,10 @@
 			}
 			x = PyDict_GetItem(x, w);
 			if (x == NULL) {
-				x = PyDict_GetItem(f->f_globals, w);
+				x = (fastglobals ? PyDict_GetItem : PyObject_GetItem)(f->f_globals, w);
 				if (x == NULL) {
-					x = PyDict_GetItem(f->f_builtins, w);
+					if (!fastglobals) PyErr_Clear();
+					x = (slowbuiltins ? PyObject_GetItem : PyDict_GetItem)(f->f_builtins, w);
 					if (x == NULL) {
 						format_exc_check_arg(
 							    PyExc_NameError,
@@ -1711,9 +1717,10 @@
 			PUSH(x);
 			break;
 
+
 		case LOAD_GLOBAL:
 			w = GETITEM(names, oparg);
-			if (PyString_CheckExact(w)) {
+			if (PyString_CheckExact(w) && fastglobals) {
 				/* Inline the PyDict_GetItem() calls.
 				   WARNING: this is an extreme speed hack.
 				   Do not try this at home. */
@@ -1727,6 +1734,7 @@
 						PUSH(x);
 						continue;
 					}
+					if(slowbuiltins) goto load_builtins_slow;
 					d = (PyDictObject *)(f->f_builtins);
 					x = d->ma_lookup(d, w, hash)->me_value;
 					if (x != NULL) {
@@ -1738,9 +1746,11 @@
 				}
 			}
 			/* This is the un-inlined version of the code above */
-			x = PyDict_GetItem(f->f_globals, w);
+			x = PyObject_GetItem(f->f_globals, w);
 			if (x == NULL) {
-				x = PyDict_GetItem(f->f_builtins, w);
+				PyErr_Clear();
+			  load_builtins_slow:
+				x = PyObject_GetItem(f->f_builtins, w);
 				if (x == NULL) {
 				  load_global_error:
 					format_exc_check_arg(

--PEIAKu/WMn1b1Hv9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="test_dictsubclass.py"

X='xx'

class D(dict):
	def __getitem__(self, item):
		if item is X: return X
		return dict.__getitem__(self, item)

d = D(globals())
d['zz'] = 'zz'

# Make sure the dict acts like it should
print d[X] == X
print d['D'] == D
print d['zz'] == 'zz'
try: d['yy']
except KeyError, detail: print "KeyError", detail
else: print "failed to get exception"

# Make sure that exec and eval() in globals() works right
exec "print (len.__name__, D.__name__)" in globals()
exec "print (lambda: (len.__name__, D.__name__))()" in globals()
print eval("len.__name__, D.__name__", globals())
print eval("(lambda: (len.__name__, D.__name__))()", globals())
try: eval(X, globals())
except NameError, detail: print detail
else: print "failed to get exception"
try: exec "zz" in globals()
except NameError, detail: print detail
else: print "failed to get exception"
try: exec "yy" in globals()
except NameError, detail: print detail
else: print "failed to get exception"

# Make sure that exec and eval() in d works right
exec "print (len.__name__, D.__name__, xx, zz)" in d
exec "print (lambda: (len.__name__, D.__name__, xx, zz))()" in d
print eval("len.__name__, D.__name__, xx, zz", d)
print eval("(lambda: (len.__name__, D.__name__, xx, zz))()", d)
try: exec "yy" in d
except NameError, detail: print detail
else: print "failed to get exception"

--PEIAKu/WMn1b1Hv9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=test_dictsubclass

test_dictsubclass
True
True
True
KeyError 'yy'
('len', 'D')
('len', 'D')
('len', 'D')
('len', 'D')
name 'xx' is not defined
name 'zz' is not defined
name 'yy' is not defined
('len', 'D', 'xx', 'zz')
('len', 'D', 'xx', 'zz')
('len', 'D', 'xx', 'zz')
('len', 'D', 'xx', 'zz')
name 'yy' is not defined

--PEIAKu/WMn1b1Hv9--


From jepler@unpythonic.net  Tue Oct 29 02:42:35 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Mon, 28 Oct 2002 20:42:35 -0600
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <20021029015845.GD12789@eecs.tufts.edu>
References: <20021017230108.GL9241@epoch.metaslash.com> <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net> <20021029001255.GB12789@eecs.tufts.edu> <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net> <20021029010029.GC12789@eecs.tufts.edu> <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net> <20021029015845.GD12789@eecs.tufts.edu>
Message-ID: <20021028204234.A1528@unpythonic.net>

On Mon, Oct 28, 2002 at 08:58:45PM -0500, Michael Gilfix wrote:
>   I don't think valgrind reports this first case [a pointer to the
> beginning of the block is found]. Only the second and
> third, from what I gathered in the docs.

See --show-reachable


From neal@metaslash.com  Tue Oct 29 03:40:16 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 28 Oct 2002 22:40:16 -0500
Subject: [Python-Dev] Valgrind on 2.2.2
In-Reply-To: <20021028204234.A1528@unpythonic.net>
References: <20021017230108.GL9241@epoch.metaslash.com>
 <200210180038.g9I0cef12864@pcp02138704pcs.reston01.va.comcast.net>
 <20021029001255.GB12789@eecs.tufts.edu>
 <200210290037.g9T0bt822570@pcp02138704pcs.reston01.va.comcast.net>
 <20021029010029.GC12789@eecs.tufts.edu>
 <200210290115.g9T1Fih22665@pcp02138704pcs.reston01.va.comcast.net>
 <20021029015845.GD12789@eecs.tufts.edu> <20021028204234.A1528@unpythonic.net>
Message-ID: <20021029034016.GE17007@epoch.metaslash.com>

On Mon, Oct 28, 2002 at 08:42:35PM -0600, Jeff Epler wrote:
> On Mon, Oct 28, 2002 at 08:58:45PM -0500, Michael Gilfix wrote:
> >   I don't think valgrind reports this first case [a pointer to the
> > beginning of the block is found]. Only the second and
> > third, from what I gathered in the docs.
> 
> See --show-reachable

--show-reachable was used when running valgrind.  
Python was not built using pydebug.

Neal


From mal@lemburg.com  Tue Oct 29 07:57:01 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 29 Oct 2002 08:57:01 +0100
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
References: <20021028203952.A1460@unpythonic.net>
Message-ID: <3DBE3F4D.1000301@lemburg.com>

Jeff Epler wrote:
> Using dict subclasses in eval() and exec
> ----------------------------------------
> 
> With the attached patch, you can now use a dict subclass which implements
> __getitem__ as the global or builtin namespace for eval() and exec.
> 
> If you do not use the feature, the performance impact is low (<2%).
> The fast case adds only one test+branch to each eval_code() call (to
> set 'fastglobals' and 'slowbuiltins'), each lookup in globals (to
> check 'fastglobals'), and each lookup in builtins (to check 'slowbuiltins').
> 
> If you do use the feature, the performance impact is unfortunately
> quite substantial (400%).
> 
> If you use subclasses of dict with eval()/exec, but do not define __getitem__,
> the performance penalty is modest (~10%).

If all you want is a way to define a default value for a failing
dictionary lookup, wouldn't it be better to simply add this feature
to the standard dictionary implementation ?

I don't think that many people will use this feature, yet everybody
would pay for the performance hit.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From mhammond@skippinet.com.au  Tue Oct 29 09:54:45 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 29 Oct 2002 20:54:45 +1100
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <3DBE3F4D.1000301@lemburg.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au>

> I don't think that many people will use this feature, yet everybody
> would pay for the performance hit.

I wonder how many reasonable real-life scenarios actually exist for this
feature?

My personal use would be for COM, to implement case-insensitive namespaces.
In some cases, case-insensitivity is actually part of the implementation
specification, and I can imagine other scenarios where case insensitive
namespaces are "reasonable" even in the face of Python's excellent decision
not to go this route.

I wonder is this is actually the root of many requests to see this feature
implemented?  If not, it would be interesting to see what the competing
desires really are.

Mark.



From guido@python.org  Tue Oct 29 12:09:22 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 07:09:22 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Mon, 28 Oct 2002 20:39:55 CST."
 <20021028203952.A1460@unpythonic.net>
References: <20021028203952.A1460@unpythonic.net>
Message-ID: <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net>

> @@ -1696,9 +1701,10 @@
>  			}
>  			x = PyDict_GetItem(x, w);
>  			if (x == NULL) {
> -				x = PyDict_GetItem(f->f_globals, w);
> +				x = (fastglobals ? PyDict_GetItem : PyObject_GetItem)(f->f_globals, w);
>  				if (x == NULL) {
> -					x = PyDict_GetItem(f->f_builtins, w);
> +					if (!fastglobals) PyErr_Clear();
> +					x = (slowbuiltins ? PyObject_GetItem : PyDict_GetItem)(f->f_builtins, w);
>  					if (x == NULL) {
>  						format_exc_check_arg(
>  							    PyExc_NameError,

I don't see the extra DECREF here (and below) needed to compensate for
the fact that PyObject_GetItem() returns the object with incremented
refcount but PyDict_GetItem() doesn't.

What are you going to do with all the *other* places where
PyDict_GetItem() is used?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Oct 29 12:43:50 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 07:43:50 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 20:54:45 +1100."
 <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au>
Message-ID: <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>

> I wonder how many reasonable real-life scenarios actually exist for
> this feature?
> 
> My personal use would be for COM, to implement case-insensitive
> namespaces.  In some cases, case-insensitivity is actually part of
> the implementation specification, and I can imagine other scenarios
> where case insensitive namespaces are "reasonable" even in the face
> of Python's excellent decision not to go this route.
> 
> I wonder is this is actually the root of many requests to see this
> feature implemented?  If not, it would be interesting to see what
> the competing desires really are.

Excellent observation.  Perhaps the people who want this can get
together and write a PEP that explains the motivation and why the
alternatives don't work?  (E.g. when you need case-insensitive
variable names, perhaps it's just as easy to preprocess the source
code?)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jepler@unpythonic.net  Tue Oct 29 14:34:26 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 29 Oct 2002 08:34:26 -0600
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <20021029143416.GA16099@unpythonic.net>

On Tue, Oct 29, 2002 at 07:09:22AM -0500, Guido van Rossum wrote:
> I don't see the extra DECREF here (and below) needed to compensate for
> the fact that PyObject_GetItem() returns the object with incremented
> refcount but PyDict_GetItem() doesn't.

oops.  I believe this can be fixed, if only by a DECREF in all paths and
an INCREF in the fast path.  However, this will worsen the "fast case"
performance figures at least a tiny bit.

[guido again]
> What are you going to do with all the *other* places where
> PyDict_GetItem() is used?

Which places do you have in mind?  The common feature request seems to
be for globals() only, and I can't immediately think of a way of
accessing globals() that my patch doesn't (attempt to) cover.

Unfortunately, I'm not particularly excited about this capability
myself.

What I think this does show, however, is that using a sequence like
(not quite C code)
    inline PyObject *PyObject_GetItem_FastForDicts(o, k) {
        if(PyDict_CheckExact(o)) {
            ret = PyDict_GetItem(o, k);
            Py_INCREF(ret);
            return ret;
        }
        return PyObject_GetItem(o, k);
    }
will impose in-the-noise penalties for 'dict' instances, modest
penalties for 'dict' subclass instances without __getitem__, and
provide the full power for general objects.


On Tue, Oct 29, 2002 at 08:57:01AM +0100, M.-A. Lemburg wrote:
> If all you want is a way to define a default value for a failing
> dictionary lookup, wouldn't it be better to simply add this feature
> to the standard dictionary implementation ?

It doesn't seem to be quite this simple, because you almost certainly
want to have items in the builtin namespace appear even if you want to
give a default on what would eventually be a NameError for a
LOAD_GLOBAL/LOAD_NAME.  You end up with something like
    class D(dict):
        def __getitem__(self, item):
            try:
                return dict.__getitem__(self, item)
            except KeyError:
                try:
                    return getattr(__builtins__, item)
                except AttributeError:
                    return "XX"
which is more complicated that a "simple" defaulting dictionary.

Unfortunately, this is another time when I am not a strong proponent of a 
feature, but merely wrote the patch to see if the "common wisdom" on which
the rejection was based is valid.  (common wisdom's been right in earlier
cases, but I'm tempted to call the speed penalties for this change
entirely tolerable if the purposes of people who do want this patch are
good ones)


On Tue, Oct 29, 2002 at 12:11:12AM -0500, Raymond Hettinger wrote
[in a private message]:
> Do you know why the following snippet doesn't work with your patch?
>
> class SpreadSheet(dict):
>     def __getitem__(self, key):
>         return eval(dict.__getitem__(self,key), self)
>
> ss = SpreadSheet()
> ss['a1'] = '5'
> ss['a2'] = 'a1*6'
> ss['a3'] = 'a2*a1'
> print ss['a3']

It took me awhile to figure this out...

LOAD_NAME first tries to load from the locals dict, where I did not
change the way that name resolution functions.  eval() uses
the same dict for locals and globals if the locals (third arg) are not
directly specified.

The following works:
    class SpreadSheet(dict):
        def __getitem__(self, key):
            return eval(dict.__getitem__(self,key), self, {})

Jeff


From guido@python.org  Tue Oct 29 15:53:16 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 10:53:16 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 08:34:26 CST."
 <20021029143416.GA16099@unpythonic.net>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net>
 <20021029143416.GA16099@unpythonic.net>
Message-ID: <200210291553.g9TFrGJ01406@odiug.zope.com>

> > I don't see the extra DECREF here (and below) needed to compensate for
> > the fact that PyObject_GetItem() returns the object with incremented
> > refcount but PyDict_GetItem() doesn't.
> 
> oops.  I believe this can be fixed, if only by a DECREF in all paths
> and an INCREF in the fast path.  However, this will worsen the "fast
> case" performance figures at least a tiny bit.

I'm already unhappy about the performance hit of your patch.  We're
trying very hard to make global and built-in access as fast as
possible.  This patch adds extra local variable references to the
"fast path".  More INCREFS and DECREFS will slow it down even more.
Not just because there's more executed code, but also because there's
more code, period, which may reduce the I-cache effectiveness.

> [guido again]
> > What are you going to do with all the *other* places where
> > PyDict_GetItem() is used?
> 
> Which places do you have in mind?  The common feature request seems
> to be for globals() only, and I can't immediately think of a way of
> accessing globals() that my patch doesn't (attempt to) cover.

There are lots of other places where PyDict_*Item is used for an
object that could be a dict subclass.  Unless you've tracked all these
down I think it wouldn't be wise to apply this patch.  We either say
(as we say now) "you can't override __getitem__ in a dict passed into
the Python interpreter and expect it to work" or we say "you can
override __getitem__ and it will work".

It would be really bad to say "you can override __getitem__ and it
will usually work but not in these twelve situations, because we
didn't bother to change legacy code."

> Unfortunately, I'm not particularly excited about this capability
> myself.

Me neither, to say the least.  I still feel hostile, both because of
the performance hit, and because I worry that it will be used to
create obfuscated code that *looks* like Python but isn't Python.

> What I think this does show, however, is that using a sequence like
> (not quite C code)
>     inline PyObject *PyObject_GetItem_FastForDicts(o, k) {
>         if(PyDict_CheckExact(o)) {
>             ret = PyDict_GetItem(o, k);
>             Py_INCREF(ret);
>             return ret;
>         }
>         return PyObject_GetItem(o, k);
>     }
> will impose in-the-noise penalties for 'dict' instances, modest
> penalties for 'dict' subclass instances without __getitem__, and
> provide the full power for general objects.

I'd like to see more benchmarking of this (as well as more motivation).

> On Tue, Oct 29, 2002 at 08:57:01AM +0100, M.-A. Lemburg wrote:
> > If all you want is a way to define a default value for a failing
> > dictionary lookup, wouldn't it be better to simply add this feature
> > to the standard dictionary implementation ?
> 
> It doesn't seem to be quite this simple, because you almost certainly
> want to have items in the builtin namespace appear even if you want to
> give a default on what would eventually be a NameError for a
> LOAD_GLOBAL/LOAD_NAME.  You end up with something like
>     class D(dict):
>         def __getitem__(self, item):
>             try:
>                 return dict.__getitem__(self, item)
>             except KeyError:
>                 try:
>                     return getattr(__builtins__, item)
>                 except AttributeError:
>                     return "XX"
> which is more complicated that a "simple" defaulting dictionary.
> 
> Unfortunately, this is another time when I am not a strong proponent of a 
> feature, but merely wrote the patch to see if the "common wisdom" on which
> the rejection was based is valid.  (common wisdom's been right in earlier
> cases, but I'm tempted to call the speed penalties for this change
> entirely tolerable if the purposes of people who do want this patch are
> good ones)
> 
> 
> On Tue, Oct 29, 2002 at 12:11:12AM -0500, Raymond Hettinger wrote
> [in a private message]:
> > Do you know why the following snippet doesn't work with your patch?
> >
> > class SpreadSheet(dict):
> >     def __getitem__(self, key):
> >         return eval(dict.__getitem__(self,key), self)
> >
> > ss = SpreadSheet()
> > ss['a1'] = '5'
> > ss['a2'] = 'a1*6'
> > ss['a3'] = 'a2*a1'
> > print ss['a3']
> 
> It took me awhile to figure this out...
> 
> LOAD_NAME first tries to load from the locals dict, where I did not
> change the way that name resolution functions.

Another deficiency of the patch.

> eval() uses
> the same dict for locals and globals if the locals (third arg) are not
> directly specified.
> 
> The following works:
>     class SpreadSheet(dict):
>         def __getitem__(self, key):
>             return eval(dict.__getitem__(self,key), self, {})
> 
> Jeff

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jepler@unpythonic.net  Tue Oct 29 17:10:25 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 29 Oct 2002 11:10:25 -0600
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <200210291553.g9TFrGJ01406@odiug.zope.com>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net> <20021029143416.GA16099@unpythonic.net> <200210291553.g9TFrGJ01406@odiug.zope.com>
Message-ID: <20021029171024.GC18602@unpythonic.net>

Still working from the version which is missing DECREFs, I get these
timings from a program intended to more directly measure the speed of
local, global, and builtin name lookups:

Vanilla Python CVS.  In effect these are three runs of the same thing,
so you get some idea of the timing variations on my system.
    $ python-cvs/python lookup-speed.py
    Timing 4000000 name lookups
        type  builtin   global    local overhead
        dict  2.74015  1.99215  1.21323  0.17610
           D  2.69901  1.99129  1.24662  0.17832
           E  2.73839  2.03171  1.25233  0.17843

CVS with my original patch (leaks references).  "D" is a dict subclass
which doesn't define __getitem__, "E" is a dict subclass which does
(simply calling dict.__getitem__ and returning the result)
    $ python-dict/python lookup-speed.py
    Timing 4000000 name lookups
        type  builtin   global    local overhead
        dict  2.81268  2.04795  1.26585  0.18117
           D  5.08308  2.85853  1.23625  0.17944
           E 81.62213 64.60481  1.26524  0.17973
This is a ~3% slowdown on the test with the heaviest use of builtins
and globals that I can think of offhand.  Slowdown:
            builtin global
     dict   3%      3%
        D   80%     40%
        E   *28     *31

I'm about done pretending to cheerlead for this patch, though if I fix
the reference problem I'll post the revised diffs.

Jeff


From jeremy@alum.mit.edu  Tue Oct 29 17:14:53 2002
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Tue, 29 Oct 2002 12:14:53 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <20021029171024.GC18602@unpythonic.net>
References: <20021028203952.A1460@unpythonic.net>
 <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net>
 <20021029143416.GA16099@unpythonic.net>
 <200210291553.g9TFrGJ01406@odiug.zope.com>
 <20021029171024.GC18602@unpythonic.net>
Message-ID: <15806.49677.886970.115803@slothrop.zope.com>

>>>>> "JE" == Jeff Epler <jepler@unpythonic.net> writes:

  JE> I'm about done pretending to cheerlead for this patch, though if
  JE> I fix the reference problem I'll post the revised diffs.

I'm not enthusiastic about this patch, but I think it's really helpful
that you've done this work.  A concrete patch, albeit incomplete, and
some real performance measurements improves the quality of discussion
on this issue.

Did you post the lookup-speed.py script in an earlier message?  I
don't think I know what exactly you are measuring.

Jeremy



From guido@python.org  Tue Oct 29 17:14:57 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 12:14:57 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 11:10:25 CST."
 <20021029171024.GC18602@unpythonic.net>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net> <20021029143416.GA16099@unpythonic.net> <200210291553.g9TFrGJ01406@odiug.zope.com>
 <20021029171024.GC18602@unpythonic.net>
Message-ID: <200210291714.g9THEwP02615@odiug.zope.com>

1. You've got to show your test code.

2. Have you tried pybench?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jepler@unpythonic.net  Tue Oct 29 17:18:29 2002
From: jepler@unpythonic.net (Jeff Epler)
Date: Tue, 29 Oct 2002 11:18:29 -0600
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <20021029171024.GC18602@unpythonic.net>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net> <20021029143416.GA16099@unpythonic.net> <200210291553.g9TFrGJ01406@odiug.zope.com> <20021029171024.GC18602@unpythonic.net>
Message-ID: <20021029171828.GD18602@unpythonic.net>

Here's the lookup-speed script.

num = 1000000
code = """
x = None
r = range(%d)
def builtin_lookup():
    for i in r:
        len; len; len; len
        len; len; len; len
        len; len; len; len
        len; len; len; len

def global_lookup():
    for i in r:
        x; x; x; x
        x; x; x; x
        x; x; x; x
        x; x; x; x

def local_lookup():
    x = None
    for i in r:
        x; x; x; x
        x; x; x; x
        x; x; x; x
        x; x; x; x

def overhead():
    for i in r: pass
""" % num

class D(dict): pass
class E(dict): __getitem__ = lambda s, k: dict.__getitem__(s, k)

print "Timing %d name lookups" % (num*4)
print "%8s %8s %8s %8s %8s" % ("type", "builtin", "global", "local", "overhead")

import time
for dc in dict, D, E:
    d = dc()
    exec code in d
    print "%8s" % dc.__name__,
    for f in ("builtin_lookup()", "global_lookup()", "local_lookup()", "overhead()"):
        t0 = time.time()
        exec f in d
        t1 = time.time()
        print "%8.5f" % (t1-t0),
    print


From guido@python.org  Tue Oct 29 17:28:29 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 12:28:29 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 12:14:53 EST."
 <15806.49677.886970.115803@slothrop.zope.com>
References: <20021028203952.A1460@unpythonic.net> <200210291209.g9TC9M924037@pcp02138704pcs.reston01.va.comcast.net> <20021029143416.GA16099@unpythonic.net> <200210291553.g9TFrGJ01406@odiug.zope.com> <20021029171024.GC18602@unpythonic.net>
 <15806.49677.886970.115803@slothrop.zope.com>
Message-ID: <200210291728.g9THSTb06525@odiug.zope.com>

BTW, the (final) patch, test script, and commentary should be uploaded
to the SF patch manager.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From DavidA@ActiveState.com  Tue Oct 29 18:20:28 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Tue, 29 Oct 2002 10:20:28 -0800
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au> <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <3DBED16C.90801@ActiveState.com>

Guido van Rossum wrote:
>>I wonder how many reasonable real-life scenarios actually exist for
>>this feature?
>>
>>My personal use would be for COM, to implement case-insensitive
>>namespaces.  In some cases, case-insensitivity is actually part of
>>the implementation specification, and I can imagine other scenarios
>>where case insensitive namespaces are "reasonable" even in the face
>>of Python's excellent decision not to go this route.
>>
>>I wonder is this is actually the root of many requests to see this
>>feature implemented?  If not, it would be interesting to see what
>>the competing desires really are.
> 
> 
> Excellent observation.  Perhaps the people who want this can get
> together and write a PEP that explains the motivation and why the
> alternatives don't work?  (E.g. when you need case-insensitive
> variable names, perhaps it's just as easy to preprocess the source
> code?)

I can think of two 'real world' use cases for this patch that I remember.

1) When I was consulting for a finite element modeling company that was using 
Python to manipulate _very_ large objects through a database.  I don't recall 
the details, but basically they wanted to be able to control database connection 
lifetimes in a way that wasn't possible at the time (Python 1.4 or so, I think 
=).  These days I think that proxies would probably be leveraged to provide the 
capability, but namespace lookup would still be useful -- one objection they had 
to using proxies is that they didn't want their users to think in terms of 
objects with attributes -- they wanted their users to say:

   a = b + c

and have the system figure out at runtime whether to evaluate b, a DB connection 
needed to be established, or not.  None of the alternatives I thought of at the 
time "worked" for various reasons.  Sorry I'm so vague, it's been a long time. 
(I don't even remember how the problem was resolved).

2) The other case where I would have used this is in building a high-power 
interactive shell for scientific users doing modeling (think Matlab on 
steroids), where I wanted the interpreter to be able to resolve names at runtime 
from pickled states -- that way the memory load was dependent on actual usage, 
but the "impression" to the user would have been that the system "never forgot".

In both of these cases, Guido would be right in saying that that's not Python, 
but that's somewhat irrelevant -- the end-users would have had a "better" user 
experience with the feature on, and other aspects of the system would have made 
it quite clear that this wasn't vanilla Python.

I like the feature, not necessarily for inclusion in core Python but as a way of 
experimenting with alternate universes in a research mode:  "What would a 
PHP-like 'isset()' feel like?" "Could we do something akin to Tcl's TclObj dual 
representations?"  Name resolution has always felt like the least "hookable" of 
Python's mechanisms (as opposed to attribute lookup, exception handling, etc.), 
and I wonder what good things could come of making it more open to experimentation.

--david

PS: Thinking of PHP, the fact that I can look up variables defined in sessions 
or cookies or whatnot really really easily is one of the reasons I'm liking PHP 
more than Python for web programming these days.  I don't know enough of the 
technical details of the web to know automatically where what data is stored and 
in what format -- and PHP doesn't make me.  Maybe web folks would use the 
dict-subclass feature to emulate this aspect of PHP.



From guido@python.org  Tue Oct 29 18:29:16 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 13:29:16 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 10:20:28 PST."
 <3DBED16C.90801@ActiveState.com>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au> <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>
 <3DBED16C.90801@ActiveState.com>
Message-ID: <200210291829.g9TITGq32726@odiug.zope.com>

> I can think of two 'real world' use cases for this patch that I
> remember.
> 
> 1) When I was consulting for a finite element modeling company that
> was using Python to manipulate _very_ large objects through a
> database.  I don't recall the details, but basically they wanted to
> be able to control database connection lifetimes in a way that
> wasn't possible at the time (Python 1.4 or so, I think =).  These
> days I think that proxies would probably be leveraged to provide the
> capability, but namespace lookup would still be useful -- one
> objection they had to using proxies is that they didn't want their
> users to think in terms of objects with attributes -- they wanted
> their users to say:
> 
>    a = b + c
> 
> and have the system figure out at runtime whether to evaluate b, a
> DB connection needed to be established, or not.  None of the
> alternatives I thought of at the time "worked" for various reasons.
> Sorry I'm so vague, it's been a long time.  (I don't even remember
> how the problem was resolved).
> 
> 2) The other case where I would have used this is in building a
> high-power interactive shell for scientific users doing modeling
> (think Matlab on steroids), where I wanted the interpreter to be
> able to resolve names at runtime from pickled states -- that way the
> memory load was dependent on actual usage, but the "impression" to
> the user would have been that the system "never forgot".
> 
> In both of these cases, Guido would be right in saying that that's
> not Python, but that's somewhat irrelevant -- the end-users would
> have had a "better" user experience with the feature on, and other
> aspects of the system would have made it quite clear that this
> wasn't vanilla Python.

I think (2) could be resolved using "ghosts" -- this is what Zope's
persistency mechanism uses.  The names must be present, but their
content is loaded on demand.  I expect that (1) probably could also be
solved this way.

> I like the feature, not necessarily for inclusion in core Python but
> as a way of experimenting with alternate universes in a research
> mode: "What would a PHP-like 'isset()' feel like?" "Could we do
> something akin to Tcl's TclObj dual representations?"  Name
> resolution has always felt like the least "hookable" of Python's
> mechanisms (as opposed to attribute lookup, exception handling,
> etc.), and I wonder what good things could come of making it more
> open to experimentation.

That's a good question.

> --david
> 
> PS: Thinking of PHP, the fact that I can look up variables defined
> in sessions or cookies or whatnot really really easily is one of the
> reasons I'm liking PHP more than Python for web programming these
> days.  I don't know enough of the technical details of the web to
> know automatically where what data is stored and in what format --
> and PHP doesn't make me.  Maybe web folks would use the
> dict-subclass feature to emulate this aspect of PHP.

But why not make it an attribute lookup?  That's how Zope (again :-)
deals with this.  Remember acquisition? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From DavidA@ActiveState.com  Tue Oct 29 19:05:06 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Tue, 29 Oct 2002 11:05:06 -0800
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au> <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>              <3DBED16C.90801@ActiveState.com> <200210291829.g9TITGq32726@odiug.zope.com>
Message-ID: <3DBEDBE2.6060708@ActiveState.com>

Guido van Rossum wrote:


 > I think (2) could be resolved using "ghosts" -- this is what Zope's
 > persistency mechanism uses.  The names must be present, but their
 > content is loaded on demand.  I expect that (1) probably could also be
 > solved this way.

That assumes that you can "preload" all of the names at initialization, correct? 
  IOW, that you know the contents of the namespace in advance.  Not a huge 
limitation, but not the same as lazy ghost-creation on lookup.

>>PS: Thinking of PHP, the fact that I can look up variables defined
>>in sessions or cookies or whatnot really really easily is one of the
>>reasons I'm liking PHP more than Python for web programming these
>>days.  I don't know enough of the technical details of the web to
>>know automatically where what data is stored and in what format --
>>and PHP doesn't make me.  Maybe web folks would use the
>>dict-subclass feature to emulate this aspect of PHP.
> 
> 
> But why not make it an attribute lookup?  That's how Zope (again :-)
> deals with this.  Remember acquisition? :-)

Yah, I remember.  I'm not saying that one can't come up with a pythonic yet 
low-impact way of presenting web constructs to users.  I guess I'm just 
expressing frustration with the fact that in PHP I don't really need to 
understand the web, whereas in Python I feel I have to.   Seems wrong. =)

--da



From guido@python.org  Tue Oct 29 19:11:13 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 14:11:13 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 11:05:06 PST."
 <3DBEDBE2.6060708@ActiveState.com>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au> <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net> <3DBED16C.90801@ActiveState.com> <200210291829.g9TITGq32726@odiug.zope.com>
 <3DBEDBE2.6060708@ActiveState.com>
Message-ID: <200210291911.g9TJBDg13209@odiug.zope.com>

>  > I think (2) could be resolved using "ghosts" -- this is what Zope's
>  > persistency mechanism uses.  The names must be present, but their
>  > content is loaded on demand.  I expect that (1) probably could also be
>  > solved this way.
> 
> That assumes that you can "preload" all of the names at
> initialization, correct?  IOW, that you know the contents of the
> namespace in advance.  Not a huge limitation, but not the same as
> lazy ghost-creation on lookup.

Correct.  I don't see this as a huge limitation -- all "autoload"
features that I'm familiar with (Emacs, Tcl; how about Perl?) require
you to know at least the auto-loaded names in advance.  (Also in my
defence you only mentioned the cost of unpickling everything as an
argument. :-)

> >>PS: Thinking of PHP, the fact that I can look up variables defined
> >>in sessions or cookies or whatnot really really easily is one of the
> >>reasons I'm liking PHP more than Python for web programming these
> >>days.  I don't know enough of the technical details of the web to
> >>know automatically where what data is stored and in what format --
> >>and PHP doesn't make me.  Maybe web folks would use the
> >>dict-subclass feature to emulate this aspect of PHP.
> > 
> > 
> > But why not make it an attribute lookup?  That's how Zope (again :-)
> > deals with this.  Remember acquisition? :-)
> 
> Yah, I remember.  I'm not saying that one can't come up with a
> pythonic yet low-impact way of presenting web constructs to users.
> I guess I'm just expressing frustration with the fact that in PHP I
> don't really need to understand the web, whereas in Python I feel I
> have to.  Seems wrong. =)

I guess this depends on which Python application you use.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Tue Oct 29 19:19:29 2002
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 29 Oct 2002 14:19:29 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <200210291911.g9TJBDg13209@odiug.zope.com>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au>
 <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>
 <3DBED16C.90801@ActiveState.com>
 <200210291829.g9TITGq32726@odiug.zope.com>
 <3DBEDBE2.6060708@ActiveState.com>
 <200210291911.g9TJBDg13209@odiug.zope.com>
Message-ID: <a05111b0eb9e48ed6b4ea@[63.120.19.221]>

At 2:11 PM -0500 10/29/02, Guido van Rossum wrote:
>  >  > I think (2) could be resolved using "ghosts" -- this is what Zope's
>>   > persistency mechanism uses.  The names must be present, but their
>>   > content is loaded on demand.  I expect that (1) probably could also be
>>   > solved this way.
>>
>>  That assumes that you can "preload" all of the names at
>>  initialization, correct?  IOW, that you know the contents of the
>>  namespace in advance.  Not a huge limitation, but not the same as
>>  lazy ghost-creation on lookup.
>
>Correct.  I don't see this as a huge limitation -- all "autoload"
>features that I'm familiar with (Emacs, Tcl; how about Perl?) require
>you to know at least the auto-loaded names in advance.  (Also in my
>defence you only mentioned the cost of unpickling everything as an
>argument. :-)

Perl doesn't require it. If a method call is made but that method 
name doesn't appear in the object's inheritance hierarchy, perl walks 
up the tree looking for a method named AUTOLOAD and if it finds one 
calls that. The AUTOLOAD method then has the opportunity to do 
something--satisfy the call, dispatch to a parent class, or pitch a 
fit. It's valid for an AUTOLOAD method to satisfy the request without 
instantiating a method for it. (In which case the next invocation of 
that method will end up back in AUTOLOAD)

Works for subroutines too, though the hierarchy isn't supposed to be 
walked for those. (Older versions of perl would do it, but that's 
been deprecated)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From guido@python.org  Tue Oct 29 19:26:26 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 14:26:26 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: Your message of "Tue, 29 Oct 2002 14:19:29 EST."
 <a05111b0eb9e48ed6b4ea@[63.120.19.221]>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au> <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net> <3DBED16C.90801@ActiveState.com> <200210291829.g9TITGq32726@odiug.zope.com> <3DBEDBE2.6060708@ActiveState.com> <200210291911.g9TJBDg13209@odiug.zope.com>
 <a05111b0eb9e48ed6b4ea@[63.120.19.221]>
Message-ID: <200210291926.g9TJQQs13694@odiug.zope.com>

> >Correct.  I don't see this as a huge limitation -- all "autoload"
> >features that I'm familiar with (Emacs, Tcl; how about Perl?) require
> >you to know at least the auto-loaded names in advance.  (Also in my
> >defence you only mentioned the cost of unpickling everything as an
> >argument. :-)
> 
> Perl doesn't require it. If a method call is made but that method 
> name doesn't appear in the object's inheritance hierarchy, perl walks 
> up the tree looking for a method named AUTOLOAD and if it finds one 
> calls that. The AUTOLOAD method then has the opportunity to do 
> something--satisfy the call, dispatch to a parent class, or pitch a 
> fit. It's valid for an AUTOLOAD method to satisfy the request without 
> instantiating a method for it. (In which case the next invocation of 
> that method will end up back in AUTOLOAD)

Um, as long we're doing *method* lookup, Python's __getattr__ can do
all that.

> Works for subroutines too, though the hierarchy isn't supposed to be 
> walked for those. (Older versions of perl would do it, but that's 
> been deprecated)

OK, so if there's a global AUTOLOAD and no global x, looking for x
will call AUTOLOAD to satisfy x, right?  Python doesn't do this, and
this is what the "hookable namespace" feature is aiming at.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Tue Oct 29 19:44:17 2002
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 29 Oct 2002 14:44:17 -0500
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <200210291926.g9TJQQs13694@odiug.zope.com>
References: <LCEPIIGDJPKCOIHOBJEPIEFIHGAA.mhammond@skippinet.com.au>
 <200210291243.g9TChoK24426@pcp02138704pcs.reston01.va.comcast.net>
 <3DBED16C.90801@ActiveState.com>
 <200210291829.g9TITGq32726@odiug.zope.com>
 <3DBEDBE2.6060708@ActiveState.com>
 <200210291911.g9TJBDg13209@odiug.zope.com>
 <a05111b0eb9e48ed6b4ea@[63.120.19.221]>
 <200210291926.g9TJQQs13694@odiug.zope.com>
Message-ID: <a05111b0fb9e4943df8fe@[63.120.19.221]>

At 2:26 PM -0500 10/29/02, Guido van Rossum wrote:
>  > >Correct.  I don't see this as a huge limitation -- all "autoload"
>>  >features that I'm familiar with (Emacs, Tcl; how about Perl?) require
>>  >you to know at least the auto-loaded names in advance.  (Also in my
>>  >defence you only mentioned the cost of unpickling everything as an
>>  >argument. :-)
>>
>>  Perl doesn't require it. If a method call is made but that method
>>  name doesn't appear in the object's inheritance hierarchy, perl walks
>>  up the tree looking for a method named AUTOLOAD and if it finds one
>>  calls that. The AUTOLOAD method then has the opportunity to do
>>  something--satisfy the call, dispatch to a parent class, or pitch a
>>  fit. It's valid for an AUTOLOAD method to satisfy the request without
>>  instantiating a method for it. (In which case the next invocation of
>>  that method will end up back in AUTOLOAD)
>
>Um, as long we're doing *method* lookup, Python's __getattr__ can do
>all that.

Sorry--it's a twisty conversation of little passages. I got somewhat 
turned around.

>  > Works for subroutines too, though the hierarchy isn't supposed to be
>>  walked for those. (Older versions of perl would do it, but that's
>>  been deprecated)
>
>OK, so if there's a global AUTOLOAD and no global x, looking for x
>will call AUTOLOAD to satisfy x, right?  Python doesn't do this, and
>this is what the "hookable namespace" feature is aiming at.

If there's an AUTOLOAD in a package, no X in that package, and you 
call X, then that package's AUTOLOAD will get called instead, yes.

So if you had:

    print(Foo::Bar());

to print the output of the subroutine Bar in the package Foo, and 
there was no subroutine Bar in the package Foo but there *is* a 
subroutine AUTOLOAD in the package Foo, then Foo's AUTOLOAD will get 
called instead. Appropriate variables are filled in so that AUTOLOAD 
method can tell what sub it was called for.

Only works for subs, not for global variables. (That's for perl 6, I expect)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk


From pobrien@orbtech.com  Tue Oct 29 22:21:58 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Tue, 29 Oct 2002 16:21:58 -0600
Subject: [Python-Dev] Keyword module cruft question
Message-ID: <200210291621.58649.pobrien@orbtech.com>

I'm working on an article about Python introspection and came across some=
thing=20
I consider a blemish. I thought I'd ask about it here and see how you fol=
ks=20
feel about it. BTW, I'm looking at Python 2.2.2 on Linux.

If you look at the keyword module, you'll see that it has a keyword attri=
bute:

>>> import keyword
>>> dir(keyword)
['__all__', '__builtins__', '__doc__', '__file__', '__name__', 'iskeyword=
',=20
'keyword', 'kwdict', 'kwlist', 'main']
>>> keyword.keyword
'yield'
>>>=20

Basically, the keyword attribute is just the last value from this process=
:

kwdict =3D {}
for keyword in kwlist:
    kwdict[keyword] =3D 1

So it is a bit of extraneous cruft, I believe. No big deal, and nothing t=
hat a

del keyword

couldn't fix. But it does look sloppy when you are writing about introspe=
ction=20
and you see an attribute that has no real value. I wish it weren't there,=
 as=20
I'd rather not have to explain it. But it got me thinking whether other=20
modules in the standard library have this issue, and whether this is=20
something that should be cleaned up. Thoughts?

--=20
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------



From guido@python.org  Tue Oct 29 22:41:57 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 17:41:57 -0500
Subject: [Python-Dev] Keyword module cruft question
In-Reply-To: Your message of "Tue, 29 Oct 2002 16:21:58 CST."
 <200210291621.58649.pobrien@orbtech.com>
References: <200210291621.58649.pobrien@orbtech.com>
Message-ID: <200210292241.g9TMfvE16004@odiug.zope.com>

> I'm working on an article about Python introspection and came across
> something I consider a blemish. I thought I'd ask about it here and
> see how you folks feel about it. BTW, I'm looking at Python 2.2.2 on
> Linux.
> 
> If you look at the keyword module, you'll see that it has a keyword
> attribute:
> 
> >>> import keyword
> >>> dir(keyword)
> ['__all__', '__builtins__', '__doc__', '__file__', '__name__', 'iskeyword', 
> 'keyword', 'kwdict', 'kwlist', 'main']
> >>> keyword.keyword
> 'yield'
> >>> 
> 
> Basically, the keyword attribute is just the last value from this
> process:
> 
> kwdict = {}
> for keyword in kwlist:
>     kwdict[keyword] = 1
> 
> So it is a bit of extraneous cruft, I believe. No big deal, and
> nothing that a
> 
> del keyword
> 
> couldn't fix. But it does look sloppy when you are writing about
> introspection and you see an attribute that has no real value. I
> wish it weren't there, as I'd rather not have to explain it. But it
> got me thinking whether other modules in the standard library have
> this issue, and whether this is something that should be cleaned
> up. Thoughts?

Lots of modules have attributes that are cruft according to this
definition.  I personally see no reason to get rid of all these --
they're harmless and "from M import *" doesn't import them.  Feel free
to hold your own code to a higher standard though. ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Tue Oct 29 22:43:52 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 29 Oct 2002 23:43:52 +0100
Subject: [Python-Dev] Patch to use dict subclasses in eval(), exec
In-Reply-To: <200210291829.g9TITGq32726@odiug.zope.com>
Message-ID: <E5F1F2B6-EB8F-11D6-BDB7-003065517236@oratrix.com>

If at some point it's decided that this patch is a good idea I 
think we can get rid of the performance hit in the normal case 
by duplicating eval_code2() with some preprocessor magic. 
eval_code2() would then call out to eval_code2_non_dict() if the 
globals weren't in a dict.

Hmm, I wonder whether there are other situations where this idea 
would work too, i.e. situations where eval_code2() could decide 
at startup that it could take a fast path...
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From martin@v.loewis.de  Tue Oct 29 23:02:16 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 00:02:16 +0100
Subject: [Python-Dev] Keyword module cruft question
In-Reply-To: <200210291621.58649.pobrien@orbtech.com>
References: <200210291621.58649.pobrien@orbtech.com>
Message-ID: <m3y98gu9vr.fsf@mira.informatik.hu-berlin.de>

"Patrick K. O'Brien" <pobrien@orbtech.com> writes:

> But it does look sloppy when you are writing about introspection and
> you see an attribute that has no real value. I wish it weren't
> there, as I'd rather not have to explain it. But it got me thinking
> whether other modules in the standard library have this issue, and
> whether this is something that should be cleaned up. Thoughts?

Normally, this is fixed with __all__: The module may have arbitrary
variables; the interface is only those identifiers mentioned in
__all__ (in absence of __all__, its only those identifiers which are
documented).

So I don't think this is an issue, in general. In a specific case,
there is nothing wrong with fixing it.

Regards,
Martin


From pobrien@orbtech.com  Tue Oct 29 23:27:19 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Tue, 29 Oct 2002 17:27:19 -0600
Subject: [Python-Dev] Keyword module cruft question
In-Reply-To: <m3y98gu9vr.fsf@mira.informatik.hu-berlin.de>
References: <200210291621.58649.pobrien@orbtech.com> <m3y98gu9vr.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210291727.19495.pobrien@orbtech.com>

On Tuesday 29 October 2002 05:02 pm, Martin v. Loewis wrote:
> "Patrick K. O'Brien" <pobrien@orbtech.com> writes:
> > But it does look sloppy when you are writing about introspection and
> > you see an attribute that has no real value. I wish it weren't
> > there, as I'd rather not have to explain it. But it got me thinking
> > whether other modules in the standard library have this issue, and
> > whether this is something that should be cleaned up. Thoughts?
>
> Normally, this is fixed with __all__: The module may have arbitrary
> variables; the interface is only those identifiers mentioned in
> __all__ (in absence of __all__, its only those identifiers which are
> documented).

Well, in this case, keyword's __all__ looks like:

__all__ =3D ["iskeyword"]

Which leaves out keyword.kwlist, a pretty useful attribute.

> So I don't think this is an issue, in general. In a specific case,
> there is nothing wrong with fixing it.

I'd also suggest that __all__ as a determiner of a module's api is severe=
ly=20
underdocumented.

Maybe I just picked the wrong module to start off my article. ;-)

--=20
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------



From pobrien@orbtech.com  Tue Oct 29 23:52:20 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Tue, 29 Oct 2002 17:52:20 -0600
Subject: [Python-Dev] Keyword module cruft question
In-Reply-To: <200210292241.g9TMfvE16004@odiug.zope.com>
References: <200210291621.58649.pobrien@orbtech.com> <200210292241.g9TMfvE16004@odiug.zope.com>
Message-ID: <200210291752.20822.pobrien@orbtech.com>

On Tuesday 29 October 2002 04:41 pm, Guido van Rossum wrote:
> Lots of modules have attributes that are cruft according to this
> definition.  I personally see no reason to get rid of all these --
> they're harmless and "from M import *" doesn't import them.  Feel free
> to hold your own code to a higher standard though. ;-)

I understand what you are saying, and respectfully disagree. Extraneous=20
attributes are harmful to the extent that one has to know to ignore them.=
=20
Knowing what to ignore is difficult when learning Python or working with =
an=20
unfamiliar module. And extraneous attributes are cumbersome when making u=
se=20
of Python's introspection features, such as dir(object), or when using a=20
Python shell or editor that offers autocompletion. And, finally, they get=
 in=20
the way while writing an article about Python introspection. ;-)

Also, little things like this make it more challenging to present Python =
in=20
its most positive light for a general audience, featuring characteristics=
 of=20
the language that are supposed to show how it is a better choice than oth=
er=20
languages. I'm not saying that having a few extraneous attributes is a hu=
ge=20
flaw. And you know how much of a fan of Python I am. But anything that ha=
s to=20
be "brushed under the rug" can create doubt among those who aren't alread=
y=20
part of the Python community. And I don't think a higher standard would h=
urt=20
the standard library.

Just something to think about.

--=20
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------



From martin@v.loewis.de  Wed Oct 30 00:14:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 01:14:41 +0100
Subject: [Python-Dev] Incorporating bsddb3
Message-ID: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>

[Since the last message triggered no response, I'm retrying.]

I have changes in my sandbox to incorporate bsddb3 into Python
proper. This essentially is a copy of bsddb3 3.4.0, with the following
modifications:
- the extension module is called _bsddb (not bsddb3._db)
- the package is called bsddb (not bsddb3)

Both test_bsddb and test_anydbm continue to work unmodified.

Is it ok to commit these changes?

Regards,
Martin


From guido@python.org  Wed Oct 30 01:38:54 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 29 Oct 2002 20:38:54 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 01:14:41 +0100."
 <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>

> I have changes in my sandbox to incorporate bsddb3 into Python
> proper. This essentially is a copy of bsddb3 3.4.0, with the following
> modifications:
> - the extension module is called _bsddb (not bsddb3._db)
> - the package is called bsddb (not bsddb3)
> 
> Both test_bsddb and test_anydbm continue to work unmodified.

What happens if you don't have the software installed to make this
work, but you do have what it takes to make the *old* bsddb module
work?  I presume that one bites the dust now?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Wed Oct 30 05:02:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 06:02:11 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> What happens if you don't have the software installed to make this
> work, but you do have what it takes to make the *old* bsddb module
> work?  I presume that one bites the dust now?

That would be bsddb2, which is still supported in the old
bsddbmodule.c, but not bsddb3.

I would propose to continue shipping bsddbmodule.c. Building and using
that would then require manual intervention.

Regards,
Martin


From guido@python.org  Wed Oct 30 05:12:35 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 00:12:35 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 06:02:11 +0100."
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>

> > What happens if you don't have the software installed to make this
> > work, but you do have what it takes to make the *old* bsddb module
> > work?  I presume that one bites the dust now?
> 
> That would be bsddb2, which is still supported in the old
> bsddbmodule.c, but not bsddb3.
> 
> I would propose to continue shipping bsddbmodule.c. Building and using
> that would then require manual intervention.

Perhaps it could be made into an alternative for _bsddb to make that
simpler (someone could even contribute code to let setup.py decide).

What's the license on the bsddb3 Python extension?

What's the license on the BerkeleyDB code from Sleepycat?  Can we
legally distribute RPMs or other binaries containing it?  (I thought
there were some restrictions that make it not open source.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python@rcn.com  Wed Oct 30 05:19:44 2002
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 30 Oct 2002 00:19:44 -0500
Subject: [Python-Dev] Keyword module cruft question
References: <200210291621.58649.pobrien@orbtech.com> <m3y98gu9vr.fsf@mira.informatik.hu-berlin.de> <200210291727.19495.pobrien@orbtech.com>
Message-ID: <006501c27fd3$f605d6c0$13d8accf@oemcomputer>

[Patrick K. O'Brien]
> Well, in this case, keyword's __all__ looks like:
> 
> __all__ = ["iskeyword"]
> 
> Which leaves out keyword.kwlist, a pretty useful attribute.

Fixed.

> I'd also suggest that __all__ as a determiner of a module's api is severely 
> underdocumented.

Agreed.  Please file a documentation bug report and assign to me.


Raymond Hettinger


From martin@v.loewis.de  Wed Oct 30 05:51:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 06:51:41 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Perhaps it could be made into an alternative for _bsddb to make that
> simpler (someone could even contribute code to let setup.py decide).

Sure.

> What's the license on the bsddb3 Python extension?

See below.

> What's the license on the BerkeleyDB code from Sleepycat?  

There are two licenses: One that they call the "open source license",
see

http://www.sleepycat.com/license.net

There is also a commercial license.

> Can we legally distribute RPMs or other binaries containing it?  (I
> thought there were some restrictions that make it not open source.)

It depends. This is the condition:

# Redistributions in any form must be accompanied by information on
# how to obtain complete source code for the DB software and any
# accompanying software that uses the DB software.  The source code
# must either be included in the distribution or be available for no
# more than the cost of distribution plus a nominal fee, and must be
# freely redistributable under reasonable conditions.

So distributing Python itself should be no problem.

It appears that the Pythonlabs distribution still uses DB 1.85. Don't
tell that Skip :-) He believes that this version has serious flaws
that can lead to data corruption...

Regards,
Martin


From martin@v.loewis.de  Wed Oct 30 06:00:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 07:00:54 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <m3k7k0sbxl.fsf@mira.informatik.hu-berlin.de>

martin@v.loewis.de (Martin v. Loewis) writes:

> See below.

Actually, look here.

/*----------------------------------------------------------------------
  Copyright (c) 1999-2001, Digital Creations, Fredericksburg, VA, USA
  and Andrew Kuchling. All rights reserved.

  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are
  met:

    o Redistributions of source code must retain the above copyright
      notice, this list of conditions, and the disclaimer that follows.

    o Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions, and the following disclaimer in
      the documentation and/or other materials provided with the
      distribution.

    o Neither the name of Digital Creations nor the names of its
      contributors may be used to endorse or promote products derived
      from this software without specific prior written permission.

  THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS AND CONTRIBUTORS *AS
  IS* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
  TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
  PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL DIGITAL
  CREATIONS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
  INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
  BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
  OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
  TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
  USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
  DAMAGE.
------------------------------------------------------------------------*/

It's all yours :-)

Regards,
Martin


From mwh@python.net  Wed Oct 30 13:00:28 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Oct 2002 13:00:28 +0000
Subject: [Python-Dev] metaclass insanity
Message-ID: <2mfzuoytcj.fsf@starship.python.net>

For moderately nefarious reasons[1] I've being trying to write a
metaclass whose instances have writable __bases__.  This in itself
isn't so hard, but having assigments to __bases__ "do the right thing"
has eluded me, basically because I can't seem to affect the mro.

Here's what I have so far:

def base_getter(klass):
    return klass.__priv_bases__

def base_setter(klass, v):
    klass.__priv_bases__ = v

class MetaMutableBases(type):
    def __new__(self, name, bases, ns):
        t = type.__new__(self, name, bases, ns)
        t.__priv_bases__ = type.__dict__["__bases__"].__get__(self)
        return t
    __bases__ = property(base_getter,
                         base_setter)
    def mro(self):
        if hasattr(self, "__priv_bases__"):
            return type("noname", self.__priv_bases__, {}).__mro__
        else:
            return type.mro(self)

I've tried variations on the same, but haven't got anything working yet.

Anyone have any ideas?  I think my brain's exploded.

And I realise this is in some sense more appropriate for c.l.py, but I
think my odds of a good answer are vastly higher here.

Cheers,
M.

[1] in http://groups.google.com/groups?selm=m2r8e99l0b.fsf%40python.net
    I posted a metaclass that attempts to automatically upgrade
    instances of its instances on redefinition.  Ideally, this would
    also frob its subclasses so they now inherited from redefined class.
    I'm aware this can go wrong in various ways, but I think it would be
    pretty handy in a large range of situations.
-- 
  This is not to say C++ = bad, Lisp = good.  It's to say
  C++ = bad irrespective of everything else.
                                       -- Alain Picard, comp.lang.lisp


From guido@python.org  Wed Oct 30 13:45:59 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 08:45:59 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 07:00:54 +0100."
 <m3k7k0sbxl.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net> <m38z0gbju4.fsf@mira.informatik.hu-berlin.de> <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net> <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <m3k7k0sbxl.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210301345.g9UDjxb21402@pcp02138704pcs.reston01.va.comcast.net>

>   Copyright (c) 1999-2001, Digital Creations, Fredericksburg, VA, USA
>   and Andrew Kuchling. All rights reserved.

Now I'm confused.  I thought this was Robun Dunn's bsddb3 module?  Is
there another?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Wed Oct 30 14:06:22 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 30 Oct 2002 09:06:22 -0500
Subject: [Python-Dev] Re: Incorporating bsddb3
Message-ID: <E186tUE-0005pH-00@ute.mems-exchange.org>

GvR wrote:
>Now I'm confused.  I thought this was Robun Dunn's bsddb3 module?  Is
>there another?

It's the same one.  Robin Dunn's original module used SWIG to wrap the
BerkeleyDB API, and I offered to rewrite the interface as a
hand-written extension module, which is why my name turns up.

(Adding my name to the copyright seemed unjustified to me at the time,
though; writing the extension was mostly a trivial process of
rearranging Robin's SWIG definitions with some Emacs/elisp work.)

--amk                                                             (www.amk.ca)
JULIET: O Romeo, Romeo, wherefore art thou Romeo?
      -- _Romeo and Juliet_, II, ii


From jacobs@penguin.theopalgroup.com  Wed Oct 30 14:15:52 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 30 Oct 2002 09:15:52 -0500 (EST)
Subject: [Python-Dev] metaclass insanity
In-Reply-To: <2mfzuoytcj.fsf@starship.python.net>
Message-ID: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com>

On 30 Oct 2002, Michael Hudson wrote:
> For moderately nefarious reasons[1] I've being trying to write a
> metaclass whose instances have writable __bases__.  This in itself
> isn't so hard, but having assigments to __bases__ "do the right thing"
> has eluded me, basically because I can't seem to affect the mro.

The mro is an internal data structure of new-style classes, so redefining
mro() doesn't change the values used.  Here is my (non-working version) that
attempts to re-assign the class of an object, although it fails on a layout
violation with Python 2.2.2.

def base_getter(cls):
  return cls.__my_bases__

def base_setter(cls,bases):
  if not bases:
    bases = (object,)
  metaclass = getattr(cls, '__metaclass__', type)
  new_cls = metaclass(cls.__name__, bases, dict(cls.__dict__))
  cls.__class__ = new_cls

class MetaBase(type):
  __bases__ = property(base_getter,base_setter)

  def __new__(cls, name, bases, ns):
    ns['__my_bases__'] = tuple(bases)
    return super(MetaBase, cls).__new__(cls, name, bases, ns)

class Foo(object):
  __metaclass__ = MetaBase
class Baz(object): pass

Foo.__bases__ = Foo.__bases__ + (Baz,)

Which results in:
  TypeError: __class__ assignment: 'Foo' object layout differs from 'MetaBase'

I haven't looked into why this is being flagged as a layout error, though
my first instinct is to say that the check is too conservative in this case.
I'll think about it more and dig into the code.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From barry@python.org  Wed Oct 30 14:17:19 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 30 Oct 2002 09:17:19 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <m3k7k0sbxl.fsf@mira.informatik.hu-berlin.de>
 <200210301345.g9UDjxb21402@pcp02138704pcs.reston01.va.comcast.net>
Message-ID: <15807.59887.992769.142566@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    >> Copyright (c) 1999-2001, Digital Creations, Fredericksburg, VA,
    >> USA and Andrew Kuchling. All rights reserved.

    GvR> Now I'm confused.  I thought this was Robun Dunn's bsddb3
    GvR> module?  Is there another?

ZC paid Robin to develop bsddb3.

Early on when we were developing the Berkeley storages for ZODB, we
specifically asked Sleepycat about redistribution and they said that
there would be no legal problem to us redistributing source or
binaries with Python, because Python is also open source.  I wouldn't
take my word for that though -- I'd want to get that in writing -- if
we decided to distribute the library.

I'm not sure we'd want to distribute either the source or binary
though.  They're fairly large.  If we did, I'd recommend BerkeleyDB
4.0.14, not the latest version, because bsddb3 doesn't yet build on
4.1.24 because of some incompatible API changes, IIUC.

Note that bsddb3's Windows distro comes with the BerkeleyDB dll's, I
beleive.

Martin, what do you plan to do about the pybsddb test suite?  It's
also fairly large.  If you do add that, I'd suggest not running it by
default, but only with a -resource flag.  Then there's the
documentation and project management issues to talk about.  Maybe we
include none of that, but simply "import" the code into Python and
re-sync whenever it's appropriate.

-Barry


From pobrien@orbtech.com  Wed Oct 30 14:23:58 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Wed, 30 Oct 2002 08:23:58 -0600
Subject: [Python-Dev] Keyword module cruft question
In-Reply-To: <006501c27fd3$f605d6c0$13d8accf@oemcomputer>
References: <200210291621.58649.pobrien@orbtech.com> <200210291727.19495.pobrien@orbtech.com> <006501c27fd3$f605d6c0$13d8accf@oemcomputer>
Message-ID: <200210300823.58866.pobrien@orbtech.com>

On Tuesday 29 October 2002 11:19 pm, Raymond Hettinger wrote:
> > I'd also suggest that __all__ as a determiner of a module's api is
> > severely underdocumented.
>
> Agreed.  Please file a documentation bug report and assign to me.

Done. SF# 631055. Thanks.

--=20
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------



From mwh@python.net  Wed Oct 30 14:36:20 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Oct 2002 14:36:20 +0000
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Kevin Jacobs's message of "Wed, 30 Oct 2002 09:15:52 -0500 (EST)"
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com>
Message-ID: <2md6psyowr.fsf@starship.python.net>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> On 30 Oct 2002, Michael Hudson wrote:
> > For moderately nefarious reasons[1] I've being trying to write a
> > metaclass whose instances have writable __bases__.  This in itself
> > isn't so hard, but having assigments to __bases__ "do the right thing"
> > has eluded me, basically because I can't seem to affect the mro.
> 
> The mro is an internal data structure of new-style classes, so redefining
> mro() doesn't change the values used.

Yeah, I noticed that eventually.

> Here is my (non-working version) that
> attempts to re-assign the class of an object, although it fails on a layout
> violation with Python 2.2.2.
> 
> def base_getter(cls):
>   return cls.__my_bases__
> 
> def base_setter(cls,bases):
>   if not bases:
>     bases = (object,)
>   metaclass = getattr(cls, '__metaclass__', type)
>   new_cls = metaclass(cls.__name__, bases, dict(cls.__dict__))
>   cls.__class__ = new_cls
> 
> class MetaBase(type):
>   __bases__ = property(base_getter,base_setter)
> 
>   def __new__(cls, name, bases, ns):
>     ns['__my_bases__'] = tuple(bases)
>     return super(MetaBase, cls).__new__(cls, name, bases, ns)
> 
> class Foo(object):
>   __metaclass__ = MetaBase
> class Baz(object): pass
> 
> Foo.__bases__ = Foo.__bases__ + (Baz,)

I don't think this has a hope of working does it?  This approach would
need to rebind "Foo" in the last line, no?

> Which results in:
>   TypeError: __class__ assignment: 'Foo' object layout differs from 'MetaBase'
> 
> I haven't looked into why this is being flagged as a layout error, though
> my first instinct is to say that the check is too conservative in this case.
> I'll think about it more and dig into the code.

I think the error message gives it away: in "cls.__class__ = new_cls",
cls.__class__ is MetaBase, new_cls is the new Foo.

I guess one way of doing this would be to reimplement mro resolution
and so on in Python, which would be annoying and inefficient.  Hmm.

Cheers,
M.

-- 
  [3] Modem speeds being what they are, large .avi files were
      generally downloaded to the shell server instead[4].
  [4] Where they were usually found by the technical staff, and
      burned to CD.                                   -- Carlfish, asr


From guido@python.org  Wed Oct 30 15:31:32 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 10:31:32 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 06:51:41 +0100."
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net> <m38z0gbju4.fsf@mira.informatik.hu-berlin.de> <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210301531.g9UFVW918144@odiug.zope.com>

> > What's the license on the BerkeleyDB code from Sleepycat?  
> 
> There are two licenses: One that they call the "open source license",
> see
> 
> http://www.sleepycat.com/license.net
> 
> There is also a commercial license.
> 
> > Can we legally distribute RPMs or other binaries containing it?  (I
> > thought there were some restrictions that make it not open source.)
> 
> It depends. This is the condition:
> 
> # Redistributions in any form must be accompanied by information on
> # how to obtain complete source code for the DB software and any
> # accompanying software that uses the DB software.  The source code
> # must either be included in the distribution or be available for no
> # more than the cost of distribution plus a nominal fee, and must be
> # freely redistributable under reasonable conditions.
> 
> So distributing Python itself should be no problem.

It's similar to GPL's "copyleft".  I think it's no different from what
we do with e.g. GNU readline, so I think it's okay.  Redistributors of
Python in binary form will have to beware though.  I wonder if we're
on thin ice with the RPMs (since we don't clarify any of this)?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Oct 30 15:46:47 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 30 Oct 2002 16:46:47 +0100
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>	<200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>	<m38z0gbju4.fsf@mira.informatik.hu-berlin.de>	<200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>  	<m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de> <200210301531.g9UFVW918144@odiug.zope.com>
Message-ID: <3DBFFEE7.6080304@lemburg.com>

Guido van Rossum wrote:
>>>What's the license on the BerkeleyDB code from Sleepycat?  
>>
>>There are two licenses: One that they call the "open source license",
>>see
>>
>>http://www.sleepycat.com/license.net
>>
>>There is also a commercial license.
>>
>>
>>>Can we legally distribute RPMs or other binaries containing it?  (I
>>>thought there were some restrictions that make it not open source.)
>>
>>It depends. This is the condition:
>>
>># Redistributions in any form must be accompanied by information on
>># how to obtain complete source code for the DB software and any
>># accompanying software that uses the DB software.  The source code
>># must either be included in the distribution or be available for no
>># more than the cost of distribution plus a nominal fee, and must be
>># freely redistributable under reasonable conditions.
>>
>>So distributing Python itself should be no problem.
> 
> 
> It's similar to GPL's "copyleft".  I think it's no different from what
> we do with e.g. GNU readline, so I think it's okay.  Redistributors of
> Python in binary form will have to beware though.  I wonder if we're
> on thin ice with the RPMs (since we don't clarify any of this)?

... but we're not including readline in the standard Python
distribution. Why would we want to include the database code
itself in Python ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From guido@python.org  Wed Oct 30 16:04:12 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 11:04:12 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 16:46:47 +0100."
 <3DBFFEE7.6080304@lemburg.com>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net> <m38z0gbju4.fsf@mira.informatik.hu-berlin.de> <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net> <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de> <200210301531.g9UFVW918144@odiug.zope.com>
 <3DBFFEE7.6080304@lemburg.com>
Message-ID: <200210301604.g9UG4CF18245@odiug.zope.com>

> > It's similar to GPL's "copyleft".  I think it's no different from what
> > we do with e.g. GNU readline, so I think it's okay.  Redistributors of
> > Python in binary form will have to beware though.  I wonder if we're
> > on thin ice with the RPMs (since we don't clarify any of this)?
> 
> ... but we're not including readline in the standard Python
> distribution. Why would we want to include the database code
> itself in Python ?

Not in the source distro, but it will end up in the RPMs and other
binary distros.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 30 16:44:36 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 11:44:36 -0500
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Your message of "Wed, 30 Oct 2002 14:36:20 GMT."
 <2md6psyowr.fsf@starship.python.net>
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com>
 <2md6psyowr.fsf@starship.python.net>
Message-ID: <200210301644.g9UGiaZ18410@odiug.zope.com>

> I guess one way of doing this would be to reimplement mro resolution
> and so on in Python, which would be annoying and inefficient.  Hmm.

Alas, yes.  You'd have to be able to write a class's tp_mro slot
(corresponding to the __mro__ attribute), but there's no way to do
that in Python, because it is too easy to cause mistakes.  E.g. all
C code that currently uses tp_mro assumes that it is a tuple whose
items are either types or classic classes.

The best thing to do would perhaps to make __mro__ assignable, but
with a check that ensures the above constraint.  I think I'd take a
patch for that.

I'd also take a patch for assignable __bases__.  Note that there are
constraints between __bases__ and __base__.

I'd also take a patch for assignable __name__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Oct 30 16:46:19 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 30 Oct 2002 17:46:19 +0100
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>	<200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>	<m38z0gbju4.fsf@mira.informatik.hu-berlin.de>	<200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>	<m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>	<200210301531.g9UFVW918144@odiug.zope.com>  	<3DBFFEE7.6080304@lemburg.com> <200210301604.g9UG4CF18245@odiug.zope.com>
Message-ID: <3DC00CDB.9060109@lemburg.com>

Guido van Rossum wrote:
>>>It's similar to GPL's "copyleft".  I think it's no different from what
>>>we do with e.g. GNU readline, so I think it's okay.  Redistributors of
>>>Python in binary form will have to beware though.  I wonder if we're
>>>on thin ice with the RPMs (since we don't clarify any of this)?
>>
>>... but we're not including readline in the standard Python
>>distribution. Why would we want to include the database code
>>itself in Python ?
> 
> Not in the source distro, but it will end up in the RPMs and other
> binary distros.

But that would violate the Sleepycat license; we are not shipping
readline or a module which statically links against readline
in the binary distros for the same reason.

As I understand, the BSD version shipped with Python so far
is using a standard BSD style license, so there's no problem
with it. However, Sleepycat added a copyleft kind of restriction
to the BSD license which makes this troublesome, esp. since the
restriction is not at all clear about what "use of the DB software"
really means. Their web-page on this is:

     http://www.sleepycat.com/licensing.html

Also, the Sleepycat license is not compatible with the Python
license in the sense that the Python license is far less
restrictive than the Sleepycat one. We'd have to make it very
clear that we have integrated third-party code into the
archive which falls under a much more restrictive license.
(FWIW, It would actually be wise to summarize all the different
licenses in the Python distro somewhere...)

So back to my original question: why go through all these
troubles ?

BTW, Sleepycat ships Berkley DB 4.1, wouldn't it be wise to
update the wrapper module to that version ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From barry@python.org  Wed Oct 30 17:33:28 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 30 Oct 2002 12:33:28 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <200210301531.g9UFVW918144@odiug.zope.com>
 <3DBFFEE7.6080304@lemburg.com>
 <200210301604.g9UG4CF18245@odiug.zope.com>
 <3DC00CDB.9060109@lemburg.com>
Message-ID: <15808.6120.271065.946000@gargle.gargle.HOWL>

>>>>> "MAL" == M  <mal@lemburg.com> writes:

    >> Not in the source distro, but it will end up in the RPMs and
    >> other binary distros.

    MAL> But that would violate the Sleepycat license; we are not
    MAL> shipping readline or a module which statically links against
    MAL> readline in the binary distros for the same reason.

Again, when we talked to Sleepycat a year and a half ago, they said it
would be ok to distribute the library with Python.  IIRC, even a
binary distro would be fine.  But I think a simple email to them would
clear the issue up.

    MAL> BTW, Sleepycat ships Berkley DB 4.1, wouldn't it be wise to
    MAL> update the wrapper module to that version ?

Because there are problems:

http://sourceforge.net/mailarchive/forum.php?thread_id=1206938&forum_id=4362

-Barry


From guido@python.org  Wed Oct 30 17:45:08 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 12:45:08 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "Wed, 30 Oct 2002 17:46:19 +0100."
 <3DC00CDB.9060109@lemburg.com>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net> <m38z0gbju4.fsf@mira.informatik.hu-berlin.de> <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net> <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de> <200210301531.g9UFVW918144@odiug.zope.com> <3DBFFEE7.6080304@lemburg.com> <200210301604.g9UG4CF18245@odiug.zope.com>
 <3DC00CDB.9060109@lemburg.com>
Message-ID: <200210301745.g9UHj8t18711@odiug.zope.com>

> >>>It's similar to GPL's "copyleft".  I think it's no different from what
> >>>we do with e.g. GNU readline, so I think it's okay.  Redistributors of
> >>>Python in binary form will have to beware though.  I wonder if we're
> >>>on thin ice with the RPMs (since we don't clarify any of this)?
> >>
> >>... but we're not including readline in the standard Python
> >>distribution. Why would we want to include the database code
> >>itself in Python ?
> > 
> > Not in the source distro, but it will end up in the RPMs and other
> > binary distros.
> 
> But that would violate the Sleepycat license; we are not shipping
> readline or a module which statically links against readline
> in the binary distros for the same reason.

It only violates the Sleepycat license if (a) we don't include
that license with the RPM or (b) we include closed-source code in that
RPM.  We're violating only (a) AFAIK and that's easily fixed.

> As I understand, the BSD version shipped with Python so far
> is using a standard BSD style license, so there's no problem
> with it. However, Sleepycat added a copyleft kind of restriction
> to the BSD license which makes this troublesome, esp. since the
> restriction is not at all clear about what "use of the DB software"
> really means. Their web-page on this is:
> 
>      http://www.sleepycat.com/licensing.html
> 
> Also, the Sleepycat license is not compatible with the Python
> license in the sense that the Python license is far less
> restrictive than the Sleepycat one. We'd have to make it very
> clear that we have integrated third-party code into the
> archive which falls under a much more restrictive license.
> (FWIW, It would actually be wise to summarize all the different
> licenses in the Python distro somewhere...)
> 
> So back to my original question: why go through all these
> troubles ?

We're not planning to incorporate Sleepycat code in the python
source distribution.  But if setup.py builds bsddb if it finds the
parts, it's hard to keep the Sleepycat binaries in the RPMs.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Wed Oct 30 18:17:26 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Oct 2002 18:17:26 +0000
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Guido van Rossum's message of "Wed, 30 Oct 2002 11:44:36 -0500"
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com> <2md6psyowr.fsf@starship.python.net> <200210301644.g9UGiaZ18410@odiug.zope.com>
Message-ID: <2my98faj0p.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > I guess one way of doing this would be to reimplement mro resolution
> > and so on in Python, which would be annoying and inefficient.  Hmm.
> 
> Alas, yes.  You'd have to be able to write a class's tp_mro slot
> (corresponding to the __mro__ attribute), but there's no way to do
> that in Python, because it is too easy to cause mistakes.  E.g. all
> C code that currently uses tp_mro assumes that it is a tuple whose
> items are either types or classic classes.
> 
> The best thing to do would perhaps to make __mro__ assignable, but
> with a check that ensures the above constraint.  I think I'd take a
> patch for that.

Shouldn't be too hard.

> I'd also take a patch for assignable __bases__.  Note that there are
> constraints between __bases__ and __base__.

Should assigning to __bases__ automatically tweak __mro__ and
__base__?  Guess so. What would assigning to __base__ do in isolation?
Perhaps that shouldn't be writeable.

> I'd also take a patch for assignable __name__.

This is practically a one-liner, isn't it?  Not hard, anyway.

And there was me wondering what I was going to do this evening.

Cheers,
M.

-- 
  Now this is what I don't get.  Nobody said absolutely anything
  bad about anything.  Yet it is always possible to just pull
  random flames out of ones ass.
         -- http://www.advogato.org/person/vicious/diary.html?start=60


From mal@lemburg.com  Wed Oct 30 18:19:57 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 30 Oct 2002 19:19:57 +0100
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>	<200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>	<m38z0gbju4.fsf@mira.informatik.hu-berlin.de>	<200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>	<m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>	<200210301531.g9UFVW918144@odiug.zope.com> <3DBFFEE7.6080304@lemburg.com>	<200210301604.g9UG4CF18245@odiug.zope.com>  	<3DC00CDB.9060109@lemburg.com> <200210301745.g9UHj8t18711@odiug.zope.com>
Message-ID: <3DC022CD.5010607@lemburg.com>

Guido van Rossum wrote:
>>>Not in the source distro, but it will end up in the RPMs and other
>>>binary distros.
>>
>>But that would violate the Sleepycat license; we are not shipping
>>readline or a module which statically links against readline
>>in the binary distros for the same reason.
> 
> It only violates the Sleepycat license if (a) we don't include
> that license with the RPM or (b) we include closed-source code in that
> RPM.  We're violating only (a) AFAIK and that's easily fixed.

Ok.

>>As I understand, the BSD version shipped with Python so far
>>is using a standard BSD style license, so there's no problem
>>with it. However, Sleepycat added a copyleft kind of restriction
>>to the BSD license which makes this troublesome, esp. since the
>>restriction is not at all clear about what "use of the DB software"
>>really means. Their web-page on this is:
>>
>>     http://www.sleepycat.com/licensing.html
>>
>>Also, the Sleepycat license is not compatible with the Python
>>license in the sense that the Python license is far less
>>restrictive than the Sleepycat one. We'd have to make it very
>>clear that we have integrated third-party code into the
>>archive which falls under a much more restrictive license.
>>(FWIW, It would actually be wise to summarize all the different
>>licenses in the Python distro somewhere...)
>>
>>So back to my original question: why go through all these
>>troubles ?
> 
> We're not planning to incorporate Sleepycat code in the python
> source distribution.  But if setup.py builds bsddb if it finds the
> parts, it's hard to keep the Sleepycat binaries in the RPMs.

It's not like it's impossible, though :-). Ideal would be to have
the Python module link against the shared lib of the Sleepycat DB
(if that's possible).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/



From barry@python.org  Wed Oct 30 18:22:24 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 30 Oct 2002 13:22:24 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <200210301531.g9UFVW918144@odiug.zope.com>
 <3DBFFEE7.6080304@lemburg.com>
 <200210301604.g9UG4CF18245@odiug.zope.com>
 <3DC00CDB.9060109@lemburg.com>
Message-ID: <15808.9056.873174.779158@gargle.gargle.HOWL>

[Apologies for any dupes: I'm trying to repost this with one CC header
to see if that fixes a problem Guido was seeing -BAW]

>>>>> "MAL" == M  <mal@lemburg.com> writes:

    >> Not in the source distro, but it will end up in the RPMs and
    >> other binary distros.

    MAL> But that would violate the Sleepycat license; we are not
    MAL> shipping readline or a module which statically links against
    MAL> readline in the binary distros for the same reason.

Again, when we talked to Sleepycat a year and a half ago, they said it
would be ok to distribute the library with Python.  IIRC, even a
binary distro would be fine.  But I think a simple email to them would
clear the issue up.

    MAL> BTW, Sleepycat ships Berkley DB 4.1, wouldn't it be wise to
    MAL> update the wrapper module to that version ?

Because there are problems:

http://sourceforge.net/mailarchive/forum.php?thread_id=1206938&forum_id=4362

-Barry


From martin@v.loewis.de  Wed Oct 30 19:31:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 20:31:41 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <15807.59887.992769.142566@gargle.gargle.HOWL>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <m3k7k0sbxl.fsf@mira.informatik.hu-berlin.de>
 <200210301345.g9UDjxb21402@pcp02138704pcs.reston01.va.comcast.net>
 <15807.59887.992769.142566@gargle.gargle.HOWL>
Message-ID: <m34rb3raea.fsf@mira.informatik.hu-berlin.de>

barry@python.org (Barry A. Warsaw) writes:

> I'm not sure we'd want to distribute either the source or binary
> though.  They're fairly large.  If we did, I'd recommend BerkeleyDB
> 4.0.14, not the latest version, because bsddb3 doesn't yet build on
> 4.1.24 because of some incompatible API changes, IIUC.

libdb33.dll is 470k, which might be acceptable. I agree that
distributing the source is pointless.

> Martin, what do you plan to do about the pybsddb test suite?  

I would ignore it, or put it in non-dist (in case this indicates the
end of the pybsddb project).

> Maybe we include none of that, but simply "import" the code into
> Python and re-sync whenever it's appropriate.

That is my proposal. Initially, there would be no user-visible change,
as it would operate in compatible mode. We could then incorporate more
documentation as we go along.

Regards,
Martin



From martin@v.loewis.de  Wed Oct 30 19:52:25 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 20:52:25 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <200210301531.g9UFVW918144@odiug.zope.com>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <200210300138.g9U1cse31882@pcp02138704pcs.reston01.va.comcast.net>
 <m38z0gbju4.fsf@mira.informatik.hu-berlin.de>
 <200210300512.g9U5CZK20380@pcp02138704pcs.reston01.va.comcast.net>
 <m3wuo0a2z6.fsf@mira.informatik.hu-berlin.de>
 <200210301531.g9UFVW918144@odiug.zope.com>
Message-ID: <m3znsvpuva.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> It's similar to GPL's "copyleft".  I think it's no different from what
> we do with e.g. GNU readline, so I think it's okay.  Redistributors of
> Python in binary form will have to beware though.  I wonder if we're
> on thin ice with the RPMs (since we don't clarify any of this)?

The RPMs should be fine: they include neither readline, nor
bsddb. Instead, those libraries must be present on the target system.

Regards,
Martin



From cpr@emsoftware.com  Wed Oct 30 20:15:55 2002
From: cpr@emsoftware.com (Chris Ryland)
Date: Wed, 30 Oct 2002 15:15:55 -0500
Subject: [Python-Dev] Fw: LAR Workshop Website Online,
Message-ID: <12d401c28051$392b23b0$6701a8c0@EM2>

Are you PythonLabs folks aware of this? It might be interesting &&||
relevant.
--
Cheers!
Chris Ryland
Em Software, Inc.
www.emsoftware.com
> -----Original Message-----
> From: Yahya H. Mirza [mailto:yahya_mirza@hotmail.com]
> Sent: Wednesday, October 30, 2002 9:08 AM
> To: Corin Day
> Cc: J=FCrg Gutknecht; davidi@borland.com; Ralph Johnson; George
Bosworth; Stan
> Lippman; Yukihiro Matsumoto; eliot@parcplace.com; David Simmons;
Gregory T.
> Sullivan; David Stutz; Dan Sugalski; ian.piumarta@inria.fr; David
Ungar;
> Shawn Woods; Misha Dmitriev; Jim Miller (COM+); Tony Williams; Dan Fay
> (RESEARCH); Mark Ryland; Andrew Palay; Mark Lewin; Don Syme; Greg
O'Shea;
> Pierre Sainoyant; Van Eden; Alan Borning; chambers@cs.washington.edu;
Marc
> E. Fiuczynski; David Notkin; zahorjan@cs.washington.edu;
> jonal@cs.washington.edu; csk@cs.washington.edu; Brad Merrill; Brian
Pepin;
> Brian Foote
> Subject: LAR Workshop Website Online,
>
>
> I would like to invite you all to register for LAR02:
>
> Our workshop website is now online and it can be accessed either
through the
> main OOPSLA Workshop page or directly via:
>
> http://hosting.msugs.ch/auroraborealis/Workshops/LAR02/LAR02Home.htm
> <http://hosting.msugs.ch/auroraborealis/Workshops/LAR02/LAR02Home.htm>
>
> If you would please take the time to Register even if you are
participating,
> and provide some feedback, I would greatly appreciate it.
>
> Our workshop starts at 8:45 AM this Monday November 4 at the OOPSLA
> Conference in the Seattle Convention Center in Rooms 613 - 614.
>
> Please feel free to invite whoever you think would be interested in
> participating.
>
> Finally I would like to thank Cori Day for doing a wonderful job on
the
> website.
>
> Sincerely,
>
> Yahya
>
> Yahya H. Mirza
> Aurora Borealis Software
> 8502 166th Ave. NE
> Redmond WA, 98052
> (425)-861-8147




From guido@python.org  Wed Oct 30 20:44:38 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 15:44:38 -0500
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Your message of "30 Oct 2002 18:17:26 GMT."
 <2my98faj0p.fsf@starship.python.net>
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com> <2md6psyowr.fsf@starship.python.net> <200210301644.g9UGiaZ18410@odiug.zope.com>
 <2my98faj0p.fsf@starship.python.net>
Message-ID: <200210302044.g9UKicx22801@odiug.zope.com>

> > The best thing to do would perhaps to make __mro__ assignable, but
> > with a check that ensures the above constraint.  I think I'd take a
> > patch for that.
> 
> Shouldn't be too hard.
> 
> > I'd also take a patch for assignable __bases__.  Note that there are
> > constraints between __bases__ and __base__.
> 
> Should assigning to __bases__ automatically tweak __mro__ and
> __base__?  Guess so.

Yes.  Note that changing __base__ should not be done lightly --
basically, the old and new base must be layout compatible, exactly
like for assignment to __class__.

> What would assigning to __base__ do in isolation?
> Perhaps that shouldn't be writeable.

Perhaps it could be writable when __bases__ is a 1-tuple.  But it's
fine if it's not writable.

> > I'd also take a patch for assignable __name__.
> 
> This is practically a one-liner, isn't it?  Not hard, anyway.

Probably.  Can't remember why I didn't do it earlier.

> And there was me wondering what I was going to do this evening.

I don't have that problem -- a Zope customer problem was waiting for
me today. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Oct 30 20:54:30 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 15:54:30 -0500
Subject: [Python-Dev] Fw: LAR Workshop Website Online,
In-Reply-To: Your message of "Wed, 30 Oct 2002 15:15:55 EST."
 <12d401c28051$392b23b0$6701a8c0@EM2>
References: <12d401c28051$392b23b0$6701a8c0@EM2>
Message-ID: <200210302054.g9UKsUn22895@odiug.zope.com>

> Are you PythonLabs folks aware of this? It might be interesting &&||
> relevant.

But I won't be able to make it. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Oct 30 14:19:11 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 30 Oct 2002 08:19:11 -0600
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15807.59999.240430.849098@montanaro.dyndns.org>

    Martin> I have changes in my sandbox to incorporate bsddb3 into Python
    Martin> proper....

    Martin> Both test_bsddb and test_anydbm continue to work unmodified.

    Martin> Is it ok to commit these changes?

I say go for it.  Better now, when it's easily backed out than later.  I use
bsddb pretty frequently, and run Python CVS on my development machine, so
I'll be able to wring it out a bit.

Skip


From martin@v.loewis.de  Wed Oct 30 21:43:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Oct 2002 22:43:54 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <15807.59999.240430.849098@montanaro.dyndns.org>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
Message-ID: <m31y67ppph.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I say go for it.  Better now, when it's easily backed out than later.  I use
> bsddb pretty frequently, and run Python CVS on my development machine, so
> I'll be able to wring it out a bit.

Thanks for the encouragement. I really need PythonLab's permission
here, though, since it will affect what the Windows distribution
contains.

If the political issues can be settled, working out the build process
for Windows should be easy.

Regards,
Martin



From guido@python.org  Wed Oct 30 22:01:17 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 30 Oct 2002 17:01:17 -0500
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: Your message of "30 Oct 2002 22:43:54 +0100."
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de> <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210302201.g9UM1Hm27151@odiug.zope.com>

> Thanks for the encouragement. I really need PythonLab's permission
> here, though, since it will affect what the Windows distribution
> contains.
> 
> If the political issues can be settled, working out the build process
> for Windows should be easy.

Barry will send an email to Sleepycat to ask exactly what the
restrictions on binary redistribution are.

There's also the issue of upgrading.  Suppose someone has created a
database using bsddb in Python 2.2.  When they upgrade to Python 2.3,
they'll get an exception when they open the database, because
Sleepycat DB 4.x can't open BSD DB 1.85 databases.  What do we tell
them to do?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@python.org  Wed Oct 30 22:07:35 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 30 Oct 2002 17:07:35 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15808.22567.772352.109103@gargle.gargle.HOWL>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    MvL> If the political issues can be settled, working out the build
    MvL> process for Windows should be easy.

I'm going to contact Sleepycat and start a dialog with them on the
licensing and binary redist issues.  I'll Cc Guido so we'll get a
definitive answer on that, hopefully soon.

-Barry


From barry@python.org  Wed Oct 30 22:10:58 2002
From: barry@python.org (Barry A. Warsaw)
Date: Wed, 30 Oct 2002 17:10:58 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
 <200210302201.g9UM1Hm27151@odiug.zope.com>
Message-ID: <15808.22770.456261.728316@gargle.gargle.HOWL>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> There's also the issue of upgrading.  Suppose someone has
    GvR> created a database using bsddb in Python 2.2.  When they
    GvR> upgrade to Python 2.3, they'll get an exception when they
    GvR> open the database, because Sleepycat DB 4.x can't open BSD DB
    GvR> 1.85 databases.  What do we tell them to do?

Run this script:

http://www.sleepycat.com/docs/utility/db_upgrade.html

-Barry


From skip@pobox.com  Wed Oct 30 22:30:57 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 30 Oct 2002 16:30:57 -0600
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <200210302201.g9UM1Hm27151@odiug.zope.com>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
 <200210302201.g9UM1Hm27151@odiug.zope.com>
Message-ID: <15808.23969.429522.883750@montanaro.dyndns.org>

    Guido> When they upgrade to Python 2.3, they'll get an exception when
    Guido> they open the database, because Sleepycat DB 4.x can't open BSD
    Guido> DB 1.85 databases.  What do we tell them to do?

The same thing you've had to tell them in the past.  Use the tools Sleepycat
provides (db_dump, db_load, etc) to update databases from old file formats
to new ones.

Skip



From martin@v.loewis.de  Thu Oct 31 06:38:35 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 31 Oct 2002 07:38:35 +0100
Subject: [Python-Dev] Incorporating bsddb3
In-Reply-To: <200210302201.g9UM1Hm27151@odiug.zope.com>
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
 <200210302201.g9UM1Hm27151@odiug.zope.com>
Message-ID: <m38z0fcduc.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> There's also the issue of upgrading.  Suppose someone has created a
> database using bsddb in Python 2.2.  When they upgrade to Python 2.3,
> they'll get an exception when they open the database, because
> Sleepycat DB 4.x can't open BSD DB 1.85 databases.  What do we tell
> them to do?

Whether this happens somewhat depends on the system. On Linux, it
won't happen, since people likely have been using Sleepycat DB,
anyway, through db_185.h.

If they really use DB 1.85 (i.e. on Windows), they need to use
db_dump/db_load.

Regards,
Martin



From barry@python.org  Thu Oct 31 06:51:45 2002
From: barry@python.org (Barry A. Warsaw)
Date: Thu, 31 Oct 2002 01:51:45 -0500
Subject: [Python-Dev] Incorporating bsddb3
References: <m3iszksrym.fsf@mira.informatik.hu-berlin.de>
 <15807.59999.240430.849098@montanaro.dyndns.org>
 <m31y67ppph.fsf@mira.informatik.hu-berlin.de>
 <200210302201.g9UM1Hm27151@odiug.zope.com>
 <m38z0fcduc.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15808.54017.738989.76521@gargle.gargle.HOWL>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    >> There's also the issue of upgrading.  Suppose someone has
    >> created a database using bsddb in Python 2.2.  When they
    >> upgrade to Python 2.3, they'll get an exception when they open
    >> the database, because Sleepycat DB 4.x can't open BSD DB 1.85
    >> databases.  What do we tell them to do?

    MvL> Whether this happens somewhat depends on the system. On
    MvL> Linux, it won't happen, since people likely have been using
    MvL> Sleepycat DB, anyway, through db_185.h.

Also note that just because they used the 1.85 version of the API
doesn't necessarily mean they've got a 1.85 version of the database
file format.  They could be using a newer version of the library in
1.85-compat mode.

-Barry


From mwh@python.net  Thu Oct 31 10:34:57 2002
From: mwh@python.net (Michael Hudson)
Date: 31 Oct 2002 10:34:57 +0000
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Guido van Rossum's message of "Wed, 30 Oct 2002 15:44:38 -0500"
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com> <2md6psyowr.fsf@starship.python.net> <200210301644.g9UGiaZ18410@odiug.zope.com> <2my98faj0p.fsf@starship.python.net> <200210302044.g9UKicx22801@odiug.zope.com>
Message-ID: <2mznsu3nhq.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > > The best thing to do would perhaps to make __mro__ assignable, but
> > > with a check that ensures the above constraint.  I think I'd take a
> > > patch for that.
> > 
> > Shouldn't be too hard.
> > 
> > > I'd also take a patch for assignable __bases__.  Note that there are
> > > constraints between __bases__ and __base__.
> > 
> > Should assigning to __bases__ automatically tweak __mro__ and
> > __base__?  Guess so.
> 
> Yes.  Note that changing __base__ should not be done lightly --
> basically, the old and new base must be layout compatible, exactly
> like for assignment to __class__.

OK.  I can crib code from type_set_class, I guess.  Or one could just
allow assignment to __bases__ when __base__ doesn't change?  __base__
is object for the majority of new-style classes, isn't it?

Brr.  There's a lot I don't know about post 2.2 typeobject.c.

> > What would assigning to __base__ do in isolation?
> > Perhaps that shouldn't be writeable.
> 
> Perhaps it could be writable when __bases__ is a 1-tuple.

Don't see the point of that.

> But it's fine if it's not writable.

Easier :)

> > > I'd also take a patch for assignable __name__.
> > 
> > This is practically a one-liner, isn't it?  Not hard, anyway.
> 
> Probably.  Can't remember why I didn't do it earlier.

It's a bit more complicated than that.

What's the deal wrt. dots in tp_name?  Is there any way for a user
defined class to end up called "something.somthing_else"?

Oh, and while we're at it, here's a bogosity:

>>> class C(object):
...  pass
... 
>>> C.__module__ 
'__main__'
>>> C.__module__ = 1
>>> C.__module__ 
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: __module__

caused by lax testing in type_set_module.

> > And there was me wondering what I was going to do this evening.
> 
> I don't have that problem -- a Zope customer problem was waiting for
> me today. :-(

Well, I didn't get it finished either.  Fiddly, this stuff.  Maybe by
tomorrow.

Cheers,
M.

-- 
  The Internet is full.  Go away.
                      -- http://www.disobey.com/devilshat/ds011101.htm


From hbl@st-andrews.ac.uk  Thu Oct 31 16:18:36 2002
From: hbl@st-andrews.ac.uk (Hamish Lawson)
Date: Thu, 31 Oct 2002 16:18:36 +0000
Subject: [Python-Dev] Possible ideas from Strongtalk (static typing for Smalltalk)?
Message-ID: <5.1.1.6.0.20021031160812.03bb5390@spey.st-andrews.ac.uk>

I recently came across an announcement about the the Strongtalk system, 
which contains the first fully developed strong, static type system for 
Smalltalk. I wondered whether there might be useful ideas there for those 
looking into static typing for Python.

http://www.cs.ucsb.edu/projects/strongtalk/pages/index.html


Hamish Lawson



From guido@python.org  Thu Oct 31 18:58:00 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 31 Oct 2002 13:58:00 -0500
Subject: [Python-Dev] metaclass insanity
In-Reply-To: Your message of "31 Oct 2002 10:34:57 GMT."
 <2mznsu3nhq.fsf@starship.python.net>
References: <Pine.LNX.4.44.0210300901560.15262-100000@penguin.theopalgroup.com> <2md6psyowr.fsf@starship.python.net> <200210301644.g9UGiaZ18410@odiug.zope.com> <2my98faj0p.fsf@starship.python.net> <200210302044.g9UKicx22801@odiug.zope.com>
 <2mznsu3nhq.fsf@starship.python.net>
Message-ID: <200210311858.g9VIw0509968@odiug.zope.com>

> > > Should assigning to __bases__ automatically tweak __mro__ and
> > > __base__?  Guess so.
> > 
> > Yes.  Note that changing __base__ should not be done lightly --
> > basically, the old and new base must be layout compatible, exactly
> > like for assignment to __class__.
> 
> OK.  I can crib code from type_set_class, I guess.  Or one could just
> allow assignment to __bases__ when __base__ doesn't change?  __base__
> is object for the majority of new-style classes, isn't it?

But if you derive from a builtin type (e.g. list or dict), __base__
will be that.

> Brr.  There's a lot I don't know about post 2.2 typeobject.c.

Me too. :-)

> > > What would assigning to __base__ do in isolation?
> > > Perhaps that shouldn't be writeable.
> > 
> > Perhaps it could be writable when __bases__ is a 1-tuple.
> 
> Don't see the point of that.
> 
> > But it's fine if it's not writable.
> 
> Easier :)

Agreed.

> > > > I'd also take a patch for assignable __name__.
> > > 
> > > This is practically a one-liner, isn't it?  Not hard, anyway.
> > 
> > Probably.  Can't remember why I didn't do it earlier.
> 
> It's a bit more complicated than that.
> 
> What's the deal wrt. dots in tp_name?  Is there any way for a user
> defined class to end up called "something.somthing_else"?

I hope not.  The dots are for extensions living inside a package;
everything before the last dot ends up as __module__.

> Oh, and while we're at it, here's a bogosity:
> 
> >>> class C(object):
> ...  pass
> ... 
> >>> C.__module__ 
> '__main__'
> >>> C.__module__ = 1
> >>> C.__module__ 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AttributeError: __module__
> 
> caused by lax testing in type_set_module.

Oops.  Can you fix it?  Or are there complications?  Seems to be
broken in 2.2 too.

> > > And there was me wondering what I was going to do this evening.
> > 
> > I don't have that problem -- a Zope customer problem was waiting for
> > me today. :-(
> 
> Well, I didn't get it finished either.  Fiddly, this stuff.  Maybe by
> tomorrow.

Great!

I'll be offline Friday through Monday -- going to a weekend conference.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From niemeyer@conectiva.com  Thu Oct 31 23:40:46 2002
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Thu, 31 Oct 2002 20:40:46 -0300
Subject: [Python-Dev] Becoming a python contributor
Message-ID: <20021031204046.A32673@ibook.distro.conectiva>

Hello!

After a long time of inactivity, I hope to be back to some python
hacking. Before getting into any coding, I'd like to discuss with
you some conceptual and boring stuff I have in my mind.

In the past I have given small contributions to the python standard
distribution. Unfortunately (for myself), I slowed down until I stopped
contributing, even though I have a great affect by the interpreter. Now
I realize that one of the reasons I've stopped contributing is because
there's a large inertia in getting stuff reviewed. The reason why this
is happening seems more aparent now that I was off for a while: there's
a small core of very busy developers working on core/essential/hard
stuff *and* in code reviewing as well.

At the same time, I've seen Guido and others bothered a few times
because of the lack of man power. So the question is: how do I, a
developer which feels capable of helping in python's development, can
get some of the tasks which take your time out of your hands? Or even,
how is it possible to improve some part of the development process by
giving people like me some instructions? Also, isn't it easy to point
out what's wrong in a commit from someone who is following the
development process for a while than taking the time to review its code
in the sourceforge patch system?

My feeling is that the Python development is currently overly
centralized, and that you might be suffering from that now, by being
unable to handover some of your tasks to someone else. I feel that
everytime I send a patch, besides being contributing, I'm also
overloading you with more stuff to review and comment and etc. Perhaps
the fallback costs for some wrong commit is too high now (did I heard
someone mentioning subversion?)?!

Could someone please discuss that issues with me, and perhaps just kick
me away saying that I'm crazy? :-)

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From niemeyer@conectiva.com  Thu Oct 31 23:47:32 2002
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Thu, 31 Oct 2002 20:47:32 -0300
Subject: [Python-Dev] Getting python-bz2 into 2.3
Message-ID: <20021031204732.B32673@ibook.distro.conectiva>

Now that I'm relieved by sharing with you my feelings, ;-)
what's the best path to get python-bz2 module into Python 2.3?
Do you think it's something which should be in the core, or it'd
be better to keep it as an external module?

The code is currently maintained at http://python-bz2.sf.net, if
someone wants to have a look at it.

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]