From nicko at nicko.org  Fri Feb  1 16:26:10 2008
From: nicko at nicko.org (Nicko van Someren)
Date: Fri, 1 Feb 2008 15:26:10 +0000
Subject: [Python-3000] Set literal
In-Reply-To: <47A10593.7080805@canterbury.ac.nz>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
Message-ID: <C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>

On 30 Jan 2008, at 23:17, Greg Ewing wrote:

> Nicko van Someren wrote:
>> Personally I'd like set comprehensions to give me mutable  sets,  
>> and so I feel set literals should do the same.
>
> Do you really want set comprehensions at all, given that
> set(genexp) exists?

Do we really want set literals at all, given that set(...) exists?

If we are going to have one then, it seems to make sense to have  
both.  If we are going to have both, I would rather that they generate  
the same type of set.  I appreciate that set literals can get  
optimised at compile time whereas comprehensions can not be optimised  
in the same way, and in the case of expressions in round brackets we  
get different types when we enclose lists vs. genexps that way, but I  
would rather we kept these things as consistent as possible.

	Nicko


From python at rcn.com  Sat Feb  2 00:56:38 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri,  1 Feb 2008 18:56:38 -0500 (EST)
Subject: [Python-3000] Comparing to None
Message-ID: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net>

It was very convenient in Py2.x to have None comparable to other types and always rank less than any other type.

I would like to restore that ability.

One example, it helps with heaps to be able to turn an entry into None and have it sift to the top where the null entries can be popped-off.


Raymond

From greg.ewing at canterbury.ac.nz  Sat Feb  2 01:07:26 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 02 Feb 2008 13:07:26 +1300
Subject: [Python-3000] Set literal
In-Reply-To: <C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
Message-ID: <47A3B43E.1060704@canterbury.ac.nz>

Nicko van Someren wrote:

> Do we really want set literals at all, given that set(...) exists?
> 
> If we are going to have one then, it seems to make sense to have  both.

Not necessarily. We don't have dict comprehensions, and
I don't remember there being many complaints about that.

Also, if generator expressions had come first, we
would probably never have gotten list comprehensions.

--
Greg

From guido at python.org  Sat Feb  2 01:44:17 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 1 Feb 2008 16:44:17 -0800
Subject: [Python-3000] Comparing to None
In-Reply-To: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net>
References: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802011644p3631a59bv6f43729a7d292a44@mail.gmail.com>

You should have been there when this was decided about two years ago.

I really don't want to be tweaking the language spec continuously
between now and the planned release date in ~August.

On Feb 1, 2008 3:56 PM, Raymond Hettinger <python at rcn.com> wrote:
> It was very convenient in Py2.x to have None comparable to other types and always rank less than any other type.
>
> I would like to restore that ability.
>
> One example, it helps with heaps to be able to turn an entry into None and have it sift to the top where the null entries can be popped-off.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Feb  2 01:45:33 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 1 Feb 2008 16:45:33 -0800
Subject: [Python-3000] Set literal
In-Reply-To: <47A3B43E.1060704@canterbury.ac.nz>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
Message-ID: <ca471dc20802011645t19e89272ld8aa906499bf1cbb@mail.gmail.com>

On Feb 1, 2008 4:07 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Not necessarily. We don't have dict comprehensions, and

We don't?

bash-3.2$ ./python.exe
Python 3.0a2+ (py3k:60207, Jan 22 2008, 16:58:59)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> {x:x**2 for x in range(10)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
>>>

Where have you been? :-)


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Feb  2 02:18:23 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri,  1 Feb 2008 20:18:23 -0500 (EST)
Subject: [Python-3000] Comparing to None
Message-ID: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>

> You should have been there when this was decided about two years ago.

IIRC, the decision was a general one about cross-type 
comparisons not being turned-on the default.  I do not 
recall a specific discussion about None.

Also, the list at the time was flooded with propositions
ranging from the reasonable to the insane. It was not always
possible to know what to respond to or the implications of each
choice. I sure wasn't aware that those conversations were to be
immediately frozen in stone. I had thought one of the purposes 
of the Py3.0 was so that we could download it and explore the 
implications of all of these choices.  I've done so and bumped
into the None comparability issue several times. If you no 
longer want feedback, that's fine. I can bring my 
experimentation with the 3.0 alpha to a close.

Raymond

From guido at python.org  Sat Feb  2 02:44:38 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 1 Feb 2008 17:44:38 -0800
Subject: [Python-3000] Comparing to None
In-Reply-To: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802011744s586f62a5o2ceec6a2662a617b@mail.gmail.com>

On Feb 1, 2008 5:18 PM, Raymond Hettinger <python at rcn.com> wrote:
> > You should have been there when this was decided about two years ago.
>
> IIRC, the decision was a general one about cross-type
> comparisons not being turned-on the default.  I do not
> recall a specific discussion about None.

I vaguely recall it was discussed but not considered an important
enough use case.

> Also, the list at the time was flooded with propositions
> ranging from the reasonable to the insane. It was not always
> possible to know what to respond to or the implications of each
> choice. I sure wasn't aware that those conversations were to be
> immediately frozen in stone.

Not immediately. But by now we've had two alpha release already. Some
stuff *does* change between alphas. But in general we need to stem the
flow of change proposals.

> I had thought one of the purposes
> of the Py3.0 was so that we could download it and explore the
> implications of all of these choices.  I've done so and bumped
> into the None comparability issue several times.

I think you need to provide a lot more motivation than a 5-line email
to change something this fundamental at this point.

> If you no
> longer want feedback, that's fine. I can bring my
> experimentation with the 3.0 alpha to a close.

To be brutally honest, your recent attempts to open up various closed
issues have wasted a lot of cycles everywhere. I'd rather focus on
moving forward with things like UserDict.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From stephen at xemacs.org  Sat Feb  2 03:16:16 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 02 Feb 2008 11:16:16 +0900
Subject: [Python-3000] Set literal
In-Reply-To: <47A3B43E.1060704@canterbury.ac.nz>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
Message-ID: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > Also, if generator expressions had come first, we
 > would probably never have gotten list comprehensions.

I don't understand.  A list comprehension is for when you know you're
going to want to work with the whole list.  Similarly (even more so)
for dictionary comprehensions.  List comprehension can be defined in
terms of a genexp, of course, but ISTM it's a worthwhile addition to
the language in its own right.


From talin at acm.org  Sat Feb  2 03:04:18 2008
From: talin at acm.org (Talin)
Date: Fri, 01 Feb 2008 18:04:18 -0800
Subject: [Python-3000] Comparing to None
In-Reply-To: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
Message-ID: <47A3CFA2.4000704@acm.org>

I would like to present an argument both for and against the proposal.

Against: I've never personally needed this capability.

For: In many languages, the 'null' value has a special polymorphic 
behavior in that it is considered a subtype of all reference types. Thus 
in C, you can assign NULL to a Foo * or a Bar * even though Foo and Bar 
aren't related.

So making NULL special seems in line with the general trend.

Raymond Hettinger wrote:
>> You should have been there when this was decided about two years ago.
> 
> IIRC, the decision was a general one about cross-type 
> comparisons not being turned-on the default.  I do not 
> recall a specific discussion about None.
> 
> Also, the list at the time was flooded with propositions
> ranging from the reasonable to the insane. It was not always
> possible to know what to respond to or the implications of each
> choice. I sure wasn't aware that those conversations were to be
> immediately frozen in stone. I had thought one of the purposes 
> of the Py3.0 was so that we could download it and explore the 
> implications of all of these choices.  I've done so and bumped
> into the None comparability issue several times. If you no 
> longer want feedback, that's fine. I can bring my 
> experimentation with the 3.0 alpha to a close.
> 
> Raymond
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org
> 

From guido at python.org  Sat Feb  2 06:21:34 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 1 Feb 2008 21:21:34 -0800
Subject: [Python-3000] Comparing to None
In-Reply-To: <47A3CFA2.4000704@acm.org>
References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
	<47A3CFA2.4000704@acm.org>
Message-ID: <ca471dc20802012121v6d0dcacdl46aa540bf4d27d77@mail.gmail.com>

On Feb 1, 2008 6:04 PM, Talin <talin at acm.org> wrote:
> I would like to present an argument both for and against the proposal.
>
> Against: I've never personally needed this capability.
>
> For: In many languages, the 'null' value has a special polymorphic
> behavior in that it is considered a subtype of all reference types. Thus
> in C, you can assign NULL to a Foo * or a Bar * even though Foo and Bar
> aren't related.
>
> So making NULL special seems in line with the general trend.

Hardly. In all languages I know of besides Python, doing *anything*
with such a NULL operand apart from passing it around or doing a
pointer equality comparison on it immediately causes a segfault or a
NullPointerException. In Python, you can already assign None to any
variable and do a pointer comparison ('is'); you can even do a value
equality ('==') without being punished, which is already better than
most languages.

Raymond is asking for *ordering* ('<', '<=' etc.) to be well-defined
when one of the arguments is None. Python 2 indeed did that, because
in Python 2 *all* values are orderable (well, *almost* all --
comparing str to unicode can sometimes raise an exception, complex
numbers can't be ordered, and there are a few other odd exceptions).
Python 3 does away with comparisons between heterogeneous types,
unless at least one of the types take special measures to be
comparable to the other -- this makes much more sense than the
arbitrary decisions in Python 2 about the relative ordering of e.g. ()
and [], or 0 and "". (Ironically, Raymond's pet type, set(), cannot be
compared to None even in Python 2.)

As long as you're comparing real numbers, negative infinity can
fulfill the role that Raymond is asking for. I suspect that for all
collection types, the "empty" value of that type is always naturally
the smallest value. If you really want to sort lists of completely
heterogeneous types, you can always use (tag, value) pairs where the
tags are small ints or strings representing type classifications (I
guess the type name would do in most cases).

I expect that, given the general trend that different types are not
comparable, making an exception for None is more likely a source of
errors than a source of joy.

--Guido

> Raymond Hettinger wrote:
> >> You should have been there when this was decided about two years ago.
> >
> > IIRC, the decision was a general one about cross-type
> > comparisons not being turned-on the default.  I do not
> > recall a specific discussion about None.
> >
> > Also, the list at the time was flooded with propositions
> > ranging from the reasonable to the insane. It was not always
> > possible to know what to respond to or the implications of each
> > choice. I sure wasn't aware that those conversations were to be
> > immediately frozen in stone. I had thought one of the purposes
> > of the Py3.0 was so that we could download it and explore the
> > implications of all of these choices.  I've done so and bumped
> > into the None comparability issue several times. If you no
> > longer want feedback, that's fine. I can bring my
> > experimentation with the 3.0 alpha to a close.
> >
> > Raymond
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org
> >
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Feb  2 06:29:05 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 1 Feb 2008 21:29:05 -0800
Subject: [Python-3000] Comparing to None
References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net>
	<47A3CFA2.4000704@acm.org>
	<ca471dc20802012121v6d0dcacdl46aa540bf4d27d77@mail.gmail.com>
Message-ID: <004201c8655c$87d6d860$6800a8c0@RaymondLaptop1>

[GvR]
> I expect that, given the general trend that different types are not
> comparable, making an exception for None is more likely a source of
> errors than a source of joy.

That is reasonable.


Raymond

From guido at python.org  Sat Feb  2 12:48:57 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 2 Feb 2008 03:48:57 -0800
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <994938.11662.qm@web31111.mail.mud.yahoo.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
Message-ID: <ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>

This should be brought up on the python-3000 list; I'm moving it there
using a Bcc to python-ideas.

To some extent it is up to the vendors who distribute binaries -- they
decide what to call it.

Perhaps we should only install "python3.0" and not "python". That is a
valid choice already and always has been (python2.1, python2.2, etc.
are always installed by default, "python" is just a convenient alias).

I think that worries about Python becoming the laughingstock of the
language world are highly exaggerated. The post you refer to sounds to
me like the typical cynical one-liner from someone who doesn't really
care, not about a position of someone who is influential in the world
of language users.

--Guido

PS. Java is the VB of this decare.

On Feb 1, 2008 10:41 PM, Ralf W. Grosse-Kunstleve <rwgk at yahoo.com> wrote:
> This is regarding the transition to Python 3, and the confusion caused
> by using the same executable name for two incompatible interpreters.
> The consequences of this struck me today when I saw this message
> appear on an important bulletin board for the field I'm working in:
>
> http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1562
>
> This posting is from a heavy-weight in the field (lookup his name with
> Google or better The Web of Science). Basically he's suggesting to forget
> about Python because it is an unreliable environment.
>
> There is also this reply calling the original posting "alarmist":
>
> http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1464
>
> And this one making fun of the whole thing:
>
> http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1675
>
> I interpret these messages as early signs of an imminent decade of confusion.
> Therefore my plea:
>
> PLEASE GIVE THE PYTHON 3 EXECUTABLE A DIFFERENT NAME AND THE SCRIPTS
> A DIFFERENT EXTENSION.
>
> It will be a terrible political setback otherwise. Many people don't
> *want* to understand even if they could. Mixing up two incompatible
> interpreters under one name will give them plenty of ammunition for
> cheap jokes and for defending their not-invented-here-or-before-I-was-16
> attitude.
>
> Ralf
>
> P.S.: My wife is from Hungary. It seems all men in Hungary have to
> have one of exactly three names: Andras, Laszlo, or Balazs. I never
> know whom she's talking about...
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Sat Feb  2 14:42:44 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 02 Feb 2008 14:42:44 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
Message-ID: <fo1s0g$emo$1@ger.gmane.org>

Guido van Rossum schrieb:
> This should be brought up on the python-3000 list; I'm moving it there
> using a Bcc to python-ideas.
> 
> To some extent it is up to the vendors who distribute binaries -- they
> decide what to call it.
> 
> Perhaps we should only install "python3.0" and not "python". That is a
> valid choice already and always has been (python2.1, python2.2, etc.
> are always installed by default, "python" is just a convenient alias).
> 
> I think that worries about Python becoming the laughingstock of the
> language world are highly exaggerated. The post you refer to sounds to
> me like the typical cynical one-liner from someone who doesn't really
> care, not about a position of someone who is influential in the world
> of language users.

I would prefer a "python3" alias analogous to the current "python" alias,
at least as long as Python 2.x is still in wide use.

Having to type "python3.0" explicitly strikes me as unnecessarily verbose.

As for module file name extensions, what speaks against allowing (not
mandating) "py3" for Python 3.x modules?

 > PS. Java is the VB of this decare.

Had a close encounter with Java recently? :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From skip at pobox.com  Sat Feb  2 15:55:10 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 2 Feb 2008 08:55:10 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <fo1s0g$emo$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
Message-ID: <18340.33870.160673.135647@montanaro-dyndns-org.local>


    Georg> I would prefer a "python3" alias analogous to the current
    Georg> "python" alias, at least as long as Python 2.x is still in wide
    Georg> use.

So, "make install" would install a python3.0 executable and make python3 a
symlink.  Makes sense to me. "make bdfl-install" could also create a symlink
to python3 called "python". ;-)

Anybody who installed multiple versions of Python on their computer today -
at least in the Unixoid world - already has to know about "make altinstall"
so they don't accidentally obliterate the python symlink.  I don't
understand what all the fuss is.

Skip


From guido at python.org  Sat Feb  2 17:08:46 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 2 Feb 2008 08:08:46 -0800
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <18340.33870.160673.135647@montanaro-dyndns-org.local>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
	<18340.33870.160673.135647@montanaro-dyndns-org.local>
Message-ID: <ca471dc20802020808s69222f2av33c0010930244f04@mail.gmail.com>

On Feb 2, 2008 6:55 AM,  <skip at pobox.com> wrote:
>
>     Georg> I would prefer a "python3" alias analogous to the current
>     Georg> "python" alias, at least as long as Python 2.x is still in wide
>     Georg> use.
>
> So, "make install" would install a python3.0 executable and make python3 a
> symlink.  Makes sense to me. "make bdfl-install" could also create a symlink
> to python3 called "python". ;-)
>
> Anybody who installed multiple versions of Python on their computer today -
> at least in the Unixoid world - already has to know about "make altinstall"
> so they don't accidentally obliterate the python symlink.  I don't
> understand what all the fuss is.

Me neither. If you don't want to type "python3.0" why not create a shell alias?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at cheimes.de  Sat Feb  2 17:31:41 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 02 Feb 2008 17:31:41 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <fo1s0g$emo$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
Message-ID: <fo25te$ch8$1@ger.gmane.org>

Georg Brandl wrote:
> I would prefer a "python3" alias analogous to the current "python" alias,
> at least as long as Python 2.x is still in wide use.
> 
> Having to type "python3.0" explicitly strikes me as unnecessarily verbose.

+1

> As for module file name extensions, what speaks against allowing (not
> mandating) "py3" for Python 3.x modules?

File extensions are only important on Windows. Or does anybody know a
system which starts scripts based on the extension like Windows does?

I see two ways to solve the program for Windows:

*  register additional file extensions for each Python version
   like .py26, .pyw26 (for pythonw.exe), .py30 etc.

* Write a wrapper for Windows which inspects the first line of
  a script, searches for "#! ... python2.6" or "#!python26" and
  starts the right interpreter.

Christian


From fumanchu at aminus.org  Sat Feb  2 18:00:31 2008
From: fumanchu at aminus.org (Robert Brewer)
Date: Sat, 2 Feb 2008 09:00:31 -0800
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	greatidea -- let's do more of those!
In-Reply-To: <ca471dc20802020808s69222f2av33c0010930244f04@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com><ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com><fo1s0g$emo$1@ger.gmane.org><18340.33870.160673.135647@montanaro-dyndns-org.local>
	<ca471dc20802020808s69222f2av33c0010930244f04@mail.gmail.com>
Message-ID: <F1962646D3B64642B7C9A06068EE1E6401FC2D4C@ex10.hostedexchange.local>

Guido van Rossum wrote:
> On Feb 2, 2008 6:55 AM,  <skip at pobox.com> wrote:
> >
> >     Georg> I would prefer a "python3" alias analogous to the current
> >     Georg> "python" alias, at least as long as Python 2.x is still
in
> >     Georg> wide use.
> >
> > So, "make install" would install a python3.0 executable and make
> > python3 a symlink.  Makes sense to me. "make bdfl-install" could
> > also create a symlink to python3 called "python". ;-)
> >
> > Anybody who installed multiple versions of Python on their computer
> > today - at least in the Unixoid world - already has to know about
> > "make altinstall" so they don't accidentally obliterate the python
> > symlink.  I don't understand what all the fuss is.
> 
> Me neither. If you don't want to type "python3.0" why not create a
> shell alias?

We just went through this with CherryPy 3, and I wish we had used
"cherrypy3" as the package name. Granted, that's a library and not an
executable, so the biggest benefit (fewer import conflicts with a new
name) doesn't really apply to Python. But there are other benefits to
"python3". You can read code examples and work with users much more
easily, saving a lot of emails in the cycle: "what version are you
using?"; "dunno--how do I find out?". We're looking at several years of
people mistakenly trying to run 2.x code on 3.x and vice-versa. That
doesn't vanish completely; there will still be minor releases, but those
are supposed to be more backward-compatible.

I'd like to hear from e.g. the sqlite3 folks about similar experiences.


Robert Brewer
fumanchu at aminus.org


From g.brandl at gmx.net  Sat Feb  2 18:52:11 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 02 Feb 2008 18:52:11 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <fo25te$ch8$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>
	<fo25te$ch8$1@ger.gmane.org>
Message-ID: <fo2ak6$qet$1@ger.gmane.org>

Christian Heimes schrieb:
> Georg Brandl wrote:
>> I would prefer a "python3" alias analogous to the current "python" alias,
>> at least as long as Python 2.x is still in wide use.
>> 
>> Having to type "python3.0" explicitly strikes me as unnecessarily verbose.
> 
> +1
> 
>> As for module file name extensions, what speaks against allowing (not
>> mandating) "py3" for Python 3.x modules?
> 
> File extensions are only important on Windows. Or does anybody know a
> system which starts scripts based on the extension like Windows does?

No, but Python looks for modules with the extension ".py" and not ".py3".

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From amcnabb at mcnabbs.org  Sat Feb  2 18:36:38 2008
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Sat, 2 Feb 2008 10:36:38 -0700
Subject: [Python-3000] Set literal
In-Reply-To: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
	<87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080202173638.GH2248@mcnabbs.org>

On Sat, Feb 02, 2008 at 11:16:16AM +0900, Stephen J. Turnbull wrote:
> Greg Ewing writes:
> 
>  > Also, if generator expressions had come first, we
>  > would probably never have gotten list comprehensions.
> 
> I don't understand.  A list comprehension is for when you know you're
> going to want to work with the whole list.  Similarly (even more so)
> for dictionary comprehensions.  List comprehension can be defined in
> terms of a genexp, of course, but ISTM it's a worthwhile addition to
> the language in its own right.

I think Greg was just pointing out that with generator expressions, you
can write:

>>> lst = list(x**2 for x in xrange(10))
>>>

which would make it less necessary to the equivalent syntax:

>>> lst = [x**2 for x in xrange(10)]
>>>


-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20080202/90433ca9/attachment.pgp 

From mwm at mired.org  Sat Feb  2 17:46:57 2008
From: mwm at mired.org (Mike Meyer)
Date: Sat, 2 Feb 2008 11:46:57 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <18340.33870.160673.135647@montanaro-dyndns-org.local>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
	<18340.33870.160673.135647@montanaro-dyndns-org.local>
Message-ID: <20080202114657.60185e56@bhuda.mired.org>

On Sat, 2 Feb 2008 08:55:10 -0600 skip at pobox.com wrote:
> Anybody who installed multiple versions of Python on their computer today -
> at least in the Unixoid world - already has to know about "make altinstall"
> so they don't accidentally obliterate the python symlink.  I don't
> understand what all the fuss is.

And - at least in my experience - chances are good that nothing but
the simplest scripts will work on multiple versions anyway, because
they don't have the same set of third party - or possibly even builtin
- modules installed, unless someone took care to make sure that
exactly that happened, and even then the scripts generally only moved
one direction.

And it's not like backwards compatibility problems are unheard of,
either with Python and pretty much every other language community.  I
think the fuss is because the Python community has generally taken
more care with this than other languages, and in this case we're
explicitly ignoring it.

    <mike

-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From stephen at xemacs.org  Sat Feb  2 22:04:30 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 03 Feb 2008 06:04:30 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <fo25te$ch8$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
Message-ID: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>

Christian Heimes writes:

 > > Having to type "python3.0" explicitly strikes me as unnecessarily verbose.
 > 
 > +1

It won't once python 3.1 is released.

 > > As for module file name extensions, what speaks against allowing (not
 > > mandating) "py3" for Python 3.x modules?

Namespace pollution, of various kinds.  Python 3.x modules will
already be placed in python3.x/site-packages anyway.

I know that in Emacs we have a registry for which mode should be used
by default for files with names matching a given regexp.  The initial
registry basically stabilized a few years back, because new
applications were bumping into extensions (the regexps are 95% of the
form "\\.abc$") previously claimed.

I think this is precisely analogous to recognizing which interpreter
should be used by extension, as well as being a practical case where
you're likely to get immediate pushback.

 > File extensions are only important on Windows. Or does anybody know a
 > system which starts scripts based on the extension like Windows does?

Mac OS X's open command does something according to extension:

chibi:tmp steve$ echo 'print "hello, world"' >> hello.py
chibi:tmp steve$ python hello.py 
hello, world
chibi:tmp steve$ open hello.py 
chibi:tmp steve$ 

I don't know what, though.  The disk spun, I waited a second, and then
the prompt returned. :-)

 > I see two ways to solve the program for Windows:
 > 
 > *  register additional file extensions for each Python version
 >    like .py26, .pyw26 (for pythonw.exe), .py30 etc.

FWIW, I can tell you that Skip will get pushback from me if he
proposes registering those extensions in XEmacs's auto-mode-alist.  I
won't absolutely veto it, but I'll certainly suggest to the other
reviewers that this is a bad idea.

 > * Write a wrapper for Windows which inspects the first line of
 >   a script, searches for "#! ... python2.6" or "#!python26" and
 >   starts the right interpreter.

How about a wrapper which (like the Unix shells) starts the
interpreter from the file named in the shebang if any, and provide a
tool for updating shebangs conveniently (for the user) and
automatically (for installers).

Of course you need to make sure that this wrapper doesn't use any
Python 3000 features.<wink>

From lists at cheimes.de  Sat Feb  2 22:15:59 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 02 Feb 2008 22:15:59 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <fo2ak6$qet$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>	<fo25te$ch8$1@ger.gmane.org>
	<fo2ak6$qet$1@ger.gmane.org>
Message-ID: <fo2mif$t4i$1@ger.gmane.org>

Georg Brandl wrote:
> No, but Python looks for modules with the extension ".py" and not ".py3".

Python should still use the .py, .pyc and .pyo extensions for Python
3.0. I'm proposing a different extension for entry points to programs -
the Python file that starts a program and contains the if __name__ ==
"__main__" magic.

Do you suggest to drop the py extension and use py3 for all python files?

Christian


From lists at cheimes.de  Sat Feb  2 22:33:07 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 02 Feb 2008 22:33:07 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>	<fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <47A4E193.6090503@cheimes.de>

Stephen J. Turnbull wrote:
> Mac OS X's open command does something according to extension:
> 
> chibi:tmp steve$ echo 'print "hello, world"' >> hello.py
> chibi:tmp steve$ python hello.py 
> hello, world
> chibi:tmp steve$ open hello.py 
> chibi:tmp steve$ 
> 
> I don't know what, though.  The disk spun, I waited a second, and then
> the prompt returned. :-)

I don't know how Mac OS X's open wrapper works. It should open the same
program as finder does when you clock on a file.

> How about a wrapper which (like the Unix shells) starts the
> interpreter from the file named in the shebang if any, and provide a
> tool for updating shebangs conveniently (for the user) and
> automatically (for installers).
> 
> Of course you need to make sure that this wrapper doesn't use any
> Python 3000 features.<wink>

I'm not interested in writing a replacement for the shebang magic. It
may lead to more problems than it's worth the additional work. The
wrapper must be implemented in C.

Christian


From adlaiff6 at gmail.com  Sat Feb  2 22:33:30 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Sat, 2 Feb 2008 16:33:30 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <fo2mif$t4i$1@ger.gmane.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<fo2ak6$qet$1@ger.gmane.org> <fo2mif$t4i$1@ger.gmane.org>
Message-ID: <cc7430500802021333ub10daa7t288562c83902159c@mail.gmail.com>

On Feb 2, 2008 4:15 PM, Christian Heimes <lists at cheimes.de> wrote:
> Python should still use the .py, .pyc and .pyo extensions for Python
> 3.0. I'm proposing a different extension for entry points to programs -
> the Python file that starts a program and contains the if __name__ ==
> "__main__" magic.

-1

It doesn't make any sense to me why we would want a different
extension just because some code is the "part to run first".  Isn't it
all the same code?

--
Cheers,
Leif

From skip at pobox.com  Sat Feb  2 22:52:31 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 2 Feb 2008 15:52:31 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <18340.58911.137440.123488@montanaro.dyndns.org>


    Stephen> FWIW, I can tell you that Skip will get pushback from me if he
    Stephen> proposes registering those extensions in XEmacs's
    Stephen> auto-mode-alist.  I won't absolutely veto it, but I'll
    Stephen> certainly suggest to the other reviewers that this is a bad
    Stephen> idea.

I would never ask for such a thing. "xyz.py" is fine by me for Python3
source files.

Skip

From greg.ewing at canterbury.ac.nz  Sat Feb  2 23:10:25 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 03 Feb 2008 11:10:25 +1300
Subject: [Python-3000] Set literal
In-Reply-To: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
	<87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <47A4EA51.3050506@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Greg Ewing writes:
> 
>  > Also, if generator expressions had come first, we
>  > would probably never have gotten list comprehensions.
> 
> I don't understand.  A list comprehension is for when you know you're
> going to want to work with the whole list.

But list(genexp) gives you very nearly the same thing --
so near that I find it hard to imagine anyone would have
thought to suggest a dedicated LC syntax if genexps had
already been present in the language.

And if anyone had, Guido would have said "-1, too many
ways to do it".

--
Greg

From g.brandl at gmx.net  Sat Feb  2 23:27:02 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 02 Feb 2008 23:27:02 +0100
Subject: [Python-3000] Set literal
In-Reply-To: <47A4EA51.3050506@canterbury.ac.nz>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>	<479D6A45.6090607@canterbury.ac.nz>	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>	<479E5B42.1010908@canterbury.ac.nz>	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>	<479E6916.3050208@canterbury.ac.nz>	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>	<47A10593.7080805@canterbury.ac.nz>	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>	<47A3B43E.1060704@canterbury.ac.nz>	<87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4EA51.3050506@canterbury.ac.nz>
Message-ID: <fo2qnh$98r$1@ger.gmane.org>

Greg Ewing schrieb:
> Stephen J. Turnbull wrote:
>> Greg Ewing writes:
>> 
>>  > Also, if generator expressions had come first, we
>>  > would probably never have gotten list comprehensions.
>> 
>> I don't understand.  A list comprehension is for when you know you're
>> going to want to work with the whole list.
> 
> But list(genexp) gives you very nearly the same thing --
> so near that I find it hard to imagine anyone would have
> thought to suggest a dedicated LC syntax if genexps had
> already been present in the language.
> 
> And if anyone had, Guido would have said "-1, too many
> ways to do it".

Hm, but wouldn't he then have removed list comprehensions
by now using the time machine?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From greg.ewing at canterbury.ac.nz  Sat Feb  2 23:45:38 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 03 Feb 2008 11:45:38 +1300
Subject: [Python-3000] [Python-ideas] Namespaces are one honking	great
 idea -- let's do more of those!
In-Reply-To: <20080202114657.60185e56@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
	<18340.33870.160673.135647@montanaro-dyndns-org.local>
	<20080202114657.60185e56@bhuda.mired.org>
Message-ID: <47A4F292.6080707@canterbury.ac.nz>

Mike Meyer wrote:

> And - at least in my experience - chances are good that nothing but
> the simplest scripts will work on multiple versions anyway, because
> they don't have the same set of third party - or possibly even builtin
> - modules installed, unless someone took care to make sure that
> exactly that happened, and even then the scripts generally only moved
> one direction.

But at least you *can* set things up and write scripts
so that the same code works across versions. It's
going to be impossible, or at least very difficult, to
write code that works unchanged in both 2.x and 3.x.

--
Greg



From greg.ewing at canterbury.ac.nz  Sat Feb  2 23:59:47 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 03 Feb 2008 11:59:47 +1300
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <47A4F5E3.7030609@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Mac OS X's open command does something according to extension:
> 
> I don't know what, though.  The disk spun, I waited a second, and then
> the prompt returned. :-)

I just tried an experiment on my system, and it appears
to want to launch IDLE -- but IDLE seems to be broken for
me, so it doesn't work properly.

But MacOSX certainly *can* launch things based on filename
extension, so it would be possible to set it up to work
the way Windows does, I think.

Personally I never use it that way -- I launch all my
text-mode Python programs from a shell, and make application
bundles of anything I want to launch from the Finder.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Feb  3 00:02:30 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 03 Feb 2008 12:02:30 +1300
Subject: [Python-3000] Set literal
In-Reply-To: <fo2qnh$98r$1@ger.gmane.org>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
	<87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4EA51.3050506@canterbury.ac.nz> <fo2qnh$98r$1@ger.gmane.org>
Message-ID: <47A4F686.2050800@canterbury.ac.nz>

Georg Brandl wrote:
> Hm, but wouldn't he then have removed list comprehensions
> by now using the time machine?

I seem to remember that LCs were being seriously considered
for removal in 3.x at one point. I'm not sure of all the
reasons why they were kept.

--
Greg

From lists at cheimes.de  Sun Feb  3 00:58:10 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 03 Feb 2008 00:58:10 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <cc7430500802021333ub10daa7t288562c83902159c@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>
	<fo25te$ch8$1@ger.gmane.org>	<fo2ak6$qet$1@ger.gmane.org>
	<fo2mif$t4i$1@ger.gmane.org>
	<cc7430500802021333ub10daa7t288562c83902159c@mail.gmail.com>
Message-ID: <fo302i$m1s$1@ger.gmane.org>

Leif Walsh wrote:
> It doesn't make any sense to me why we would want a different
> extension just because some code is the "part to run first".  Isn't it
> all the same code?

It makes perfectly sense for some OSes. On Windows there is already a
second file extension for Python scripts called "pyw".


From stephen at xemacs.org  Sun Feb  3 01:55:08 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 03 Feb 2008 09:55:08 +0900
Subject: [Python-3000] Set literal
In-Reply-To: <47A4EA51.3050506@canterbury.ac.nz>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A3B43E.1060704@canterbury.ac.nz>
	<87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4EA51.3050506@canterbury.ac.nz>
Message-ID: <87tzkq97v7.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > But list(genexp) gives you very nearly the same thing [as a list
 > comprehension] -- so near that I find it hard to imagine anyone
 > would have thought to suggest a dedicated LC syntax if genexps had
 > already been present in the language.

Point taken, except that I find your faith in human nature
touching.<wink>  *Somebody* would have suggested it!

From stephen at xemacs.org  Sun Feb  3 02:05:01 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 03 Feb 2008 10:05:01 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <18340.58911.137440.123488@montanaro.dyndns.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<18340.58911.137440.123488@montanaro.dyndns.org>
Message-ID: <87sl0a97eq.fsf@uwakimon.sk.tsukuba.ac.jp>

skip at pobox.com writes:
 > 
 >     Stephen> FWIW, I can tell you that Skip will get pushback from me if he
 >     Stephen> proposes registering those extensions in XEmacs's
 >     Stephen> auto-mode-alist.  I won't absolutely veto it, but I'll
 >     Stephen> certainly suggest to the other reviewers that this is a bad
 >     Stephen> idea.
 > 
 > I would never ask for such a thing. "xyz.py" is fine by me for Python3
 > source files.

You personally wouldn't.  If users of Python-mode asked for it, you
would pass on the requests, though, I suppose.  And they will if those
extensions get registered in Windows as executable.

I've long thought what the Emacsen should do is learn to recognize
shebangs and other content-based file magic.  XEmacs *can*, I think
GNU can too, but we haven't worked out how to do it in a way that
interacts well with the extension recognition feature (which is what
Ye Olde Guard favors).



From stephen at xemacs.org  Sun Feb  3 02:24:31 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 03 Feb 2008 10:24:31 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A4F1D5.5080405@canterbury.ac.nz>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
Message-ID: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido asked that this thread be moved to python-3000.  Reply-To set.

Greg Ewing writes:

 > Adam Olsen wrote:
 > > They also have often broken python code that depended on one or two
 > > small details, and we haven't needed a new extension before.
 > 
 > But we haven't had such major incompatibilities
 > between versions before.
 > 
 > At the least, I think the executable should only
 > be installed as "python3", and never just "python"
 > (unless you know what you're doing and ask for it).

I don't see any need to shorten "python3.0" to "python3".

Also, another exception should be if there is no other Python
installed.  Then installing a link at "python" should be OK.


From rrr at ronadam.com  Sun Feb  3 02:38:27 2008
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 02 Feb 2008 19:38:27 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A4E193.6090503@cheimes.de>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de>
Message-ID: <47A51B13.4020500@ronadam.com>



Christian Heimes wrote:
> Stephen J. Turnbull wrote:
>> Mac OS X's open command does something according to extension:
>>
>> chibi:tmp steve$ echo 'print "hello, world"' >> hello.py
>> chibi:tmp steve$ python hello.py 
>> hello, world
>> chibi:tmp steve$ open hello.py 
>> chibi:tmp steve$ 
>>
>> I don't know what, though.  The disk spun, I waited a second, and then
>> the prompt returned. :-)
> 
> I don't know how Mac OS X's open wrapper works. It should open the same
> program as finder does when you clock on a file.
> 
>> How about a wrapper which (like the Unix shells) starts the
>> interpreter from the file named in the shebang if any, and provide a
>> tool for updating shebangs conveniently (for the user) and
>> automatically (for installers).
>>
>> Of course you need to make sure that this wrapper doesn't use any
>> Python 3000 features.<wink>
> 
> I'm not interested in writing a replacement for the shebang magic. It
> may lead to more problems than it's worth the additional work. The
> wrapper must be implemented in C.
> 
> Christian


Instead of a shebang which depends on the shell, maybe a version specifier 
of some sort could be used?

    # -*- pyversions: 2.5, 2.6 -*-

So if a python 3.x detects a too low a version, maybe it can try to restart 
the program with the highest installed version specified.  (Or some 
variation of this.)

Ron


From adlaiff6 at gmail.com  Sun Feb  3 02:52:14 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Sat, 2 Feb 2008 20:52:14 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <47A51B13.4020500@ronadam.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
Message-ID: <cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>

On Feb 2, 2008 8:38 PM, Ron Adam <rrr at ronadam.com> wrote:
> Instead of a shebang which depends on the shell, maybe a version specifier
> of some sort could be used?
>
>     # -*- pyversions: 2.5, 2.6 -*-
>
> So if a python 3.x detects a too low a version, maybe it can try to restart
> the program with the highest installed version specified.  (Or some
> variation of this.)

There is already an idiom with other packages (pygtk etc.) that
suggests syntax like

try:
    python.require('3.1')
except:
    print('Some warning about version incompatibility')
    exit(1)

This concern seems better addressed within the language itself, rather
than in the shebang (which would make the decision depend on the shell
and the packaging involved).

-- 
Cheers,
Leif

From skip at pobox.com  Sun Feb  3 04:58:48 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 2 Feb 2008 21:58:48 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
Message-ID: <18341.15352.788886.98828@montanaro.dyndns.org>


    Leif> There is already an idiom with other packages (pygtk etc.) that
    Leif> suggests syntax like

    Leif> try:
    Leif>     python.require('3.1')
    Leif> except:
    Leif>     print('Some warning about version incompatibility')
    Leif>     exit(1)

It won't work in this case.  Compilation of the module might well fail
because of Python2/Python3 syntax differences.

Skip

From greg.ewing at canterbury.ac.nz  Sun Feb  3 08:26:26 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 03 Feb 2008 20:26:26 +1300
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <47A56CA2.1010701@canterbury.ac.nz>

Stephen J. Turnbull wrote:

> I don't see any need to shorten "python3.0" to "python3".

I was thinking that "python3" would be the 3.x series
equivalent of what "python" is now, i.e. installation
of 3.0 would create links called "python3.0" and (unless
you're doing altinstall) also "python3".

> Also, another exception should be if there is no other Python
> installed.  Then installing a link at "python" should be OK.

Until you subsequently go to install one of the 2.x
versions.

--
Greg


From greg at krypto.org  Sun Feb  3 08:47:50 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 2 Feb 2008 23:47:50 -0800
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
Message-ID: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>

yep we've already been through that problem in the past when list
comprehensions, generators and with were added to name a few.  since python
3 code is highly unlikely to even parse with a 2.x interpreter much of the
time thats a reason to consider a .py3 extension if this precedent of not
caring is to be broken.

-0 on using .py3 instead of .py for python 3.  i think its slightly silly
but i won't object.

+1 on devising a way (to appear starting in 2.6 and 3.0) of marking the
minimum language version required by a file near the top so that the parser
can bail with a useful error message instead of one confusing to the lay
person.  similar to a from future import type of thing perhaps?  or should
python parse a #! line and magically notice a required major version number
in the python binary name (evil but it would work and magically annotate a
bunch of existing code as 2 vs 3)?

also yes python 3 should build and install as 'python3' as the short form of
its name though realistically anyone sane will write their code with #!
lines specifying the actual version in the executable name and have shell
aliases setup if they want an even shorter name.

On 2/2/08, skip at pobox.com <skip at pobox.com> wrote:
>
>
>     Leif> There is already an idiom with other packages (pygtk etc.) that
>     Leif> suggests syntax like
>
>     Leif> try:
>     Leif>     python.require('3.1')
>     Leif> except:
>     Leif>     print('Some warning about version incompatibility')
>     Leif>     exit(1)
>
> It won't work in this case.  Compilation of the module might well fail
> because of Python2/Python3 syntax differences.
>
> Skip
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080202/4bbec1cf/attachment.htm 

From tom at vector-seven.com  Sun Feb  3 09:22:08 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 03 Feb 2008 19:22:08 +1100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
Message-ID: <47A579B0.60208@vector-seven.com>


Leif Walsh wrote:
> On Feb 2, 2008 8:38 PM, Ron Adam <rrr at ronadam.com> wrote:
>  
>> Instead of a shebang which depends on the shell, maybe a version 
>> specifier
>> of some sort could be used?
>>
>>     # -*- pyversions: 2.5, 2.6 -*-
>>
>> So if a python 3.x detects a too low a version, maybe it can try to 
>> restart
>> the program with the highest installed version specified.  (Or some
>> variation of this.)
>>     
>
> There is already an idiom with other packages (pygtk etc.) that
> suggests syntax like
>
> try:
>     python.require('3.1')
> except:
>     print('Some warning about version incompatibility')
>     exit(1)
>
> This concern seems better addressed within the language itself, rather
> than in the shebang (which would make the decision depend on the shell
> and the packaging involved).
>
>   
Great idea. Since we've already got everything we need in version_info, 
this would be trivial and could address most of the issues raised ... 
something like this?

 >>> def checkversion(maj, min, rel):
...     version = sys.version_info
...     if version[0] < maj or version[1] < min or version[2] < rel:
...             raise Exception('Version %d.%d.%d of Python is required 
for this program' % (maj, min, rel))
...
 >>> checkversion(2, 5, 0)
 >>> checkversion(3, 0, 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "<stdin>", line 4, in checkversion
Exception: Version 3.0.0 of Python is required for this program
 >>>

Cheers,
T



From tom at vector-seven.com  Sun Feb  3 09:23:20 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 03 Feb 2008 19:23:20 +1100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>
	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>	<47A4E193.6090503@cheimes.de>
	<47A51B13.4020500@ronadam.com>	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
Message-ID: <47A579F8.5080302@vector-seven.com>

skip at pobox.com wrote:
>     Leif> There is already an idiom with other packages (pygtk etc.) that
>     Leif> suggests syntax like
>
>     Leif> try:
>     Leif>     python.require('3.1')
>     Leif> except:
>     Leif>     print('Some warning about version incompatibility')
>     Leif>     exit(1)
>
> It won't work in this case.  Compilation of the module might well fail
> because of Python2/Python3 syntax differences.
>
>   
Ah, of course. Scratch my last email. :)

Cheers,
T


From tom at vector-seven.com  Sun Feb  3 09:02:02 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 03 Feb 2008 19:02:02 +1100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>
	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>	<47A4E193.6090503@cheimes.de>
	<47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
Message-ID: <47A574FA.9010308@vector-seven.com>


Leif Walsh wrote:
> On Feb 2, 2008 8:38 PM, Ron Adam <rrr at ronadam.com> wrote:
>   
>> Instead of a shebang which depends on the shell, maybe a version specifier
>> of some sort could be used?
>>
>>     # -*- pyversions: 2.5, 2.6 -*-
>>
>> So if a python 3.x detects a too low a version, maybe it can try to restart
>> the program with the highest installed version specified.  (Or some
>> variation of this.)
>>     
>
> There is already an idiom with other packages (pygtk etc.) that
> suggests syntax like
>
> try:
>     python.require('3.1')
> except:
>     print('Some warning about version incompatibility')
>     exit(1)
>
> This concern seems better addressed within the language itself, rather
> than in the shebang (which would make the decision depend on the shell
> and the packaging involved).
>
>   
Great idea. Since we've already got everything we need in version_info, 
this would be trivial and could address most of the issues raised ... 
something like this?

 >>> def checkversion(maj, min, rel):
...     version = sys.version_info
...     if version[0] < maj or version[1] < min or version[2] < rel:
...             raise Exception('Version %d.%d.%d of Python is required 
for this program' % (maj, min, rel))
...
 >>> checkversion(2, 5, 0)
 >>> checkversion(3, 0, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in checkversion
Exception: Version 3.0.0 of Python is required for this program
 >>>

Cheers,
T


From stephen at xemacs.org  Sun Feb  3 10:15:33 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 03 Feb 2008 18:15:33 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A56CA2.1010701@canterbury.ac.nz>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A56CA2.1010701@canterbury.ac.nz>
Message-ID: <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Stephen J. Turnbull wrote:
 > 
 > > I don't see any need to shorten "python3.0" to "python3".
 > 
 > I was thinking that "python3" would be the 3.x series
 > equivalent of what "python" is now, i.e. installation
 > of 3.0 would create links called "python3.0" and (unless
 > you're doing altinstall) also "python3".

Well, yes, I understood that.  I just don't see a need for it.  At any
given time I have a preferred Python, and I alias python to that.
When I need one of the others, I invoke them as "pythonX.Y".  Even on
a shared system, most people will be allowed to have a private ~/bin
directory or shell aliases.

 > > Also, another exception should be if there is no other Python
 > > installed.  Then installing a link at "python" should be OK.
 > 
 > Until you subsequently go to install one of the 2.x
 > versions.

If somebody's introduction to Python is Python 3, I don't see why
they'd want to go back except for a specific app.  It *is* a better
language than Python 2 or Python 1.  Such an app will have an
appropriate shebang or wrapper script.

That doesn't mean that it would be *bad* if they want to go back,
that's entirely up to them.  I just think it will be rare enough that
they can ask somebody how to create a symlink if they need to.

From larry at hastings.org  Sun Feb  3 12:21:22 2008
From: larry at hastings.org (Larry Hastings)
Date: Sun, 03 Feb 2008 03:21:22 -0800
Subject: [Python-3000] Set literal
In-Reply-To: <C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>	<479D6A45.6090607@canterbury.ac.nz>	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>	<479E5B42.1010908@canterbury.ac.nz>	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>	<479E6916.3050208@canterbury.ac.nz>	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
Message-ID: <47A5A3B2.5090805@hastings.org>

Nicko van Someren wrote:
> Do we really want set literals at all, given that set(...) exists?
>
> If we are going to have one then, it seems to make sense to have  
> both.  If we are going to have both, I would rather that they generate  
> the same type of set.


Maybe the postings crossed in the ether, but Guido said as much on the 
30th; either they'd both change or neither would change.

Personally I like the idea of changing them to frozensets; a) it's easy 
to cast to set() if you want mutability, b) it makes the

    if x in { 1, 2, 5 }: # three, sir!

idiom faster.  Yeah, it's a bit ticklish wrt dict 
literals/comprehensions returning mutable types, but at least tuple() 
would have company in the immutable constants department.  So +1 from 
me... for what little that is worth.

When compiling a mutable type with an immutable equivalent, does Python 
generate create-empty-object/insert-each-item bytecodes, or does it 
create an immutable constant then cast it to the appropriate type?  My 
intuition is that the latter would be faster; given the professionalism 
of the Python development community, I assume both approaches have been 
tried, and the faster one was switched to long ago?  A coworker of mine 
got a tiny--but measurable--speedup by applying that style of 
optimization to PHP arrays.


/larry/

p.s. or maybe the create-empty-object takes an optional parameter of 
"and reserve this much space inside"?  That would probably even the 
playing field.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080203/e3d481cf/attachment.htm 

From janssen at parc.com  Sun Feb  3 19:31:57 2008
From: janssen at parc.com (Bill Janssen)
Date: Sun, 3 Feb 2008 10:31:57 PST
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <47A579F8.5080302@vector-seven.com> 
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
	<47A579F8.5080302@vector-seven.com>
Message-ID: <08Feb3.103157pst."58696"@synergy1.parc.xerox.com>

> skip at pobox.com wrote:
> >     Leif> There is already an idiom with other packages (pygtk etc.) that
> >     Leif> suggests syntax like
> >
> >     Leif> try:
> >     Leif>     python.require('3.1')
> >     Leif> except:
> >     Leif>     print('Some warning about version incompatibility')
> >     Leif>     exit(1)
> >
> > It won't work in this case.  Compilation of the module might well fail
> > because of Python2/Python3 syntax differences.
> >
> >   
> Ah, of course. Scratch my last email. :)
> 
> Cheers,
> T

Does that mean you'd need a new keyword, something like

     requireversion 3.1

Bill

From adlaiff6 at gmail.com  Sun Feb  3 19:45:27 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Sun, 3 Feb 2008 13:45:27 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
Message-ID: <cc7430500802031045q312df430k5766ba7a907f41ae@mail.gmail.com>

On Feb 2, 2008 10:58 PM,  <skip at pobox.com> wrote:
>
>     Leif> There is already an idiom with other packages (pygtk etc.) that
>     Leif> suggests syntax like
>
>     Leif> try:
>     Leif>     python.require('3.1')
>     Leif> except:
>     Leif>     print('Some warning about version incompatibility')
>     Leif>     exit(1)
>
> It won't work in this case.  Compilation of the module might well fail
> because of Python2/Python3 syntax differences.

Well, then this isn't something to be implemented for python 2, it's
something to consider for python 3 and greater.  I see what you're
saying though, and perhaps in the interim, a shebang fix will suffice.

-- 
Cheers,
Leif

From skip at pobox.com  Sun Feb  3 21:10:26 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 3 Feb 2008 14:10:26 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <cc7430500802031045q312df430k5766ba7a907f41ae@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
	<cc7430500802031045q312df430k5766ba7a907f41ae@mail.gmail.com>
Message-ID: <18342.8114.345326.669792@montanaro-dyndns-org.local>


    >> It won't work in this case.  Compilation of the module might well
    >> fail because of Python2/Python3 syntax differences.

    Leif> Well, then this isn't something to be implemented for python 2,
    Leif> it's something to consider for python 3 and greater.  

It won't work there either.  It's quite possible that a module written for
Python 2 won't compile under Python 3.  Compilation is currently the only
route to execution.  You'd have to come up with some other scheme.  Someone
suggested something akin to a "from __future__" facility or the #! trick you
and others mentioned, but that would only work for Python 2.6 and above.

Skip


From greg.ewing at canterbury.ac.nz  Sun Feb  3 21:41:35 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 04 Feb 2008 09:41:35 +1300
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A574FA.9010308@vector-seven.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<47A574FA.9010308@vector-seven.com>
Message-ID: <47A626FF.9050008@canterbury.ac.nz>

> Leif Walsh wrote:

>>try:
>>    python.require('3.1')

A run-time test isn't sufficient, because not all 2.x
code will even parse in 3.x. Something syntactic is
needed.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Feb  3 21:47:27 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 04 Feb 2008 09:47:27 +1300
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A56CA2.1010701@canterbury.ac.nz>
	<87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <47A6285F.90008@canterbury.ac.nz>

Stephen J. Turnbull wrote:

> If somebody's introduction to Python is Python 3, I don't see why
> they'd want to go back except for a specific app.

A specified app seems quite a likely reason to me. And
if they've gotten used to "python" taking them to python3.x,
they'll get a surprise when it suddenly starts pointing to
2.x after they installed an older Python for what to them
was meant to be a restricted purpose.

I just think the python2 and python3 worlds should be
kept separate by default, seeing as there are both forwards
and backwards incompatibilities.

--
Greg

From jim at zope.com  Sun Feb  3 22:27:56 2008
From: jim at zope.com (Jim Fulton)
Date: Sun, 3 Feb 2008 16:27:56 -0500
Subject: [Python-3000] [Python-Dev]  inst_persistent_id
In-Reply-To: <acd65fa20801241825g54dc8232h5bc4721426a7bb4d@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>
	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<acd65fa20801241825g54dc8232h5bc4721426a7bb4d@mail.gmail.com>
Message-ID: <832CE438-9760-4DAE-95FD-34DF1ACF8871@zope.com>


On Jan 24, 2008, at 9:25 PM, Alexandre Vassalotti wrote:

> On Jan 24, 2008 9:47 AM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Jan 23, 2008, at 4:30 PM, Alexandre Vassalotti wrote:
>>> I am not sure what you mean by "cPickle.Pickler and  
>>> cPickle.Unpickler
>>> subclassible in the same way as the pickle classes." It is  
>>> possible to
>>> subclass the C implementation. However, the C implementation does  
>>> not
>>> expose pickle "private" methods, such as Pickler.save_int and
>>> Unpickler.load_tuple. This is fine because these were never  
>>> documented
>>> as part of the interface of pickle.
>>
>> That doesn't mean that they aren't overridden.  I've overridden them
>> in specialized applications.  I suspect that others have as well.
>>
>
> Well even in pickle.py, you can't override the save_* and load_*
> methods (at least, directly), since they are called via a dispatch
> dictionary.

You have to override the dictionary too. :)

>> I'm not really looking for anything that isn't there already.  The
>> python pickle version is more extensible that cPickle and I find that
>> valuable.  I don't want to make cPickle more flexible.  I'd willing  
>> to
>> trade off speed and flexibility depending on the application.  I  
>> don't
>> want to lose either in the interest of unification.
>
> How much of cPickle is really speed critical?

cPickle is all about speed.

> Personally, I think
> built-in types pickling should be optimized as much as possible and
> other things, such as user-defined classes and extension types
> pickling, could be done in Python.

The whole reduce dance should be in C, after all, it already is.

Most user-defined types inherit the reduce method(s) provided by  
object, which is already written in C.

>> Having said that, I would agree that there are certain aspects of
>> their external APIs that ought to be unified.  A good example is
>> handling of globals.
>
> Could you elaborate about this, please; how the handling of globals by
> cPickle is different from pickle?


pickle.Unpickler has a find_class method that loads globals. This can  
be overridden in a subclass to provide alternate behavior.

cPickle.Unpickler has a find_global attribute that can be assigned to  
provide alternate behavior. (Internally, a find_class C member is used  
for hysterical reasons. :)

Jim

--
Jim Fulton
Zope Corporation



From rrr at ronadam.com  Sun Feb  3 22:09:29 2008
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 03 Feb 2008 15:09:29 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <18342.8114.345326.669792@montanaro-dyndns-org.local>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
	<cc7430500802031045q312df430k5766ba7a907f41ae@mail.gmail.com>
	<18342.8114.345326.669792@montanaro-dyndns-org.local>
Message-ID: <47A62D89.6020202@ronadam.com>



skip at pobox.com wrote:
>     >> It won't work in this case.  Compilation of the module might well
>     >> fail because of Python2/Python3 syntax differences.
> 
>     Leif> Well, then this isn't something to be implemented for python 2,
>     Leif> it's something to consider for python 3 and greater.  
> 
> It won't work there either.  It's quite possible that a module written for
> Python 2 won't compile under Python 3.  Compilation is currently the only
> route to execution.  You'd have to come up with some other scheme.  Someone
> suggested something akin to a "from __future__" facility or the #! trick you
> and others mentioned, but that would only work for Python 2.6 and above.
> 
> Skip

The tokenizer.c file has the routines to detect encoding specifiers before 
the file is compiled.  So it could be done there right after the file 
encoding is detected.

Ron










From jim at zope.com  Sun Feb  3 22:37:18 2008
From: jim at zope.com (Jim Fulton)
Date: Sun, 3 Feb 2008 16:37:18 -0500
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>
	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
Message-ID: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>


On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote:

> On Jan 24, 2008 8:59 PM, Aahz <aahz at pythoncraft.com> wrote:
>> On Thu, Jan 24, 2008, Jim Fulton wrote:
>>>
>>> I don't know what that means.  I don't know that Jython or  
>>> IronPython
>>> need to support cPickle. Honestly, I'd be happy to see a *much*
>>> smaller standard library and, IMO, the standard library doesn't need
>>> to include pickle or cPickle.
>>
>> While it's easy enough to work around, my company would be unhappy if
>> pickle were removed, and so would my previous company.
>
> I don't know why Jim is saying this (*),

I think the standard library is bloated.  I'd much prefer to see a  
leaner standard library that really provides features that are close  
to the language and provide a packaging system that make it easy to  
install other packages as needed.  I think setuptools is a pretty good  
start at this.

> but it's not going to happen.
> pickle is here to stay. cPickle is a different story; 99% of programs
> shouldn't be bothered with the choice between the two.

IMO, the programs the people who care about cPickle don't want to  
sacrifice speed to get the same flexibility as pickle. Similarly, I  
don't want to lose pickles flexibility in the interest of unification.

> (*) I can only speculate that it's because Jim, whose ZODB is probably
> pickle's most intensive user,

Maybe, but I'm not sure.

> would like to "own" it

Not particularly.

> and would like to
> evolve it much faster than the nearly frozen version in the standard
> library.

Fortunately, that's not the case anymore. (Except maybe for settling  
the inst_persistent_id optimization.)

> I would like to suggest that Jim can evolve pickle without
> removing it from the standard library -- he could just include a fork
> in the Zope package.

Of course. No need to suggest it. :)

As I said earlier, I have a general desire to see a smaller standard  
library.

Jim

--
Jim Fulton
Zope Corporation



From brett at python.org  Sun Feb  3 23:33:51 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 3 Feb 2008 14:33:51 -0800
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>
	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
Message-ID: <bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>

On Feb 3, 2008 1:37 PM, Jim Fulton <jim at zope.com> wrote:
>
> On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote:
>
> > On Jan 24, 2008 8:59 PM, Aahz <aahz at pythoncraft.com> wrote:
> >> On Thu, Jan 24, 2008, Jim Fulton wrote:
> >>>
> >>> I don't know what that means.  I don't know that Jython or
> >>> IronPython
> >>> need to support cPickle. Honestly, I'd be happy to see a *much*
> >>> smaller standard library and, IMO, the standard library doesn't need
> >>> to include pickle or cPickle.
> >>
> >> While it's easy enough to work around, my company would be unhappy if
> >> pickle were removed, and so would my previous company.
> >
> > I don't know why Jim is saying this (*),
>
> I think the standard library is bloated.  I'd much prefer to see a
> leaner standard library that really provides features that are close
> to the language and provide a packaging system that make it easy to
> install other packages as needed.  I think setuptools is a pretty good
> start at this.
>

But we need a solution, not a start. While the stdlib is bloated, it
is being trimmed down in Py3K already. If you want to trim more then
push for stuff to be removed on a module-to-module basis. But going
from "batteries included" to "batteries easily downloaded" is quite a
shift for Python. And as of right now I really don't see how that
level of a shift can be coded, discussed, and put into Py3K before it
is released. Let alone how difficult it would be to get the manpower
lined up to maintain a server that is going to get hammered by module
downloads (PyPI still goes down on occasion and this would just
increase the strain on the server by a good amount).

> > but it's not going to happen.
> > pickle is here to stay. cPickle is a different story; 99% of programs
> > shouldn't be bothered with the choice between the two.
>
> IMO, the programs the people who care about cPickle don't want to
> sacrifice speed to get the same flexibility as pickle. Similarly, I
> don't want to lose pickles flexibility in the interest of unification.
>

Flexibility is not going anywhere; the C implementation is the one
that needs to change. If the speed is not fast enough then patches are
welcome. But I know I am tired of maintaining two test suites, two
sets of docs, explaining to users why they can't subclass stuff from
cPickle, etc. Plus alternative Python implementations should not have
to say "we are incompatible with the stdlib because we don't have
cPickle and its quirks".

-Brett

From p.f.moore at gmail.com  Sun Feb  3 23:55:43 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 3 Feb 2008 22:55:43 +0000
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <79990c6b0802031454s79fffcbam14379225c9c11568@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
	<79990c6b0802031454s79fffcbam14379225c9c11568@mail.gmail.com>
Message-ID: <79990c6b0802031455i75eeabf1jfd916a9c71561a41@mail.gmail.com>

Whoops, that was meant to go to the list. Sorry.

On 03/02/2008, Brett Cannon <brett at python.org> wrote:
> But we need a solution, not a start. While the stdlib is bloated, it
> is being trimmed down in Py3K already. If you want to trim more then
> push for stuff to be removed on a module-to-module basis. But going
> from "batteries included" to "batteries easily downloaded" is quite a
> shift for Python.

Absolutely. For a nice example of a "worst case" scenario, I'm behind
a corporate filewall which doesn't even let out HTTP except via the
Windows APIs (Windows ISA proxy, or some such). I know there are
workarounds for this, but that's not the point. Add to that no
compiler available plus a restrictive corporate policy, and there's
pretty much nothing that isn't in the core that would ever be "easily
downloaded".

I've no idea how representative I am, or what I might be
representative of (:-)) but for some of us, a large core is a real
benefit.

Paul.

From greg.ewing at canterbury.ac.nz  Mon Feb  4 00:15:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 04 Feb 2008 12:15:22 +1300
Subject: [Python-3000] Set literal
In-Reply-To: <47A5A3B2.5090805@hastings.org>
References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net>
	<ca471dc20801242012t3abffa0fy27bba39bc980e982@mail.gmail.com>
	<aac2c7cb0801261950j31f804feg28fd523e0bafcaa2@mail.gmail.com>
	<00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1>
	<ca471dc20801262120h342069e6ye29cc7d7a2f7e8e@mail.gmail.com>
	<479D6A45.6090607@canterbury.ac.nz>
	<ca471dc20801280802s68875edck907008f797a4ccc5@mail.gmail.com>
	<479E5B42.1010908@canterbury.ac.nz>
	<ca471dc20801281503n3f02344bx156502fb2ceaecde@mail.gmail.com>
	<479E6916.3050208@canterbury.ac.nz>
	<53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org>
	<47A10593.7080805@canterbury.ac.nz>
	<C2CFD6B1-4D50-41E9-A1D7-E4DBD5B0B346@nicko.org>
	<47A5A3B2.5090805@hastings.org>
Message-ID: <47A64B0A.6050306@canterbury.ac.nz>

Larry Hastings wrote:
> When compiling a mutable type with an immutable equivalent, does Python 
> generate create-empty-object/insert-each-item bytecodes, or does it 
> create an immutable constant then cast it to the appropriate type?

Last time I looked, for lists and dicts it creates an empty
object and adds items to it one at a time, and for tuples it
pushes all the items onto the stack and then uses a special
create-tuple bytecode.

For frozenset constructors it could use a similar technique as
for tuples. If set comprehensions were to produce frozen sets,
something else would need to be done, such as creating it
mutable initially and freezing it after construction, or
having a special bytecode that's allowed to add an item to
a frozenset.

> p.s. or maybe the create-empty-object takes an optional parameter of 
> "and reserve this much space inside"?

That wouldn't work for set comprehensions, since there's no
way of telling ahead of time how many items it will contain.
It mightn't help much anyway, given Python's proportional
reallocation strategy.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From stephen at xemacs.org  Mon Feb  4 04:55:29 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 04 Feb 2008 12:55:29 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A6285F.90008@canterbury.ac.nz>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A56CA2.1010701@canterbury.ac.nz>
	<87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A6285F.90008@canterbury.ac.nz>
Message-ID: <873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > I just think the python2 and python3 worlds should be
 > kept separate by default, seeing as there are both forwards
 > and backwards incompatibilities.

I'm afraid that's exactly what will happen: the two worlds will tend
to be *kept* separate despite the common ground.

 > A specified app seems quite a likely reason to me.

My point is that they'll install that Python for use only by the app.
They won't want to change their default Python.

 > And if they've gotten used to "python" taking them to python3.x,
 > they'll get a surprise when it suddenly starts pointing to 2.x
 > after they installed an older Python for what to them was meant to
 > be a restricted purpose.

I wonder how many people out there will be "surprised" by sudden
changes in the version of Python installed.  On Mac OS X 10.4 +
MacPorts, MacPorts does not install an /opt/local/bin/python at all,
so that although /opt/local/bin takes precedence over /usr/bin, you
get the default Mac OS X install of Python 2.3 as "python".  Linux
distributions (at least Debian and Gentoo) "anoint" a particular
Python version as the one that infrastructure software uses and that
is the default unless the user explicitly sets a different version as
"python".  Installing multiple versions of Python does not reset the
"python" link.

So it seems to me that this particular issue can easily be handled if
Python source builds by default only install a "python" link if (a)
there is none or (b) the newly installed version is an upgrade of the
current "python".



From stephen at xemacs.org  Mon Feb  4 05:16:56 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 04 Feb 2008 13:16:56 +0900
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>
	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
Message-ID: <871w7t8ifb.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:
 > On Feb 3, 2008 1:37 PM, Jim Fulton <jim at zope.com> wrote:

 > > I think the standard library is bloated.  I'd much prefer to see a
 > > leaner standard library that really provides features that are close
 > > to the language and provide a packaging system that make it easy to
 > > install other packages as needed.  I think setuptools is a pretty good
 > > start at this.
 > 
 > But we need a solution, not a start. While the stdlib is bloated, it
 > is being trimmed down in Py3K already. If you want to trim more then
 > push for stuff to be removed on a module-to-module basis. But going
 > from "batteries included" to "batteries easily downloaded" is quite a
 > shift for Python.

Trimming the standard library does not necessarily mean "batteries not
included".  It could mean that instead of getting form-fitting super-
high-output custom <whatever the current technology is>, you get a
pair of AA Duracell alkalines that you can buy by the gross at CostCo.

In other words, take whatever the module maintainer is recommending,
run the full test suite, and if everything outside of the "official
stdlib" passes its own unit tests and any random cross-module tests
that happen to touch it, ship it with the main distribution.

This is conceptually a substantial step down in quality control.  But
in practice, is it really so different from what happens now?

From ncoghlan at gmail.com  Mon Feb  4 10:28:38 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 04 Feb 2008 19:28:38 +1000
Subject: [Python-3000] pickle, cPickle,
 and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>	<20080125045906.GA21722@panix.com>	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
Message-ID: <47A6DAC6.3000301@gmail.com>

Brett Cannon wrote:
> If you want to trim more then
> push for stuff to be removed on a module-to-module basis. But going
> from "batteries included" to "batteries easily downloaded" is quite a
> shift for Python.

Not to mention it being a serious pain in the rear for those of us who 
have to get licenses vetted by our contracts department before we're 
allowed to copy open source code across to isolated networks with no 
direct access to stuff that is "in the cloud".

A batteries-included standard library is a *huge* boon in that kind of 
environment - one license to vet and one installer to copy across, 
instead of suffering a death of a thousand cuts as we have to retrieve 
different bits and pieces from PyPI (or rewrite our own versions from 
scratch).

We-don't-all-work-in-web-app-development-ly,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From qrczak at knm.org.pl  Mon Feb  4 11:00:19 2008
From: qrczak at knm.org.pl (Marcin =?UTF-8?Q?=E2=80=98Qrczak=E2=80=99?= Kowalczyk)
Date: Mon, 04 Feb 2008 11:00:19 +0100
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of	those!
In-Reply-To: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1202119219.21638.4.camel@qrnik>

Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze:

> I don't see any need to shorten "python3.0" to "python3".

There is a need. Using #!/usr/bin/python3.0 would break as soon as
python3.1 is released, while #!/usr/bin/python3 would be fine, at least
in the next few years.

Using #!/usr/bin/python is not fine in an environment with both Python 2
and Python 3 installed, and such an environment is reasonable if one has
Python 2 programs running in production which have not been updated to
Python 3 yet, while he wants to start using Python 3 for other programs.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


From stephen at xemacs.org  Mon Feb  4 12:22:04 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 04 Feb 2008 20:22:04 +0900
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of	those!
In-Reply-To: <1202119219.21638.4.camel@qrnik>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
Message-ID: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>

Marcin ?Qrczak? Kowalczyk writes:

 > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze:
 > 
 > > I don't see any need to shorten "python3.0" to "python3".
 > 
 > There is a need. Using #!/usr/bin/python3.0 would break as soon as
 > python3.1 is released, while #!/usr/bin/python3 would be fine, at
 > least in the next few years.

First of all, under current policy, installing Python 3.1 would not
uninstall or overwrite Python 3.0, so nothing "breaks" when the user
uses #!/usr/bin/python3.0.  On the contrary, Python 3.1 itself may be
buggy.  I do not think is it appropriate for the Python installer to
make that decision for the user by creating a link to the most recent
python3.x.

Second, the point here is not whether any given user has such a need.
I may disagree with your reasoning above, but you apparently feel such
a need, and that's good enough for me.  I have no objection if you
make a link python3 -> python3.0.

What I see no need for is a policy statement *by Python* that there
"should" be a "python3" link, or which python3.x is should point to.
I don't think that Python providing a simple tool to help the user do
it is a good idea, either, because most OS distributions already
provide them, and they tend to be rather obnoxious about "rogue" users
who do it by hand or use 3d party (including upstream) tools to do it.


From mwm at mired.org  Sun Feb  3 02:20:44 2008
From: mwm at mired.org (Mike Meyer)
Date: Sat, 2 Feb 2008 20:20:44 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one
	honking	great	idea -- let's do more of those!
In-Reply-To: <47A4F292.6080707@canterbury.ac.nz>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org>
	<18340.33870.160673.135647@montanaro-dyndns-org.local>
	<20080202114657.60185e56@bhuda.mired.org>
	<47A4F292.6080707@canterbury.ac.nz>
Message-ID: <20080202202044.5177a6e2@bhuda.mired.org>

On Sun, 03 Feb 2008 11:45:38 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Mike Meyer wrote:
> 
> > And - at least in my experience - chances are good that nothing but
> > the simplest scripts will work on multiple versions anyway, because
> > they don't have the same set of third party - or possibly even builtin
> > - modules installed, unless someone took care to make sure that
> > exactly that happened, and even then the scripts generally only moved
> > one direction.
> But at least you *can* set things up and write scripts
> so that the same code works across versions.

Upwards, anyway. But I find that, unless one organization is
maintaining all of them (meaning either your system has no stock
python install, or you're using it), you wind up with multiple python
installations and scripts that only work on one as a matter of course,
because that's easier than trying to make real applications work on
all of them.

> It's going to be impossible, or at least very difficult, to write
> code that works unchanged in both 2.x and 3.x.

Which in practice - at least on the applications I work on - is more a
matter of semantics than anything else. When we decide to switch to
3.x, we'll do exactly what we do for going to a new 2.Y, or a new
version of some third party library we use: create a new python build
with the libraries and tools we need, and a new application
distribution to go with it, then run them both through the qa cycle
until they pass, and we can put them in production.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Sun Feb  3 19:27:51 2008
From: mwm at mired.org (Mike Meyer)
Date: Sun, 3 Feb 2008 13:27:51 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
	<52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>
Message-ID: <20080203132751.79169d2c@bhuda.mired.org>

On Sat, 2 Feb 2008 23:47:50 -0800 "Gregory P. Smith" <greg at krypto.org> wrote:

> +1 on devising a way (to appear starting in 2.6 and 3.0) of marking the
> minimum language version required by a file near the top so that the parser
> can bail with a useful error message instead of one confusing to the lay
> person.  similar to a from future import type of thing perhaps? 

-1 on any such programmer-maintained marking. The key phrase is
"programmer-maintained." I think the confusion caused by this being
*wrong*, while less common than simply trying to use the wrong
version, will be much more painful because the user will start with
the assumption that they have a good enough version.

If the goal is to lower user confusion, how about having the 2.X line
issue better warnings if it sees 3.0 features? Just checking syntax
errors against that list and tweaking the warning?

> or should
> python parse a #! line and magically notice a required major version number
> in the python binary name (evil but it would work and magically annotate a
> bunch of existing code as 2 vs 3)?

-2 on using the shebang line for this. The 2.X version-specific
binaries are python2.X, so to use that, you'd have to make the shebang
line invoke python2.X. Right now, if you upgrade an installation from
2.X to 2.X+1, you can expect most of the scripts to work assuming you
take do care to install the same module set. Requiring explicit minor
versions breaks that.

> also yes python 3 should build and install as 'python3' as the short form of
> its name though realistically anyone sane will write their code with #!
> lines specifying the actual version in the executable name and have shell
> aliases setup if they want an even shorter name.

+1, so long as we keep the python alias as well (maybe with the
altinstall target only creating python3 if it doesn't exist). This
will help with the transition from 2 to 3, and then later from 3 to 4,
as we can use python3 vs. python4 in scripts where we now can't use
python2, because it isn't setup by default.

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Mon Feb  4 06:54:13 2008
From: mwm at mired.org (Mike Meyer)
Date: Mon, 4 Feb 2008 00:54:13 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A56CA2.1010701@canterbury.ac.nz>
	<87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A6285F.90008@canterbury.ac.nz>
	<873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080204005413.5ba02566@bhuda.mired.org>

On Mon, 04 Feb 2008 12:55:29 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Greg Ewing writes:
> I wonder how many people out there will be "surprised" by sudden
> changes in the version of Python installed.

It's happened in the past. I've been dealing with supporting multiple
incompatible versions on the same hosts for the last two years, and
users running the "wrong" python was a regular occurrence until I
convinced people that it really mattered.

> On Mac OS X 10.4 + MacPorts, MacPorts does not install an
> /opt/local/bin/python at all, so that although /opt/local/bin takes
> precedence over /usr/bin, you get the default Mac OS X install of
> Python 2.3 as "python".

Actually, Leopard comes with 2.5.1. Kudos to jkh & his team for the
upgrade. Also, /opt/local/bin should only take precedence for users;
commands run via sudo and from OS X applications that didn't come from
MacPorts should ignore /opt/local/bin.

> Linux distributions (at least Debian and Gentoo) "anoint" a
> particular Python version as the one that infrastructure software
> uses and that is the default unless the user explicitly sets a
> different version as "python".

And Red Hat. And - since all three provide (well, I haven't checked
Gentoo in a while) system tools that use their "anointed" python - any
distribution that uses those tools (most notably Ubuntu). Further,
those tools depend on third party modules that may only be available
as part of that OS release, so replacing the system python is a
seriously bad idea.

> Installing multiple versions of Python does not reset the "python"
> link.

Depends on how you do the install. The standard python source
distribution will install the python link with "make install", but
installs things in by default /usr/local, so won't break the system
python, or most of the system tools (some do use #!/usr/bin/env python
instead of #!/usr/bin/python). If you do "make altinstall", it won't
install that symlink. If you configure it to install in /usr instead
of /usr/local and do "make install", the python symlink will get reset
(and if you're on one of those Linux systems, your life is about to
get really, really unpleasant).

I haven't looked at the macports version in a while, but suspect that
it used the altinstall target to avoid surprises. On the other hand, I
had no problems on Tiger adding the symlink to /opt/local/bin, and
making sure it was first on my path (but not roots!).

> So it seems to me that this particular issue can easily be handled if
> Python source builds by default only install a "python" link if (a)
> there is none or (b) the newly installed version is an upgrade of the
> current "python".

Neither of those is really good enough. How do you decide that "there
is none"? leaving it in the hands of the installers is the only sure
solution. For the second part, if you upgrade the python without
making sure you've upgraded *all* the third party modules in the
python library, you'll break scripts that depend on the modules you
didn't upgrade.

Living with multiple python installs isn't really a problem. Upgrading
an existing installation isn't really a problem, either. Both work
fine if done with a bit of care. It's up to the person building the
distribution (or doing the install) to make provide that care. But
we've been doing it for a while now, and it's not a serious issue.

      <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From ncoghlan at gmail.com  Mon Feb  4 15:09:47 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 05 Feb 2008 00:09:47 +1000
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
Message-ID: <47A71CAB.8040303@gmail.com>

Guido van Rossum wrote:
> This should be brought up on the python-3000 list; I'm moving it there
> using a Bcc to python-ideas.
> 
> To some extent it is up to the vendors who distribute binaries -- they
> decide what to call it.
> 
> Perhaps we should only install "python3.0" and not "python". That is a
> valid choice already and always has been (python2.1, python2.2, etc.
> are always installed by default, "python" is just a convenient alias).
> 
> I think that worries about Python becoming the laughingstock of the
> language world are highly exaggerated. The post you refer to sounds to
> me like the typical cynical one-liner from someone who doesn't really
> care, not about a position of someone who is influential in the world
> of language users.

I personally haven't seen anything to convince me that the 2.x -> 3.0 
upgrade cycle is going to be significantly worse from a deployment point 
of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also 
possible (e.g. code using 'with' or 'as' as identifiers runs just fine 
on 2.4 or 2.5, but that code is going to break in 2.6).

Yes, the breakages in 3.0 are more significant and more widespread, but 
the tools to support migration and the time allowed for migration are 
correspondingly increased. I would also expect nearly all systems to 
stay with a 2.x release as the default python at least until after 3.1 
comes out.

The only question is what to do about applications that want to declare 
themselves as python 3 applications so that the platform knows how to 
handle them correctly, but also want to fail gracefully if someone tries 
to run them with the wrong version. And really, that problem already 
exists for 2.x applications that require a certain minimum version (e.g. 
if you use with statements or conditional expressions in your main 
module, even with a future statement, then it will fail to compile on 
all versions prior to 2.5).

I think the general solution is the same in all such cases: the affected 
application or library (not the language) should include a short and 
simple entry point module that uses the bare minimum amount of code 
needed to ensure the correct runtime environment. Something like:

import sys
REQUIRED_VERSION = (3, 0, 0)
if sys.version_info < REQUIRED_VERSION:
   sys.exit("Require Python version %d.%d.%d, using version %d.%d.%d"
            % (REQUIRED_VERSION + sys.version_info[:3]))
import real_main
real_main.main()

For environments where the default version of Python doesn't meet the 
application's requirements, then it will be up to the owner of that 
environment to work out how to get that specific application to run with 
the correct version (e.g. add a python3 alias and modify the main 
script's shebang line or use a wrapper script or GUI shortcut that 
explicitly invokes the correct python version).

Actually adding a python3 alias as part of the default py3k installation 
would bother me a little, as we'd then be stuck with providing it for 
the life of the 3.x series - adding cruft while trying to remove it 
seems like a backwards step.

And for in-house development, web-based applications or embedded devices 
where the developers have control of the run-time environment as well as 
the application code, then there is absolutely no problem at all, as the 
system will be configured to use the right version of the interpreter no 
matter which version it happens to be.

Cheers,
Nick.

P.S. Good thing we decided to keep string mod-formatting despite the 
addition of the format method ;)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Mon Feb  4 14:52:01 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 4 Feb 2008 07:52:01 -0600
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
Message-ID: <18343.6273.636210.6919@montanaro-dyndns-org.local>

This discussion of what to install Python 3.0 as reminded me that we've
already encountered this problem before.  As I recall, when Python 2.0 was
released Red Hat steadfastly refused to ship it with their then current
version of Linux.  It wasn't until the next RH product came out that they
shipped Python 2.x, and then the executable was called (I think) python2.

I no longer remember the details very well, but I wonder if there's
a lesson to be learned from that experience as we decide how to install
Python 3.0.

Skip

From lists at cheimes.de  Mon Feb  4 16:21:46 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 04 Feb 2008 16:21:46 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <20080203132751.79169d2c@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>
	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>	<47A4E193.6090503@cheimes.de>
	<47A51B13.4020500@ronadam.com>	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>	<18341.15352.788886.98828@montanaro.dyndns.org>	<52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>
	<20080203132751.79169d2c@bhuda.mired.org>
Message-ID: <47A72D8A.3030700@cheimes.de>

Mike Meyer wrote:
> -2 on using the shebang line for this. The 2.X version-specific
> binaries are python2.X, so to use that, you'd have to make the shebang
> line invoke python2.X. Right now, if you upgrade an installation from
> 2.X to 2.X+1, you can expect most of the scripts to work assuming you
> take do care to install the same module set. Requiring explicit minor
> versions breaks that.

I'm on Mike's side here. There is no need to fix the Unix (and Mac OS X)
side of the problem because it was never broken in the first place.

The only problematic environment is (as usual) Windows. Windows has no
shebang and python.exe is most likely not in PATH. One has to go through
the registry to find Python 2.x and its executable.

For Windows the Python stdlib should get some means of starting a script
with a well defined Python version without relying on the file
extension. One possible way is to assign the .py and .pyw extension with
a small wrapper program written in C which parses the #! shebang line,
gets the right python.exe from the registry and starts the script with it.

Another way is a new Python extension (maybe .pys for python start) for
ini file like startup files:

---
[Python]
version=2.5,2.6
module=myprogram.main
gui=True

[Env]
pythonpath=..
---

The file causes the startup program to search for 2.5 and 2.6 in the
registry. It sets the env var PYTHONPATH to .. (relative to the location
of the file) and starts the program with pythonw.exe -m "myprogram.main".

Christian


From janssen at parc.com  Mon Feb  4 16:39:21 2008
From: janssen at parc.com (Bill Janssen)
Date: Mon, 4 Feb 2008 07:39:21 PST
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of those!
In-Reply-To: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> 
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <08Feb4.073927pst."58696"@synergy1.parc.xerox.com>

> First of all, under current policy, installing Python 3.1 would not
> uninstall or overwrite Python 3.0, so nothing "breaks" when the user

What policy?

Bill

From rrr at ronadam.com  Mon Feb  4 16:45:53 2008
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 04 Feb 2008 09:45:53 -0600
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A72D8A.3030700@cheimes.de>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>	<47A4E193.6090503@cheimes.de>	<47A51B13.4020500@ronadam.com>	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>	<18341.15352.788886.98828@montanaro.dyndns.org>	<52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>	<20080203132751.79169d2c@bhuda.mired.org>
	<47A72D8A.3030700@cheimes.de>
Message-ID: <47A73331.4050000@ronadam.com>



Christian Heimes wrote:
> Mike Meyer wrote:
>> -2 on using the shebang line for this. The 2.X version-specific
>> binaries are python2.X, so to use that, you'd have to make the shebang
>> line invoke python2.X. Right now, if you upgrade an installation from
>> 2.X to 2.X+1, you can expect most of the scripts to work assuming you
>> take do care to install the same module set. Requiring explicit minor
>> versions breaks that.
> 
> I'm on Mike's side here. There is no need to fix the Unix (and Mac OS X)
> side of the problem because it was never broken in the first place.
> 
> The only problematic environment is (as usual) Windows. Windows has no
> shebang and python.exe is most likely not in PATH. One has to go through
> the registry to find Python 2.x and its executable.
> 
> For Windows the Python stdlib should get some means of starting a script
> with a well defined Python version without relying on the file
> extension. One possible way is to assign the .py and .pyw extension with
> a small wrapper program written in C which parses the #! shebang line,
> gets the right python.exe from the registry and starts the script with it.
> 
> Another way is a new Python extension (maybe .pys for python start) for
> ini file like startup files:

I like .pst better,  or maybe reuse .pth files?

Ron

> ---
> [Python]
> version=2.5,2.6
> module=myprogram.main
> gui=True
> 
> [Env]
> pythonpath=..
> ---
> 
> The file causes the startup program to search for 2.5 and 2.6 in the
> registry. It sets the env var PYTHONPATH to .. (relative to the location
> of the file) and starts the program with pythonw.exe -m "myprogram.main".
> 
> Christian
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/rrr%40ronadam.com
> 

From lists at cheimes.de  Mon Feb  4 16:53:42 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 04 Feb 2008 16:53:42 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A73331.4050000@ronadam.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>	<fo1s0g$emo$1@ger.gmane.org>	<fo25te$ch8$1@ger.gmane.org>	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>	<47A4E193.6090503@cheimes.de>	<47A51B13.4020500@ronadam.com>	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>	<18341.15352.788886.98828@montanaro.dyndns.org>	<52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>	<20080203132751.79169d2c@bhuda.mired.org>
	<47A72D8A.3030700@cheimes.de> <47A73331.4050000@ronadam.com>
Message-ID: <47A73506.60401@cheimes.de>

Ron Adam wrote:
> I like .pst better,  or maybe reuse .pth files?

MS Outlook uses the file extension .pst. Python has a history to "claim"
the .py? namespace. I like to stick to the tradition.

Christian

From guido at python.org  Mon Feb  4 17:20:15 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 4 Feb 2008 08:20:15 -0800
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
In-Reply-To: <18343.6273.636210.6919@montanaro-dyndns-org.local>
References: <18343.6273.636210.6919@montanaro-dyndns-org.local>
Message-ID: <ca471dc20802040820p279b4fd3ueb1445c7dea5d900@mail.gmail.com>

On Feb 4, 2008 5:52 AM,  <skip at pobox.com> wrote:
> This discussion of what to install Python 3.0 as reminded me that we've
> already encountered this problem before.  As I recall, when Python 2.0 was
> released Red Hat steadfastly refused to ship it with their then current
> version of Linux.  It wasn't until the next RH product came out that they
> shipped Python 2.x, and then the executable was called (I think) python2.
>
> I no longer remember the details very well, but I wonder if there's
> a lesson to be learned from that experience as we decide how to install
> Python 3.0.

IMO the lesson to be learned is that that was a bad idea -- other
vendors have not followed suit. It also gave Python a bad name because
for years, typing "python" on Red Hat systems gave you an ancient
version.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From charles.merriam at gmail.com  Mon Feb  4 19:04:27 2008
From: charles.merriam at gmail.com (Charles Merriam)
Date: Mon, 4 Feb 2008 10:04:27 -0800
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<ca471dc20802020348m2267f03ev7ebcc04c3a6ae0ec@mail.gmail.com>
	<fo1s0g$emo$1@ger.gmane.org> <fo25te$ch8$1@ger.gmane.org>
	<87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com>
	<cc7430500802021752s5ddd2917n1b438db23c4e1308@mail.gmail.com>
	<18341.15352.788886.98828@montanaro.dyndns.org>
	<52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com>
Message-ID: <b8c6f6c40802041004h376e1295kd75ed614abd85051@mail.gmail.com>

How about the ellipses as a Python 3 marker?

>From a previous discussion, the error messages on this null-op should
be made better in Python 3.0 anyway.

If ellipses are allowed as a stand-alone statement, much like pass,
then we could just have these messages:

Python 2.6:  "Unexpected '...'.   Ellipses not supported until Python
3.0.   Another cause might be pasting in code from within the IDLE
interpreter.

Python 3.0:  'Unexpected '...'.  Might be caused by pasting in code
from within the IDLE interpreter.

An idiom that might be used during the early transition period:

   ...    # Require python 3.x
   print ("I'm Pythonic!")

The idiom would disappear with time, not add special cruft to the
language, and would give a reasonable compile-time error message.

Just a thought,

Charles
---
Charles Merriam
e:  charles.merriam at gmail.com
u:  http://www.charlesmerriam.com
q:  "Crufty solutions make crufty curmudgeons."


On Feb 2, 2008 11:47 PM, Gregory P. Smith <greg at krypto.org> wrote:
> yep we've already been through that problem in the past when list
> comprehensions, generators and with were added to name a few.  since python
> 3 code is highly unlikely to even parse with a 2.x interpreter much of the
> time thats a reason to consider a .py3 extension if this precedent of not
> caring is to be broken.
>
> -0 on using .py3 instead of .py for python 3.  i think its slightly silly
> but i won't object.
>
> +1 on devising a way (to appear starting in 2.6 and 3.0) of marking the
> minimum language version required by a file near the top so that the parser
> can bail with a useful error message instead of one confusing to the lay
> person.  similar to a from future import type of thing perhaps?  or should
> python parse a #! line and magically notice a required major version number
> in the python binary name (evil but it would work and magically annotate a
> bunch of existing code as 2 vs 3)?
>
> also yes python 3 should build and install as 'python3' as the short form of
> its name though realistically anyone sane will write their code with #!
> lines specifying the actual version in the executable name and have shell
> aliases setup if they want an even shorter name.
>
>
>
> On 2/2/08, skip at pobox.com <skip at pobox.com> wrote:
> >
> >
> >
> >
> >     Leif> There is already an idiom with other packages (pygtk etc.) that
> >     Leif> suggests syntax like
> >
> >     Leif> try:
> >     Leif>     python.require('3.1')
> >     Leif> except:
> >     Leif>     print('Some warning about version incompatibility')
> >     Leif>     exit(1)
> >
> > It won't work in this case.  Compilation of the module might well fail
> > because of Python2/Python3 syntax differences.
> >
> > Skip

From charles.merriam at gmail.com  Mon Feb  4 20:24:55 2008
From: charles.merriam at gmail.com (Charles Merriam)
Date: Mon, 4 Feb 2008 11:24:55 -0800
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <47A6DAC6.3000301@gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
	<47A6DAC6.3000301@gmail.com>
Message-ID: <b8c6f6c40802041124p35c0474do419d682783098f61@mail.gmail.com>

What about a formal dependency plan instead?

Python would remain 'batteries included', perhaps verging on 'kitchen
sink included'.

Those users working to embed Python or work in other situations where
a large set of available libraries is a problem would have some help
in cutting down the footprint in an particular installation.  That is,
one could freely delete some set of libraries and be fairly sure the
resulting library set would still run.  What was left would be a
smaller footprint interpreted language, just not 'Python'.

Under what circumstances is this a poor idea?   "Hard drive space is
expensive" is not an acceptable argument.

Charles
---------
Charles Merriam
e:  charles.merriam at gmail.com
u:  http://charlesmerriam.com
q:  "Toolboxes have more tools than just a hammer."


On Feb 4, 2008 1:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Brett Cannon wrote:
> > If you want to trim more then
> > push for stuff to be removed on a module-to-module basis. But going
> > from "batteries included" to "batteries easily downloaded" is quite a
> > shift for Python.
>
> Not to mention it being a serious pain in the rear for those of us who
> have to get licenses vetted by our contracts department before we're
> allowed to copy open source code across to isolated networks with no
> direct access to stuff that is "in the cloud".
>
> A batteries-included standard library is a *huge* boon in that kind of
> environment - one license to vet and one installer to copy across,
> instead of suffering a death of a thousand cuts as we have to retrieve
> different bits and pieces from PyPI (or rewrite our own versions from
> scratch).
>
> We-don't-all-work-in-web-app-development-ly,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/charles.merriam%40gmail.com
>

From ndbecker2 at gmail.com  Mon Feb  4 20:34:31 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 04 Feb 2008 14:34:31 -0500
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
References: <18343.6273.636210.6919@montanaro-dyndns-org.local>
	<ca471dc20802040820p279b4fd3ueb1445c7dea5d900@mail.gmail.com>
Message-ID: <fo7pc8$498$1@ger.gmane.org>

Guido van Rossum wrote:

> On Feb 4, 2008 5:52 AM,  <skip at pobox.com> wrote:
>> This discussion of what to install Python 3.0 as reminded me that we've
>> already encountered this problem before.  As I recall, when Python 2.0
>> was released Red Hat steadfastly refused to ship it with their then
>> current
>> version of Linux.  It wasn't until the next RH product came out that they
>> shipped Python 2.x, and then the executable was called (I think) python2.
>>
>> I no longer remember the details very well, but I wonder if there's
>> a lesson to be learned from that experience as we decide how to install
>> Python 3.0.
> 
> IMO the lesson to be learned is that that was a bad idea -- other
> vendors have not followed suit. It also gave Python a bad name because
> for years, typing "python" on Red Hat systems gave you an ancient
> version.
> 

That was unfortunate.  What they should have done, IMO, is install python as
a link to the latest version.  Where they had tools that had not been
ported, those tools should have explicitly said #!/usr/bin/python1.5.

Would be even nicer if python could say:
require python_version >= x.y
or some such.


From adlaiff6 at gmail.com  Mon Feb  4 22:12:42 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Mon, 4 Feb 2008 16:12:42 -0500
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
In-Reply-To: <fo7pc8$498$1@ger.gmane.org>
References: <18343.6273.636210.6919@montanaro-dyndns-org.local>
	<ca471dc20802040820p279b4fd3ueb1445c7dea5d900@mail.gmail.com>
	<fo7pc8$498$1@ger.gmane.org>
Message-ID: <cc7430500802041312h7ad3506apc7b4809fd9cc6804@mail.gmail.com>

On Feb 4, 2008 2:34 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> That was unfortunate.  What they should have done, IMO, is install python as
> a link to the latest version.  Where they had tools that had not been
> ported, those tools should have explicitly said #!/usr/bin/python1.5.
>
> Would be even nicer if python could say:
> require python_version >= x.y
> or some such.

I already had this suggestion shot down in the "Namespaces are etc."
thread.  The argument goes a bit like "if the os calls the wrong
version of python to compile and run the code, it might not even
compile, in which case the 'require version whatever' line of python
code will never be executed".

--
Cheers,
Leif

From ncoghlan at gmail.com  Mon Feb  4 23:01:18 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 05 Feb 2008 08:01:18 +1000
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
In-Reply-To: <cc7430500802041312h7ad3506apc7b4809fd9cc6804@mail.gmail.com>
References: <18343.6273.636210.6919@montanaro-dyndns-org.local>	<ca471dc20802040820p279b4fd3ueb1445c7dea5d900@mail.gmail.com>	<fo7pc8$498$1@ger.gmane.org>
	<cc7430500802041312h7ad3506apc7b4809fd9cc6804@mail.gmail.com>
Message-ID: <47A78B2E.70706@gmail.com>

Leif Walsh wrote:
> On Feb 4, 2008 2:34 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> Would be even nicer if python could say:
>> require python_version >= x.y
>> or some such.
> 
> I already had this suggestion shot down in the "Namespaces are etc."
> thread.  The argument goes a bit like "if the os calls the wrong
> version of python to compile and run the code, it might not even
> compile, in which case the 'require version whatever' line of python
> code will never be executed".

Not only that, but applications which wish to provide a user-friendly 
error message when run with an incompatible version of the interpreter 
can already do so by checking sys.version_info in a main module that 
steps around the compilation issue by only using very limited constructs 
(more on that in my contribution to the namespaces thread).

This approach has the virtue of being able to give a nice error message 
with versions of the interpreter all the way back to 2.0 (when 
version_info was added) instead of relying on something that would at 
best be added in 2.6.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From stephen at xemacs.org  Tue Feb  5 00:20:00 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 05 Feb 2008 08:20:00 +0900
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
 more of	those!
In-Reply-To: <20080204123057.1004e485@mbook-fbsd>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
Message-ID: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Meyer writes:
 > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
 > > Marcin ___Qrczak___ Kowalczyk writes:
 > >  > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze:
 > >  > > I don't see any need to shorten "python3.0" to "python3".
 > >  > There is a need. Using #!/usr/bin/python3.0 would break as soon as
 > >  > python3.1 is released, while #!/usr/bin/python3 would be fine, at
 > >  > least in the next few years.
 > > First of all, under current policy, installing Python 3.1 would not
 > > uninstall or overwrite Python 3.0
 > 
 > Wrong. The recommend "make install" overwrites the "python" link.

Irrelevant.  That "python" link is *not involved* in the shebang
quoted above, and it is not part of any Python installation, precisely
because it is designed (nowadays) to point to a real installed python
somewhere else.  (What I mean is that even though setup.py does make
one, that doesn't mean that version of Python has any ownership rights
to it.)

 > > so nothing "breaks" when the user uses #!/usr/bin/python3.0.
 > 
 > Unless, of course, the user used their systems packaging software
 > to upgrade the default python install, and that's been moved to the
 > newer version.  In that case, every packaging system I've ever used
 > will remove the old version,

Making that decision is the distro's prerogative, nay, its sacred
duty.  But here, we're discussing what Python should do, not what
distros should do.

 > > On the contrary, Python 3.1 itself may be buggy.  I do not think is
 > > it appropriate for the Python installer to make that decision for
 > > the user by creating a link to the most recent python3.x.
 > 
 > Except it currently does, both during the install from source

Then altinstall should be made the default.

 > *and* as installed by most packaging systems.

Different issue.  That *is* a packaging system's prerogative IMO.

 > > What I see no need for is a policy statement *by Python* that there
 > > "should" be a "python3" link, or which python3.x is should point to.
 > 
 > So do you think we should do away with the "python" link as well?

Yes.  The decision to have one should be made by the user or the
distro.  This doesn't mean that Python shouldn't provide an install
target that makes one, but it should require more typing than the
default install (and thus an explicit decision).

 > The reason OS distributions are hard on "rogue" users who change the
 > meaning of "python" is because their system tools quit working when
 > you do that, as they all use "python".

No, it's a more general policy.  That is an important justification in
the case of python, but the tools generally do blow up if users mess
with other "virtualized" files, too, because the metadata is incorrect.

 > Having the python install provide the major version link as well as
 > the versionless "python" link means it becomes reasonable to
 > distribute scripts that use link, because most installs will have
 > it. It encourages vendors to use that link in their system-specific
 > scripts, so they can provide a backwards-incompatible version of
 > python, and make "python" invoke it for users. Both of these are good
 > things.

Sure, both are good things.  But having setup.py provide them is not.
If the user and/or the distro is providing the link, setup.py should
assume that they know what they are doing.

"How can setup.py know that?" you ask.  Yes, *that is precisely the
problem.*  setup.py doesn't and can't know whether the user, the
distro, or a previous avatar of setup.py created that link.  Odds are
very good that the distro knows far more about what the user wants
than the user does (ie, it knows the dependency graph in detail, and
the user just wants that graph satisfied), but setup.py does not know.
And should not know.  That's the distro's job.

From barry at python.org  Tue Feb  5 00:20:28 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Feb 2008 18:20:28 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of	those!
In-Reply-To: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <A5E4827A-BAE7-4FE1-B5F3-984EEDFE2DFA@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 4, 2008, at 6:20 PM, Stephen J. Turnbull wrote:

>>> On the contrary, Python 3.1 itself may be buggy.  I do not think is
>>> it appropriate for the Python installer to make that decision for
>>> the user by creating a link to the most recent python3.x.
>>
>> Except it currently does, both during the install from source
>
> Then altinstall should be made the default.

I think this is the best solution.   +1

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBR6edvXEjvBPtnXfVAQLaaAQApF4obEK27TTEy71pigvMnNdd4rhd5Vnp
AV+kvyPGaaFY0u5Eq4Q1HZlxyoQ32FEvCKqwpFLl9OQ+Bb16qcLVbXfycAxQFONA
ITUuxYPBDSeyBV8pQnggQkJXEi1ziuSiYk/KgabZi5fOV+kGWoRF+wZSdDbgMS1M
M0miCaXtFog=
=S7kE
-----END PGP SIGNATURE-----

From rwgk at yahoo.com  Tue Feb  5 00:34:56 2008
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Mon, 4 Feb 2008 15:34:56 -0800 (PST)
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
Message-ID: <448576.20652.qm@web31111.mail.mud.yahoo.com>

Nick Coghlan wrote:

> I personally haven't seen anything to convince me that the 2.x -> 3.0 
> upgrade cycle is going to be significantly worse from a deployment point 
> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also 
> possible (e.g. code using 'with' or 'as' as identifiers runs just fine 
> on 2.4 or 2.5, but that code is going to break in 2.6).

The 2.x -> 3.0 transition will be *very* different. We're using Python
since 2000, version 1.5.2. I've always upgraded to the latest release
and got it working with a few fairly minor tweaks. Importantly, I
could do this without breaking backward compatibility if I wanted to.
At some point we had to give up on 1.5.2 compatibility, but that was
just because of Boost.Python requirements (only works with new-style
classes).

IIUC, there is no conservative upgrade path for Python 3.0. Once the
transition is done, the code will only work with 3.0. The bridge to
older Python versions is completely broken. Python 2.x will not even
parse most Python 3 scripts and vice versa. That's a situation we never
had before. Even though the scripts look very similar at face value,
at the parsing level they are as incompatible as e.g. perl and Python.

It is a given that you have to plan for a transition period that is
measured in years: if you have dependencies on 3rd-party packages
(e.g. wxPython) you cannot start using Python 3 until all 3rd-party
packages have gone through the transition. We are using a ton of
3rd-party packages. That's what open source and object-orientation is
all about. The millions of lines of sources that have accumulated over
the years cannot be converted over night. My *optimistic* time horizon
for our project is five years before I can realistically think of
giving up on Python 2, and start *a few weeks of work* to push all
our stuff over to Python 3. During that time, all serious development
work will have to be postponed.

My pessimistic time horizon is "forever". Python 3 has the potential
to become the software version of the Itanium if the deployment isn't
handled carefully. Worse, it could even reflect badly on Python 2
if the naive user just knows that "python script" usually ends in a
syntax error unless he thinks twice about what machine he is using
to run which script.

As Python developers you have to realize that for many people Python is
"just" a core around which they build much bigger applications. If the
success of Python is to continue, you have to give your large user
base a clear path to working with two co-existing Python versions,
so that installing a Python 3 application doesn't break all Python 2
applications (some of which may never be converted since the original
developers have moved on). For example, I really need to be able
to use wxPython based on Python 2 and wxPython based on Python 3
simultaneously on my Windows machine. I'm thinking this is only
possible if Python 3 uses a new file extension.

Ideally, I'd even want to have both Python 2 and Python 3 scripts in
the same directory, while incrementally converting our applications.
I'd want to cp script.py script.py3, then edit the .py3 version
until it works, while still being able to run the old version during
the migration period. Of course, I could also start a new Python 3
specific directory structure, but then I'd have to jump back and
forth between two complex trees all the time, which is very annoying
and time consuming; and I may have to copy many non-Python files that
I wouldn't have to maintain in two copies otherwise.

I think it will be essential to give the Python 3 interpreter a new
name and the scripts a new extension. I suggest using "py3" for both.

Ralf




From stephen at xemacs.org  Tue Feb  5 01:04:47 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 05 Feb 2008 09:04:47 +0900
Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco?
In-Reply-To: <fo7pc8$498$1@ger.gmane.org>
References: <18343.6273.636210.6919@montanaro-dyndns-org.local>
	<ca471dc20802040820p279b4fd3ueb1445c7dea5d900@mail.gmail.com>
	<fo7pc8$498$1@ger.gmane.org>
Message-ID: <87ve54w9nk.fsf@uwakimon.sk.tsukuba.ac.jp>

Neal Becker writes:

 > Would be even nicer if python could say: require python_version >=
 > x.y or some such.

The starting point of this discussion was that Python 3 proves this to
be impossible.  The app cannot be sure at write time that no Python
released in the future will make the app illegal.

The sensible thing for the app to say is "I require <named API>" and
the sensible reply for the python executable is one of "Yo! I can do
that!" or "Take this note over to X.Y, and he'll take care of it for
you" or "sorry, I dunno nobody that can do that for ya."

A way to implement this would be for apps to use #!/usr/bin/pythonX.Y
shebangs, and for Python W.Z to install a link pythonX.Y -> pythonW.Z
if (a) pythonX.Y isn't already a real file and (b) W.Z does provide
the full X.Y API.  Crude, but it can be done *now* *without* support
from the Python language itself.  It would take care of breaks in
backward compatibility nicely.  Eg, 3.0 simply wouldn't install any
python2.X -> python3.0 links.  (Yes, I know users won't retrofit, and
it's even worse for the distros.  But that's true of any scheme like
this.)

Dunno if this can be applied to Windows, unfortunately.

From jimjjewett at gmail.com  Tue Feb  5 01:34:21 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 4 Feb 2008 19:34:21 -0500
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <b8c6f6c40802041124p35c0474do419d682783098f61@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
	<47A6DAC6.3000301@gmail.com>
	<b8c6f6c40802041124p35c0474do419d682783098f61@mail.gmail.com>
Message-ID: <fb6fbf560802041634q4f4d5f6cje60eac1a1309e0a1@mail.gmail.com>

On 2/4/08, Charles Merriam <charles.merriam at gmail.com> wrote:
> What about a formal dependency plan instead?

> Python would remain 'batteries included', perhaps verging on 'kitchen
> sink included'.

> Those users working to embed Python or work in other situations where
> a large set of available libraries is a problem would have some help
> in cutting down the footprint in an particular installation.  That is,
> one could freely delete some set of libraries and be fairly sure the
> resulting library set would still run.  What was left would be a
> smaller footprint interpreted language, just not 'Python'.

> Under what circumstances is this a poor idea?   "Hard drive space is
> expensive" is not an acceptable argument.

(1)   Many library modules *use* other modules, but don't really
*rely* on them.  Only a little functionality would be lost.  That sort
of roadmap would encourage the functionality to get left out entirely,
and would certainly increase the amount of fallback boilerplate.

(2)  In bureaucratic environments, the libraries are OK because they
are standard; if a more minimal distribution becomes equally standard,
that will cause problems.  (It wouldn't be nearly as bad as taking
them out and requiring downloads, but it would still be a problem,
because of the need to justify the non-minimal version.)

(3)  If storage footprint is important, then it is also important for
deployed applications -- and it becomes awkward if you have to
sprinkle your own code with fallbacks and workarounds in case the
standard library was clipped.

That said, it might be useful if it were kept at unofficial status,
such as a wiki page.

-jJ

From amcnabb at mcnabbs.org  Tue Feb  5 04:06:34 2008
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Mon, 4 Feb 2008 20:06:34 -0700
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
Message-ID: <20080205030634.GA2267@mcnabbs.org>

I really like your point about how Python should give a better error
than a SyntaxError when a Python 2 script is run on a Python 3
interpreter.  I agree that the cause of problems should be pointed out
clearly to end users.  However, I disagree that adding a new file
extension is necessary.

On Mon, Feb 04, 2008 at 03:34:56PM -0800, Ralf W. Grosse-Kunstleve wrote:
> 
> If the success of Python is to continue, you have to give your large
> user base a clear path to working with two co-existing Python
> versions, so that installing a Python 3 application doesn't break all
> Python 2 applications (some of which may never be converted since the
> original developers have moved on). For example, I really need to be
> able to use wxPython based on Python 2 and wxPython based on Python 3
> simultaneously on my Windows machine. I'm thinking this is only
> possible if Python 3 uses a new file extension.

In Python 2, "import wxPython" will import wxPython based on Python 2,
and on Python 3, it will import wxPython based on Python 3.  There
wouldn't be a Python 2 version of wxPython in the python3.0
site-packages.


> Ideally, I'd even want to have both Python 2 and Python 3 scripts in
> the same directory, while incrementally converting our applications.
> I'd want to cp script.py script.py3, then edit the .py3 version until
> it works, while still being able to run the old version during the
> migration period.

I kind of like the idea of doing the following:

import sys
major_version = sys.version_info[0]
if major_version == 2:
    import script2
    script2.main()
elif major_version == 3:
    import script3
    script3.main()

It really shouldn't be too bad, I think.


-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20080204/e1a27d8b/attachment.pgp 

From foom at fuhm.net  Tue Feb  5 07:15:42 2008
From: foom at fuhm.net (James Y Knight)
Date: Tue, 5 Feb 2008 01:15:42 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <20080205030634.GA2267@mcnabbs.org>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080205030634.GA2267@mcnabbs.org>
Message-ID: <7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net>

On Feb 4, 2008, at 10:06 PM, Andrew McNabb wrote:

> In Python 2, "import wxPython" will import wxPython based on Python 2,
> and on Python 3, it will import wxPython based on Python 3.  There
> wouldn't be a Python 2 version of wxPython in the python3.0
> site-packages.

Not much fun for users of the PYTHONPATH env var though, eh?

James

From ncoghlan at gmail.com  Tue Feb  5 16:01:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Feb 2008 01:01:05 +1000
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
Message-ID: <47A87A31.8070208@gmail.com>

Ralf W. Grosse-Kunstleve wrote:
> Nick Coghlan wrote:
> 
>> I personally haven't seen anything to convince me that the 2.x -> 3.0 
>> upgrade cycle is going to be significantly worse from a deployment point 
>> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also 
>> possible (e.g. code using 'with' or 'as' as identifiers runs just fine 
>> on 2.4 or 2.5, but that code is going to break in 2.6).
> 
> The 2.x -> 3.0 transition will be *very* different. We're using Python
> since 2000, version 1.5.2. I've always upgraded to the latest release
> and got it working with a few fairly minor tweaks. Importantly, I
> could do this without breaking backward compatibility if I wanted to.
> At some point we had to give up on 1.5.2 compatibility, but that was
> just because of Boost.Python requirements (only works with new-style
> classes).
> 
> IIUC, there is no conservative upgrade path for Python 3.0. Once the
> transition is done, the code will only work with 3.0. The bridge to
> older Python versions is completely broken. Python 2.x will not even
> parse most Python 3 scripts and vice versa. That's a situation we never
> had before.

I still see this as a difference in degree rather than a difference in 
kind. As soon as a developer puts "from __future__ import 
absolute_imports" or "from __future__ import with_statement" in their 
script every version of Python prior to 2.5 is going to refuse to even 
compile it. If they provide an old script which uses an identifier which 
is now a keyword, it is the more recent versions which are going to 
object violently.

Either way, the solution is the same: run that particular script with a 
version of the interpreter that can handle it correctly. Make that 
happen by whatever means your platform makes available (on Windows it 
usually means batch files, on Unix'y systems a version specific shebang 
line or a shell script).

> Even though the scripts look very similar at face value,
> at the parsing level they are as incompatible as e.g. perl and Python.

While there is almost certainly going to be incompatibility at the 
module level, it is perfectly possible to have compatibility at the 
application level. Granted, obtaining it won't be anywhere near as easy 
as it is when staying within the 2.x series, but it is far from a lost 
cause.

Specifically, the application may need a launcher module that fiddles 
with sys.path or package __path__ entries based on the running Python 
version. Alternatively, the application may be installed into 
site-packages and executed using the -m switch so that the interpreter 
automatically picks up the correct version of the application for the 
interpreter which is running.

(Note again that all of these issues are only particularly difficult to 
deal with in the case of applications which are written in Python and 
intended to be used on a system someone else controls. When the same 
entity controls both the application and the runtime environment - such 
as web services, embedded systems, or applications that bundle their own 
Python interpreter - it is merely necessary that the environment is kept 
in sync as the needs of the code change)

> As Python developers you have to realize that for many people Python is
> "just" a core around which they build much bigger applications.

Strange as it may seem, at least some of us do exactly that as part of 
our day jobs ;)

> If the
> success of Python is to continue, you have to give your large user
> base a clear path to working with two co-existing Python versions,
> so that installing a Python 3 application doesn't break all Python 2
> applications (some of which may never be converted since the original
> developers have moved on).

You mean the way you can already have Python 1.5, 2.0, 2.1, 2.2, 2.3, 
2.4 and 2.5 all installed on the same machine, and invoke different 
versions for different applications?

Yes, only one of them can be the 'default Python' for the machine, and 
changing that can be an interesting exercise, but that is hardly a 
problem that is limited to Python - it's the case for *any* scripting 
language interpreter. To choose an example which affected me relatively 
recently, how many shell scripts did the Ubuntu folks break when they 
decided to change their default shell from bash to dash "because it is 
faster" instead of explicitly invoking dash for the particular scripts 
they wanted to speed up? That was one configuration change I reverted 
pretty darn quickly, because there are a hell of a lot of shell scripts 
out there which rely on bash extensions, even if the developers didn't 
realise it when they wrote the scripts.

While I'm +1 on the idea of making altinstall the default for setup.py 
and the Windows installer (forcing the administrator to explicitly 
request that the new Python version be made the default Python 
installation for the machine), I'm a pretty strong -1 on officially 
blessing the idea of a python3 alias or a special py3 extension on 
Windows (end users are obviously free to create those if they want, of 
course).

Who knows, with the rate at which virtual machine technology is 
improving, this whole thing may become a non-issue by the time the 3.x 
series is even ready for production use. One VM with a 2.x default 
Python, another VM with a 3.x default Python, various applications 
associated with each, and seamless integration into the underlying 
platform so the user never even notices a VM is involved.

(OK, I don't actually hold out much hope of that particular scenario 
coming to pass in the next few years, but I'm trying to point out here 
that you're pushing for a language level solution to a system level 
problem: how do we reliably associate files with an application that 
understands them? What do we do when different versions of that 
application permit the files to contain different things? Yes, these 
issues matter for the 2.x->3.x transition, but moving to a new 
interpreter name and a new file extension is nothing more than a 
bandaid, and a pretty ugly one that we would have to live with for a 
long time)

> For example, I really need to be able
> to use wxPython based on Python 2 and wxPython based on Python 3
> simultaneously on my Windows machine. I'm thinking this is only
> possible if Python 3 uses a new file extension.

And how much trouble do you have using wxPython with 2.4 and 2.5 on the 
same machine? They're completely different extension modules, and they 
can live side by side on one machine quite happily (it's one of the main 
benefits of installing things to site-packages). That behaviour won't 
change just because one of those Python version numbers now has a 3 at 
the start.

Making heavy use of PYTHONPATH can cause problems, but again that's the 
result of a platform limitation, where it isn't easy to get different 
applications to see different values for environment variables without 
the use of launch scripts.

> Ideally, I'd even want to have both Python 2 and Python 3 scripts in
> the same directory, while incrementally converting our applications.
> I'd want to cp script.py script.py3, then edit the .py3 version
> until it works, while still being able to run the old version during
> the migration period.

Sure, you can do it that way if you really want to, but that isn't the 
recommended way.

And invoking Python 3 for the scripts that need it would be the same as 
invoking any other specific Python version (batch file or shortcut on 
Windows, version-specific shebang line or shell script on POSIX).

> Of course, I could also start a new Python 3
> specific directory structure, but then I'd have to jump back and
> forth between two complex trees all the time, which is very annoying
> and time consuming; and I may have to copy many non-Python files that
> I wouldn't have to maintain in two copies otherwise.

The intent is that you should never need to edit files directly in both 
trees - instead, it should be possible to work almost entirely in the 
Python 2 tree, with the 2to3 tool automatically creating the Python 3 
tree for you (kind of like a cross-compilation step that accepts valid 
Python 2 code and emits the corresponding Python 3 code). Any test 
failures (or other problems) in the Python 3 version would be fixed by 
modifying the Python 2 source and reconverting.

Having an option to ask the 2to3 tool to automatically copy any 
non-Python files it finds might be an interesting idea, although using 
an existing file synchronisation tool like rsync or a version control 
branch (and then running 2to3 to make modifications after the other 
files are in sync) would probably be a better one.

> I think it will be essential to give the Python 3 interpreter a new
> name and the scripts a new extension. I suggest using "py3" for both.

And I think that would be an ugly hack that we would then be stuck with 
for the life of the 3.x series.

Would the renamed python3 still be looking at all the same environment 
variables (PYTHONPATH, PYTHONHOME, etc)? Or would all those have to be 
renamed as well? Should import change to accept .py3 files? If not, why 
not (after all, the line between execution and import gets pretty blurry 
where the -m switch and the runpy module are concerned)? If yes, should 
import be changed to not accept .py files? When python3 was asked to 
execute a directory or zipfile, would it look for __main__.py3, 
__main__.py or either?

Also note that while it currently will be feasible to write quite 
substantial programs that run on both Python 2 and Python 3 without 
modification (albeit only with a total avoidance of the print statement 
and function, as well as taking a great deal of care in remaining 
str/unicode agnostic), some of the possible answers to the questions 
posed in the previous paragraph would make it downright impossible.

The mechanisms that have been used to date to support multiple Python 
versions on a single system have worked pretty well, and I don't believe 
that they're suddenly going to start failing horribly just because the 
permitted degree of backward incompatibility between 2.x and 3.0 is 
higher than that between 2.x and 2.(x+2).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Tue Feb  5 14:37:34 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 5 Feb 2008 07:37:34 -0600
Subject: [Python-3000] PYTHONPATH in Python 3.0 (was: Namespaces are one
	honking ...)
In-Reply-To: <7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080205030634.GA2267@mcnabbs.org>
	<7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net>
Message-ID: <18344.26270.730092.20238@montanaro-dyndns-org.local>


    James> Not much fun for users of the PYTHONPATH env var though, eh?

PYTHONPATH3 anyone?  Or maybe .pth files are sufficient.  So far I've been
able to mangle my PYTHONSTARTUP file to work with both Python 2 and Python
3.  That's not very sophisticated code though.

Skip

From mwm at mired.org  Mon Feb  4 18:30:57 2008
From: mwm at mired.org (Mike Meyer)
Date: Mon, 4 Feb 2008 12:30:57 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
 more of	those!
In-Reply-To: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080204123057.1004e485@mbook-fbsd>

On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Marcin ___Qrczak___ Kowalczyk writes:
>  > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze:
>  > > I don't see any need to shorten "python3.0" to "python3".
>  > There is a need. Using #!/usr/bin/python3.0 would break as soon as
>  > python3.1 is released, while #!/usr/bin/python3 would be fine, at
>  > least in the next few years.
> First of all, under current policy, installing Python 3.1 would not
> uninstall or overwrite Python 3.0

Wrong. The recommend "make install" overwrites the "python" link.

> so nothing "breaks" when the user uses #!/usr/bin/python3.0.

Unless, of course, the user used their systems packaging software to
upgrade the default python install, and that's been moved to the newer
version. In that case, every packaging system I've ever used will
remove the old version, thus breaking any scripts that were foolish
enough to use that form.

Which is why those packaging systems pretty much all install python
scripts using "python" for the executable.

> On the contrary, Python 3.1 itself may be buggy.  I do not think is
> it appropriate for the Python installer to make that decision for
> the user by creating a link to the most recent python3.x.

Except it currently does, both during the install from source *and* as
installed by most packaging systems. If the packaging system does
things right, this isn't really a problem - until you get to 3.0,
because it breaks backwards compatibility.

> What I see no need for is a policy statement *by Python* that there
> "should" be a "python3" link, or which python3.x is should point to.

So do you think we should do away with the "python" link as well?

> I don't think that Python providing a simple tool to help the user do
> it is a good idea, either, because most OS distributions already
> provide them, and they tend to be rather obnoxious about "rogue" users
> who do it by hand or use 3d party (including upstream) tools to do it.

The reason OS distributions are hard on "rogue" users who change the
meaning of "python" is because their system tools quit working when
you do that, as they all use "python". So when they start shipping
python3, they'll tend to make "python" invoke the old version until
they convert all their scrips, as RH did with 1 and 2. Except RH still
has a python2 alias, so they can easily switch their scripts to
python2 if they chose to do so. Personally, I'd rather they do the
latter than the former, so people will get a modern version of python
when they ask for python on that OS.

Most of the time, having a default for both python and python of a
specific major version is pointless, because you're only going to have
one major version installed, even if you have multiple minor versions
installed, so they're both the same. The exception is when a new major
version arrives breaking backwards compatibility. At that time, it's
nice to have a default for the major versions so scripts can pick the
major version if they're not picky about minor versions (and given how
much work we do making sure the language is backwards compatible at
that level, there are a lot of those) at the same time as users pick
what they want to get when they say "python".

Having the python install provide the major version link as well as
the versionless "python" link means it becomes reasonable to
distribute scripts that use link, because most installs will have
it. It encourages vendors to use that link in their system-specific
scripts, so they can provide a backwards-incompatible version of
python, and make "python" invoke it for users. Both of these are good
things.

	<mike

-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Tue Feb  5 00:38:07 2008
From: mwm at mired.org (Mike Meyer)
Date: Mon, 4 Feb 2008 18:38:07 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of	those!
In-Reply-To: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080204183807.412b14d8@bhuda.mired.org>

On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Mike Meyer writes:
>  > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
>  > > Marcin ___Qrczak___ Kowalczyk writes:
>  > >  > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze:
>  > >  > > I don't see any need to shorten "python3.0" to "python3".
>  > >  > There is a need. Using #!/usr/bin/python3.0 would break as soon as
>  > >  > python3.1 is released, while #!/usr/bin/python3 would be fine, at
>  > >  > least in the next few years.
>  > > First of all, under current policy, installing Python 3.1 would not
>  > > uninstall or overwrite Python 3.0
>  > Wrong. The recommend "make install" overwrites the "python" link.
> Irrelevant.  That "python" link is *not involved* in the shebang
> quoted above, and it is not part of any Python installation, precisely

The python link is part of the python install on every system I've
dealt with - including from the source tarballs - as they all install
it, and the vast majority of software that needs python to run expect
to find it there.

>  > > so nothing "breaks" when the user uses #!/usr/bin/python3.0.
>  > Unless, of course, the user used their systems packaging software
> Making that decision is the distro's prerogative, nay, its sacred
> duty.  But here, we're discussing what Python should do, not what
> distros should do.

[...]

>  > *and* as installed by most packaging systems.
> Different issue.  That *is* a packaging system's prerogative IMO.

Right. And the packaging system is what most people deal with in the
real world. Ignoring the real world is a good way to become
irrelevant. And while we can't control them (GNU/Linux packagers have
a really nasty habit of installing software configured in ways the
author doesn't want to support), we can at least nudge them in the
right direction by making it easy to do things in what we consider to
be the best way.

>  > > What I see no need for is a policy statement *by Python* that there
>  > > "should" be a "python3" link, or which python3.x is should point to.
>  > So do you think we should do away with the "python" link as well?
> Yes.  The decision to have one should be made by the user or the
> distro.  This doesn't mean that Python shouldn't provide an install
> target that makes one, but it should require more typing than the
> default install (and thus an explicit decision).

So you want the default behavior after installing python to be that
the "python" command not be found? So that people who see software
that "needs python" will grab and install python, and then have the
software not work because they it no longer installs that link by
default? Or do you expect everyone who installs python software to fix
the executables to refer to the version of python they have installed?
Or maybe just install every version some software author was using
when they wrote the software?

>  > Having the python install provide the major version link as well as
>  > the versionless "python" link means it becomes reasonable to
>  > distribute scripts that use link, because most installs will have
>  > it. It encourages vendors to use that link in their system-specific
>  > scripts, so they can provide a backwards-incompatible version of
>  > python, and make "python" invoke it for users. Both of these are good
>  > things.
> Sure, both are good things.  But having setup.py provide them is not.
> If the user and/or the distro is providing the link, setup.py should
> assume that they know what they are doing.

If the users are using a packaging system, then what setup.py does -
or does not do - is largely irrelevant. Users will get what the
packagers want them to have, and there's not jack that the people
actually writing the software can do about it.

If the users get what setup.py provides, then they built from source,
so they at least know how to install and deal with a build
environment, and it's reasonable to assume that they may know a little
bit more than that.

We can either determine a good way to deal with the issues this
raises, and nudge package builders and people doing installs from
source in that direction by making those things the default, or at
least easy; or we can do what was done in the 1->2 transition and let
them all work it out for themselves, and risk once again having some
popular distro on which "python" is obsolete for the next N years.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Tue Feb  5 00:56:42 2008
From: mwm at mired.org (Mike Meyer)
Date: Mon, 4 Feb 2008 18:56:42 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
Message-ID: <20080204185642.73659254@bhuda.mired.org>

On Mon, 4 Feb 2008 15:34:56 -0800 (PST) "Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com> wrote:
> As Python developers you have to realize that for many people Python is
> "just" a core around which they build much bigger applications. If the
> success of Python is to continue, you have to give your large user
> base a clear path to working with two co-existing Python versions,
> so that installing a Python 3 application doesn't break all Python 2
> applications (some of which may never be converted since the original
> developers have moved on). For example, I really need to be able
> to use wxPython based on Python 2 and wxPython based on Python 3
> simultaneously on my Windows machine. I'm thinking this is only
> possible if Python 3 uses a new file extension.

I think the key word here is "Windows". I have no trouble doing all of
this on a Unix system without changing the extensions, but it's a lot
less dependent on file extensions.

> Ideally, I'd even want to have both Python 2 and Python 3 scripts in
> the same directory, while incrementally converting our applications.
> I'd want to cp script.py script.py3, then edit the .py3 version
> until it works, while still being able to run the old version during
> the migration period. Of course, I could also start a new Python 3
> specific directory structure, but then I'd have to jump back and
> forth between two complex trees all the time, which is very annoying
> and time consuming; and I may have to copy many non-Python files that
> I wouldn't have to maintain in two copies otherwise.

Now, let me throw in my requirements. Most modern packaging systems
can handle the upgrade from 2.x to 2.y; they'll automatically upgrade
all the packages that depend on python, meaning you'll get versions
installed for 2.y after the upgrade process is done, and anything that
uses only packages from the packaging system will keep on working.

I don't want the solution for 3.0 to break the ability to do 2.x
upgrades that way. Further, I want the same solution to work for 3.0
to 3.1.

> I think it will be essential to give the Python 3 interpreter a new
> name and the scripts a new extension. I suggest using "py3" for both.

I really, really hate this idea. But if it's the only workable
solution on Windows, then it's the only workable solution.

Question: how many of the editor packages for editing python files
will work properly on both python and python 3.0 files?

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Tue Feb  5 17:22:32 2008
From: mwm at mired.org (Mike Meyer)
Date: Tue, 5 Feb 2008 11:22:32 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A87A31.8070208@gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<47A87A31.8070208@gmail.com>
Message-ID: <20080205112232.34bc5c80@mbook-fbsd>

On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:

> Ralf W. Grosse-Kunstleve wrote:
> > Nick Coghlan wrote:
> > 
> >> I personally haven't seen anything to convince me that the 2.x -> 3.0 
> >> upgrade cycle is going to be significantly worse from a deployment point 
> >> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also 
> >> possible (e.g. code using 'with' or 'as' as identifiers runs just fine 
> >> on 2.4 or 2.5, but that code is going to break in 2.6).
> > 
> > The 2.x -> 3.0 transition will be *very* different. We're using Python
> > since 2000, version 1.5.2. I've always upgraded to the latest release
> > and got it working with a few fairly minor tweaks. Importantly, I
> > could do this without breaking backward compatibility if I wanted to.
> > At some point we had to give up on 1.5.2 compatibility, but that was
> > just because of Boost.Python requirements (only works with new-style
> > classes).
> > 
> > IIUC, there is no conservative upgrade path for Python 3.0. Once the
> > transition is done, the code will only work with 3.0. The bridge to
> > older Python versions is completely broken. Python 2.x will not even
> > parse most Python 3 scripts and vice versa. That's a situation we never
> > had before.
> 
> I still see this as a difference in degree rather than a difference in 
> kind. As soon as a developer puts "from __future__ import 
> absolute_imports" or "from __future__ import with_statement" in their 
> script every version of Python prior to 2.5 is going to refuse to even 
> compile it. If they provide an old script which uses an identifier which 
> is now a keyword, it is the more recent versions which are going to 
> object violently.

The one real difference is that you can generally go from 2.X to 2.Y>X
without mucking with your scripts so long as you update all the
modules you use as well. In an environment managed with package
systems, this is usually easy.

> Either way, the solution is the same: run that particular script with a 
> version of the interpreter that can handle it correctly. Make that 
> happen by whatever means your platform makes available (on Windows it 
> usually means batch files, on Unix'y systems a version specific shebang 
> line or a shell script).

Version-specific shebangs break the above property. It's possible to
have multiple versions installed and have everything work without
going to that. But your system vendor probably won't support it :-(.

> (Note again that all of these issues are only particularly difficult to 
> deal with in the case of applications which are written in Python and 
> intended to be used on a system someone else controls. When the same 
> entity controls both the application and the runtime environment - such 
> as web services, embedded systems, or applications that bundle their own 
> Python interpreter - it is merely necessary that the environment is kept 
> in sync as the needs of the code change)

Right. I live in the last world, because I've found that it's nearly
impossible to get "someone else" to provide adequate python builds.

> > As Python developers you have to realize that for many people Python is
> > "just" a core around which they build much bigger applications.
> 
> Strange as it may seem, at least some of us do exactly that as part of 
> our day jobs ;)

Yup.

> > If the
> > success of Python is to continue, you have to give your large user
> > base a clear path to working with two co-existing Python versions,
> > so that installing a Python 3 application doesn't break all Python 2
> > applications (some of which may never be converted since the original
> > developers have moved on).
> 
> You mean the way you can already have Python 1.5, 2.0, 2.1, 2.2, 2.3, 
> 2.4 and 2.5 all installed on the same machine, and invoke different 
> versions for different applications?
> 
> Yes, only one of them can be the 'default Python' for the machine,

Actually, that's not *quite* true.

One of the reasons you can install Python from source on most systems
- even using the default install - without breaking all the tools that
depend on the system's anointed version is because it install
/usr/local/bin/python. A system required version almost always lives
in /usr/bin, and is invoked with that path. Even systems that don't
have Python in the default install generally put packages somewhere
other than /usr/local (FreeBSD is the one exception I know of).

For one client, we had /usr/bin/python, a 64-bit 2.3 python;
/oap/bin/python, a 32-bit 2.4 python without ssl support (from the IT
support group, because they wanted the same language "everywhere"),
and /home/<application>/bin/python, a 64-bit 2.4 python with the ssl
support we required. The scripts all invoked the python they needed by
absolute path. The biggest problem we had was getting the developers
environment set up so they invoked the right python at the command
line for testing.

> language interpreter. To choose an example which affected me relatively 
> recently, how many shell scripts did the Ubuntu folks break when they 
> decided to change their default shell from bash to dash "because it is 
> faster" instead of explicitly invoking dash for the particular scripts 
> they wanted to speed up? That was one configuration change I reverted 
> pretty darn quickly, because there are a hell of a lot of shell scripts 
> out there which rely on bash extensions, even if the developers didn't 
> realise it when they wrote the scripts.

And those of use who use systems where bash *isn't* the default shell,
and never has been, wish all those developers would get a clue, and
write "#!/usr/bin/env bash" (or at least "#!/bin/bash") instead of
"#!/bin/sh", and reserve the latter for scripts that really are
*shell* scripts. Sorry, that's a sore spot...

> While I'm +1 on the idea of making altinstall the default for setup.py 
> and the Windows installer (forcing the administrator to explicitly 
> request that the new Python version be made the default Python 
> installation for the machine), I'm a pretty strong -1 on officially 
> blessing the idea of a python3 alias or a special py3 extension on 
> Windows (end users are obviously free to create those if they want, of 
> course).

I like the python3 alias (and python2 in 2.X), because it moves us
from a single default python to having a default python2 and python3,
which means scripts that care can ask for that without breaking the
ability to upgrade python2 or python3 without breaking those scripts,
while still allowing the default python to be either python2 or
python3.

> Who knows, with the rate at which virtual machine technology is 
> improving, this whole thing may become a non-issue by the time the 3.x 
> series is even ready for production use. One VM with a 2.x default 
> Python, another VM with a 3.x default Python, various applications 
> associated with each, and seamless integration into the underlying 
> platform so the user never even notices a VM is involved.

That just changes the environment from one where you have to find the
right python to one where you have to find the right VM. But you don't
have to go to that extreme; just keeping the different pythons
separated by directory, and making sure the scripts specify the right
directory as part of the shebang works (at lest for Unix).

	  <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From rhamph at gmail.com  Tue Feb  5 18:43:40 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 5 Feb 2008 10:43:40 -0700
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <20080204185642.73659254@bhuda.mired.org>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
Message-ID: <aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>

On Feb 4, 2008 4:56 PM, Mike Meyer <mwm at mired.org> wrote:
> On Mon, 4 Feb 2008 15:34:56 -0800 (PST) "Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com> wrote:
> > As Python developers you have to realize that for many people Python is
> > "just" a core around which they build much bigger applications. If the
> > success of Python is to continue, you have to give your large user
> > base a clear path to working with two co-existing Python versions,
> > so that installing a Python 3 application doesn't break all Python 2
> > applications (some of which may never be converted since the original
> > developers have moved on). For example, I really need to be able
> > to use wxPython based on Python 2 and wxPython based on Python 3
> > simultaneously on my Windows machine. I'm thinking this is only
> > possible if Python 3 uses a new file extension.
>
> I think the key word here is "Windows". I have no trouble doing all of
> this on a Unix system without changing the extensions, but it's a lot
> less dependent on file extensions.

So why don't we add a windows equivalent of the shebang?  Files could
then start like this:

#!/usr/bin/python2.3
#?C:/python23/python

Of course, something better than ? needs to be chosen.

A problem with this approach is that, if you only had 2.3 and 2.5
installed, both it'd be completely ignored.  You'd need to install a
newer version (2.6 or 3.0) to force the use of 2.3.


-- 
Adam Olsen, aka Rhamphoryncus

From pje at telecommunity.com  Tue Feb  5 18:57:58 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 05 Feb 2008 12:57:58 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.co
 m>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
Message-ID: <20080205175802.3FBF93A409E@sparrow.telecommunity.com>

At 10:43 AM 2/5/2008 -0700, Adam Olsen wrote:
>So why don't we add a windows equivalent of the shebang?  Files could
>then start like this:
>
>#!/usr/bin/python2.3
>#?C:/python23/python

FYI, setuptools uses and supports #! lines on Windows, with the 
executable path in quotes if it contains spaces.  For a given script 
"foo", it generates two files:

     foo.exe
     foo-script.py

And the .exe is just a standardized .exe file that looks in an 
adjacent -script.py file of the same name for the #! line.

(This is of course only for scripts generated using 
setuptools-specific features.)


>A problem with this approach is that, if you only had 2.3 and 2.5
>installed, both it'd be completely ignored.  You'd need to install a
>newer version (2.6 or 3.0) to force the use of 2.3.

Setuptools works with versions 2.3 through 2.5, but of course that's 
because of the .exe wrappers.


From rhamph at gmail.com  Tue Feb  5 19:13:11 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 5 Feb 2008 11:13:11 -0700
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
Message-ID: <aac2c7cb0802051013y6b389da8w8af07beb9b117b0d@mail.gmail.com>

On Feb 5, 2008 10:57 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:43 AM 2/5/2008 -0700, Adam Olsen wrote:
> >So why don't we add a windows equivalent of the shebang?  Files could
> >then start like this:
> >
> >#!/usr/bin/python2.3
> >#?C:/python23/python
>
> FYI, setuptools uses and supports #! lines on Windows, with the
> executable path in quotes if it contains spaces.  For a given script
> "foo", it generates two files:
>
>      foo.exe
>      foo-script.py
>
> And the .exe is just a standardized .exe file that looks in an
> adjacent -script.py file of the same name for the #! line.
>
> (This is of course only for scripts generated using
> setuptools-specific features.)
>
>
> >A problem with this approach is that, if you only had 2.3 and 2.5
> >installed, both it'd be completely ignored.  You'd need to install a
> >newer version (2.6 or 3.0) to force the use of 2.3.
>
> Setuptools works with versions 2.3 through 2.5, but of course that's
> because of the .exe wrappers.

Sounds good enough to me.  Maybe the only thing "wrong" is people
aren't sufficiently informed of how to be version-specific?


-- 
Adam Olsen, aka Rhamphoryncus

From fumanchu at aminus.org  Tue Feb  5 19:34:22 2008
From: fumanchu at aminus.org (Robert Brewer)
Date: Tue, 5 Feb 2008 10:34:22 -0800
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of those!
Message-ID: <F1962646D3B64642B7C9A06068EE1E6402075467@ex10.hostedexchange.local>

I took the liberty of asking DRH's advice based on his experiences with
sqlite/2/3, and he graciously replied:

> -----Original Message-----
> From: drh at hwaci.com [mailto:drh at hwaci.com]
> Sent: Tuesday, February 05, 2008 10:02 AM
> To: Robert Brewer
> Subject: Re: Python 3 needs your help
> 
> "Robert Brewer" <fumanchu at aminus.org> wrote:
> > Hello,
> >
> > There has been some discussion [1] lately on whether to rename the
> > "python" interpreter to "python3" for Python 3.0. I immediately
> > thought of "sqlite3" of course, and wondered if you had any advice
> > for the Python developers about the benefits or hardships of your
> > naming choices for sqlite. If you had just a couple minutes to add
> > your experience to the discussion, it'd be a big help.
> 
> The primary reason from going from "sqlite" to "sqlite3" was to
> allow the same application to link against both libraries at the
> same time.  This was important during the transition period since
> some programs needed to be able to read an SQLite v2 database then
> write the content over to an SQLite v3 database as part of the
> upgrade process.  Even today, three years into version 3, there
> still exist important programs (ex: PHP) which link against both
> libraries by default.
> 
> I do not recall encountering any problems with the "sqlite3" name.
> Nobody has complained.  And we have not had any negative experiences.
> Everything has worked out well.
> 
> If you are referring to the name of the SQLite CLI program as opposed
> to the library, it is convenient to have a separate name for the
> version 3 CLI since we often need to both the older version 2 and
> new version 3 programs at the same time.  For example, to upgrade a
> legacy database:
> 
>     sqlite olddatabase .dump | sqlite3 newdatbase
> 
> There are still many sqlite version 2 databases around, and so it
> is important to be able to have both older "sqlite" and newer
"sqlite3"
> binaries in your PATH.  I can imagine that it will similarly be
> convenient to have both "python" and "python3" in your PATH at the
> same time.
> 
> I think calling it "python3" is probably a good idea.
> 
> --
> D. Richard Hipp <drh at hwaci.com>


From lists at cheimes.de  Tue Feb  5 21:04:25 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 05 Feb 2008 21:04:25 +0100
Subject: [Python-3000] Free list for small longs
Message-ID: <47A8C149.1020407@cheimes.de>

I've implemented a free list for small long objects with a size of 1 or
-1. I wanted to test how large the malloc overhead is. The result was
astonishing. The free list quadrupled the speed of a simple test:

$ ./python -m timeit "for i in range(100): list(range(1000))"

Without patch:
10 loops, best of 3: 79 msec per loop

With patch:
10 loops, best of 3: 20.8 msec per loop

Since the free list is limited to small longs, it will consume less than
1,5 MB on a 64bit OS and less than 900kb on a 32bit OS in a worst case
scenario (2 * (1<<15) ~ 65k objects with a size of 14 / 22 byte each).

http://bugs.python.org/issue2013

Christian

From skip at pobox.com  Tue Feb  5 21:13:29 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 5 Feb 2008 14:13:29 -0600
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47A8C149.1020407@cheimes.de>
References: <47A8C149.1020407@cheimes.de>
Message-ID: <18344.50025.175774.601181@montanaro-dyndns-org.local>


    Christian> I've implemented a free list for small long objects with a
    Christian> size of 1 or -1.

I'm not sure what you mean by "size of 1 or -1".  Do you mean you only keep
the numbers 1 and -1 on the free list?  It's not obvious to me what a size
of -1 is.  Do you mean positive and negative numbers which fit in one byte
or one long word?

Skip

From brett at python.org  Tue Feb  5 21:23:07 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Feb 2008 12:23:07 -0800
Subject: [Python-3000] Free list for small longs
In-Reply-To: <18344.50025.175774.601181@montanaro-dyndns-org.local>
References: <47A8C149.1020407@cheimes.de>
	<18344.50025.175774.601181@montanaro-dyndns-org.local>
Message-ID: <bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>

On Feb 5, 2008 12:13 PM,  <skip at pobox.com> wrote:
>
>     Christian> I've implemented a free list for small long objects with a
>     Christian> size of 1 or -1.
>
> I'm not sure what you mean by "size of 1 or -1".  Do you mean you only keep
> the numbers 1 and -1 on the free list?  It's not obvious to me what a size
> of -1 is.  Do you mean positive and negative numbers which fit in one byte
> or one long word?

I think Christian means single digit integers. But I thought we
already did this for positive numbers? So is the proposal to also
cover negative numbers?

And obviously having a single loop that explicitly tests the
optimization is not exactly indicative of real-world use. =)

-Brett

From lists at cheimes.de  Tue Feb  5 21:26:44 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 05 Feb 2008 21:26:44 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <18344.50025.175774.601181@montanaro-dyndns-org.local>
References: <47A8C149.1020407@cheimes.de>
	<18344.50025.175774.601181@montanaro-dyndns-org.local>
Message-ID: <47A8C684.20002@cheimes.de>

skip at pobox.com wrote:
> I'm not sure what you mean by "size of 1 or -1".  Do you mean you only keep
> the numbers 1 and -1 on the free list?  It's not obvious to me what a size
> of -1 is.  Do you mean positive and negative numbers which fit in one byte
> or one long word?

I should have explained how longs use the object size. The absolute
value of Python long is stored is an array of digits (unsigned short).
For Negative values Python longs set the ob_size to a negative value.

>From Include/longintpr.h:

/* Long integer representation.
   The absolute value of a number is equal to
        SUM(for i=0 through abs(ob_size)-1) ob_digit[i] * 2**(SHIFT*i)
   Negative numbers are represented with ob_size < 0;
   zero is represented by ob_size == 0.
   In a normalized number, ob_digit[abs(ob_size)-1] (the most significant
   digit) is never zero.  Also, in all cases, for all valid i,
        0 <= ob_digit[i] <= MASK.
   The allocation function takes care of allocating extra memory
   so that ob_digit[0] ... ob_digit[abs(ob_size)-1] are actually available.

   CAUTION:  Generic code manipulating subtypes of PyVarObject has to
   aware that longs abuse  ob_size's sign bit.
*/

Christian

From lists at cheimes.de  Tue Feb  5 21:31:17 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 05 Feb 2008 21:31:17 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>
References: <47A8C149.1020407@cheimes.de>	
	<18344.50025.175774.601181@montanaro-dyndns-org.local>
	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>
Message-ID: <47A8C795.10001@cheimes.de>

Brett Cannon wrote:
> I think Christian means single digit integers. But I thought we
> already did this for positive numbers? So is the proposal to also
> cover negative numbers?

Ne, we don't use a free list for longs. In the 2.x series floats and
ints are allocated in blocks of 1000 objects. The 3.x series uses the
approach for floats only. Longs are always allocated with a free list
unless the value is in between -5 and +256. These longs are statically
allocated using an array.

> And obviously having a single loop that explicitly tests the
> optimization is not exactly indicative of real-world use. =)

Damn, you got me  ;)

Christian



From martin at v.loewis.de  Tue Feb  5 21:44:50 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Feb 2008 21:44:50 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47A8C149.1020407@cheimes.de>
References: <47A8C149.1020407@cheimes.de>
Message-ID: <47A8CAC2.2070609@v.loewis.de>

> Since the free list is limited to small longs, it will consume less than
> 1,5 MB on a 64bit OS and less than 900kb on a 32bit OS in a worst case
> scenario (2 * (1<<15) ~ 65k objects with a size of 14 / 22 byte each).

I'm puzzled as to how you arrive at this upper bound. You can surely
have more than 2**16 integer objects whose abs values are all
below 2**15, no?

smallints=[]
for i in range(2**20):
   smallints.append(i % 30000)

Regards,
Martin

From lists at cheimes.de  Tue Feb  5 22:47:11 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 05 Feb 2008 22:47:11 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47A8CAC2.2070609@v.loewis.de>
References: <47A8C149.1020407@cheimes.de> <47A8CAC2.2070609@v.loewis.de>
Message-ID: <foalh0$60e$1@ger.gmane.org>

Martin v. L?wis wrote:
> I'm puzzled as to how you arrive at this upper bound. You can surely
> have more than 2**16 integer objects whose abs values are all
> below 2**15, no?

Oh ... I see. You are right and I did a mistake by confusing the amount
of objects with the range. I'd better add an upper bound check.


From jcea at argo.es  Tue Feb  5 22:58:20 2008
From: jcea at argo.es (Jesus Cea)
Date: Tue, 05 Feb 2008 22:58:20 +0100
Subject: [Python-3000] pickle, cPickle,
 and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>	<acd65fa20801121633l2ae5202j8a7d75d7933ce91e@mail.gmail.com>	<EFB69A57-3547-4F53-AFA0-871ED571764F@zope.com>	<acd65fa20801201939m191b7d95s7b4ae7dae617ada4@mail.gmail.com>	<A862A59F-7C37-4E09-A9FF-D5B8C32965B3@zope.com>	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>	<20080125045906.GA21722@panix.com>	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
Message-ID: <47A8DBFC.5090704@argo.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
| On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote:
|> (*) I can only speculate that it's because Jim, whose ZODB is probably
|> pickle's most intensive user,
|
| Maybe, but I'm not sure.

Any persistence system is pickle intensive. Aside ZODB, we have Durus
also. Do not forget remote call systems, like Pyro or most instances of
parallel python attempts.

pickle/cpickle performance is important.

In fact, I find "disturbing" that "bencode" (from BitTorrent), done 100%
in python, is faster that cPickle. Yes, "bencode" can't encode arbitrary
objects, but still...

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at argo.es http://www.argo.es/~jcea/ _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
~                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBR6jb+5lgi5GaxT1NAQLhnAP8COAFON7zALBX+4peFKH6LwlxA65IptqW
Xg40Z3UZUo5bi67vkCvI/SuNLYpcvez2S5LTBrrlCHTWu4DSdI7/K9KPzQ8rbIdn
EfCwuvSWOAKJ/mvTFA1LP0RixNd/+MTllFq+9TbOIe8EYZ/UbNLj4StqQBNGydjY
KAmntN4VBnE=
=C+TR
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Tue Feb  5 23:30:14 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Feb 2008 23:30:14 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>
	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>
Message-ID: <47A8E376.8000308@v.loewis.de>

> I think Christian means single digit integers. But I thought we
> already did this for positive numbers? So is the proposal to also
> cover negative numbers?

We don't have a free list, but a cache for them, and only for numbers
between -5 and +256. Christian proposes a freelist, similar to the
special allocate in 2.x.

Regards,
Martin

From charles.merriam at gmail.com  Tue Feb  5 23:39:36 2008
From: charles.merriam at gmail.com (Charles Merriam)
Date: Tue, 5 Feb 2008 14:39:36 -0800
Subject: [Python-3000] pickle, cPickle,
	and the standard library (was Re: [Python-Dev] inst_persistent_id)
In-Reply-To: <fb6fbf560802041634q4f4d5f6cje60eac1a1309e0a1@mail.gmail.com>
References: <A2A6D220-45A7-4C04-8D4E-CBBB919D55E1@zope.com>
	<acd65fa20801231330r3fc24b24sbad1c62579a42a29@mail.gmail.com>
	<19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com>
	<20080125045906.GA21722@panix.com>
	<ca471dc20801242107x262bdcb0y9990df77a61a7a39@mail.gmail.com>
	<2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com>
	<bbaeab100802031433r186e8b21td7598cace019b7@mail.gmail.com>
	<47A6DAC6.3000301@gmail.com>
	<b8c6f6c40802041124p35c0474do419d682783098f61@mail.gmail.com>
	<fb6fbf560802041634q4f4d5f6cje60eac1a1309e0a1@mail.gmail.com>
Message-ID: <b8c6f6c40802051439o2d4b87cm33f0db72e234a4d0@mail.gmail.com>

I agree.  A wiki page for "leaving just the head of the snake" would
be the correct solution.

On Feb 4, 2008 4:34 PM, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 2/4/08, Charles Merriam <charles.merriam at gmail.com> wrote:
> > What about a formal dependency plan instead?
>
> > Python would remain 'batteries included', perhaps verging on 'kitchen
> > sink included'.
>
> > Those users working to embed Python or work in other situations where
> > a large set of available libraries is a problem would have some help
> > in cutting down the footprint in an particular installation.  That is,
> > one could freely delete some set of libraries and be fairly sure the
> > resulting library set would still run.  What was left would be a
> > smaller footprint interpreted language, just not 'Python'.
>
> > Under what circumstances is this a poor idea?   "Hard drive space is
> > expensive" is not an acceptable argument.
>
> (1)   Many library modules *use* other modules, but don't really
> *rely* on them.  Only a little functionality would be lost.  That sort
> of roadmap would encourage the functionality to get left out entirely,
> and would certainly increase the amount of fallback boilerplate.
>
> (2)  In bureaucratic environments, the libraries are OK because they
> are standard; if a more minimal distribution becomes equally standard,
> that will cause problems.  (It wouldn't be nearly as bad as taking
> them out and requiring downloads, but it would still be a problem,
> because of the need to justify the non-minimal version.)
>
> (3)  If storage footprint is important, then it is also important for
> deployed applications -- and it becomes awkward if you have to
> sprinkle your own code with fallbacks and workarounds in case the
> standard library was clipped.
>
> That said, it might be useful if it were kept at unofficial status,
> such as a wiki page.
>
> -jJ
>

From brett at python.org  Wed Feb  6 00:05:29 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Feb 2008 15:05:29 -0800
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47A8E376.8000308@v.loewis.de>
References: <47A8C149.1020407@cheimes.de>
	<18344.50025.175774.601181@montanaro-dyndns-org.local>
	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>
	<47A8E376.8000308@v.loewis.de>
Message-ID: <bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>

On Feb 5, 2008 2:30 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I think Christian means single digit integers. But I thought we
> > already did this for positive numbers? So is the proposal to also
> > cover negative numbers?
>
> We don't have a free list, but a cache for them, and only for numbers
> between -5 and +256. Christian proposes a freelist, similar to the
> special allocate in 2.x.

Gotcha.

Well, I am a little leery of another free list. I personally don't
love the fact that there are various caches and free lists laying
about in the code without a central place to turn them off or clear
them (or at least list them in a comment somewhere). And the problem
with the free lists that the caches don't have is that they assume
everyone gets a performance perk from them when there is a chance no
one will ever touch a thing in the free lists.

-Brett

From lists at cheimes.de  Wed Feb  6 00:51:20 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 06 Feb 2008 00:51:20 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>
	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>
Message-ID: <foaspo$tbc$1@ger.gmane.org>

Brett Cannon wrote:
> Well, I am a little leery of another free list. I personally don't
> love the fact that there are various caches and free lists laying
> about in the code without a central place to turn them off or clear
> them (or at least list them in a comment somewhere). And the problem
> with the free lists that the caches don't have is that they assume
> everyone gets a performance perk from them when there is a chance no
> one will ever touch a thing in the free lists.

I understand your objection. Most free lists and caches are limited but
some are not. I'm worried about the float and int blocks. Python's
memory management never frees an int or float block. Code like

>>> x = [float(i) for i in list(range(1000000))]
>>> del x

creates a million float and int objects but *never* frees the memory
until Python exists. It's a pathologic case but it shows a problem. I've
started to implement an API to compact the free lists,
PyNAME_CompactFreelist() and sys._compact_freelists(). My new
sys._compact_freelists() deallocates the float and int blocks which do
not contain a referenced object.

Most free lists are limited except the int (2.x only ) and float blocks.
The overview may not be complete and 2.x may have more free lists.

bytesobject
-----------
no free list, only a single nullbyte

classobject
-----------
unlimited *free_list of PyMethodObject

dictobject
----------
*free_dicts[] limited to 80 entries

frameobject
-----------
a zombie frame on each code object
limited *free_list (linked list) with 200 entries max

floatobject
-----------
unlimited blocks of allocated floats with 1000 entries each
The blocks are a single linked list and the free floats are
linked by ob_type (ob_type points to the next free float).

intobject (2.x)
---------------
same as floatobject

listobject
----------
*free_lists[] limited to 80 entries

longobject
----------
cache for small number between -5 and +256
my patch for a limited free list

methodobject
------------
unlimited *free_list of PyCFunctionObject

setobject
---------
*free_sets[] limited to 80 entries

stringobject
------------
cache for nullstring and strings with one entry
interned strings (2.x only)

tupleobject
-----------
20 free lists for tuples with up to 20 entries. Each free list is
limited to 2000 entries.

unicodeobject
-------------
*unicode_freelist limited to 1024 entries


Christian


From stephen at xemacs.org  Wed Feb  6 01:32:59 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 06 Feb 2008 09:32:59 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking
	great	idea -- let's do more of those!
In-Reply-To: <aac2c7cb0802051013y6b389da8w8af07beb9b117b0d@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<aac2c7cb0802051013y6b389da8w8af07beb9b117b0d@mail.gmail.com>
Message-ID: <87sl06udok.fsf@uwakimon.sk.tsukuba.ac.jp>

Adam Olsen writes:

 > > Setuptools [pybang] works with versions 2.3 through 2.5, but of
 > > course that's because of the .exe wrappers.

As independent corroboration, this is basically the same way that
DJGPP (the DOS-extended version of GCC) provides Unix-y features like
access to the environment variables and command line parsing
(including redirection and pipes).  Of course since DJGPP produces
.exes, all it requires to use the feature is an #include.

 > Sounds good enough to me.  Maybe the only thing "wrong" is people
 > aren't sufficiently informed of how to be version-specific?

I think it's worse than that.  People don't *want* to be version-
specific (rather, they feel it is being "imposed" on them), and they
generally believe that their particular feature dependencies are quite
general and deserve conservation across version changes.

Some, like the "field heavyweight" quoted by the OP, are refreshingly
pragmatic about it.  They're quite happy to use a language that is
pretty crappy for most purposes today considered practical because it
does a great job of continuing to run the programs written to address
the "practical purposes" of three decades ago (if I interpret the "76"
in "SHELX-76" correctly).

Others, like the OP, want to freeze Python and request that updated
versions be considered an internal fork in the project rather than
evolutionary[1] progress whenever their inconvenience tolerance (which
is clearly high in the OP's case, let's not belittle that!) is
exceeded.

Footnotes: 
[1]  Cf. "punctuated equilibrium" for my notion of "evolution."


From stephen at xemacs.org  Wed Feb  6 02:32:53 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 06 Feb 2008 10:32:53 +0900
Subject: [Python-3000] Namespaces are one honking great idea --
	let's	do	more of	those!
In-Reply-To: <20080204183807.412b14d8@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
Message-ID: <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Meyer writes:
 > On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
 > > Mike Meyer writes:
 > >  > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
 > >  > Wrong. The recommend "make install" overwrites the "python" link.
 > > Irrelevant.  That "python" link is *not involved* in the shebang
 > > quoted above, and it is not part of any Python installation, precisely
 > 
 > The python link is part of the python install on every system I've
 > dealt with - including from the source tarballs - as they all install
 > it, and the vast majority of software that needs python to run expect
 > to find it there.

And your point might be?  Did you snip the part where I explained that
that is irrelevant to my point before reading it?  If not, please
explain; I've already explained (better expressed as "setup.py
installdefault", see below) how I propose that practical issue be
dealt with both sanely and compatibly.

 > Right. And the packaging system is what most people deal with in the
 > real world. Ignoring the real world is a good way to become
 > irrelevant. And while we can't control them (GNU/Linux packagers have
 > a really nasty habit of installing software configured in ways the
 > author doesn't want to support), we can at least nudge them in the
 > right direction by making it easy to do things in what we consider to
 > be the best way.

GNUbies *will* ignore you.  "GNU maintainers" do not care, by
Stallman's definition, about the "right way" for any system but the
GNU System.  Unfortunately, there is no such thing as The GNU System,
so each packager feels free to define what's right for the GNU System
in terms of their own practices.  This is paradigmatic for why
"python3" is a nonsolution.

To paraphrase the good Samuel Clemens, I think the reports of
impending Python irrelvancy are vastly overblown.  As my friend Tom
Lord puts it, the vast majority of software doesn't exist yet.  The
battleground for relevancy lies in the future.

Finally, I have been arguing that the right way is precisely to leave
it up to the packagers because they have the knowledge and the
technology to deal with dependencies.  Python (the project) does not.
As Nick Coghlan put it, "you're pushing for a language level solution
to a system level problem: how do we reliably associate files with an
application that understands them?"

 > So you want the default behavior after installing python to be that
 > the "python" command not be found?

Maybe.  It wouldn't be hard to detect that there is no "python" on
$PATH and finish either the configuration stage or the install stage
with the warning,

    **** There is no 'python' on $PATH. ****
    If you want this version of Python to become your default "python",
    run "setup.py installdefault" now.
    (You may need Administrator or root privileges to do so.)

 > > Sure, both are good things.  But having setup.py provide them is not.
 > > If the user and/or the distro is providing the link, setup.py should
 > > assume that they know what they are doing.
 > 
 > If the users are using a packaging system, then what setup.py does -
 > or does not do - is largely irrelevant. Users will get what the
 > packagers want them to have, and there's not jack that the people
 > actually writing the software can do about it.

No.  If the user is root installing Python as support for system
software, then what setup.py does can mess up the packaging system,
and this is the most likely result of having "python" and/or "python2"
and/or "python3" links made by default.

OTOH, if they're installing into a non-system location, the text above
is more than sufficient warning to those with the ability and courage
to successfully run "setup.py install" from tarballs.

 > If the users get what setup.py provides, then they built from source,
 > so they at least know how to install and deal with a build
 > environment, and it's reasonable to assume that they may know a little
 > bit more than that.

That's exactly what I just wrote.  I don't understand why you draw the
conclusion you do, that a "python3" link somehow becomes a good idea
(rather than a crock pandering to lazy developers downstream).

 > We can either determine a good way to deal with the issues this
 > raises,

Nick's point, and my point, is that all the good ways require either
downstream developer discipline in use of shebangs that are adequate,
or system-level support.

 > and nudge package builders and people doing installs from
 > source in that direction by making those things the default, or at
 > least easy;

Although we disagree on what the semantics should be, what could be
easier than "setup.py installdefault", and still retain some semblance
of providing minimal opportunity for exercise of responsibility by the
sysadmin?

 > or we can do what was done in the 1->2 transition and let
 > them all work it out for themselves,

The whole problem is that the RightThang[tm] has nothing to do with
Python package builders and people who install Python from source.
The problem is entirely downstream from Python: it is that app
developers target a specific feature set but are unwilling to make the
miniscule effort to announce it with a properly specific shebang.

If in fact the Python 2.x for x in 0, ..., 6 are nearly always
backward compatible, then a better way (but still not perfect) for the
system or user to solve the problem is for the system to create a
"python2.y" alias for python2.x, for y in 0, ..., x-1.  If you detect
a bug (eg, the one that came in with the introduction of boolean
True), then you can simply override the alias with an installation of
the needed version.  *This doesn't work with "python2" shebangs.*  You
must also fix the app's shebangs.  If you upgrade the app, you must
remember to fix the shebangs again.  ("Remember" includes maintaining
local and vendor branches in a SCM of course.)

 > and risk once again having some popular distro on which "python" is
 > obsolete for the next N years.

Huh?  Python 1.5.2 was then, and remains now, a better language for my
purposes than any version of awk, FORTRAN, or Perl I've ever used. :-)

I only wrote about 10 Python scripts on Red Hat systems, but none of
them barfed because "python" was "obsolete."  Red Hat had *lots* of
other problems, which is why I abandoned it.  Having to type
"python2.2" occasionally was the least of my complaints about RHL
(which I have been pronouncing "are hell" since the last millennium: I
was shocked, SHOCKED, I tell you, when they chose to make that
pronunciation the official name of their flagship certified product!)


From rwgk at yahoo.com  Wed Feb  6 04:53:25 2008
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 5 Feb 2008 19:53:25 -0800 (PST)
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
Message-ID: <865955.18383.qm@web31106.mail.mud.yahoo.com>

Stephen J. Turnbull wrote:

> Some, 
like 
the 
"field 
heavyweight" 
quoted 
by 
the 
OP, 
are 
refreshingly
> pragmatic 
about 
it.  
They're 
quite 
happy 
to 
use 
a 
language 
that 
is
> pretty 
crappy 
for 
most 
purposes 
today 
considered 
practical 
because 
it
> does 
a 
great 
job 
of 
continuing 
to 
run 
the 
programs 
written 
to 
address
> the 
"practical 
purposes" 
of 
three 
decades 
ago 
(if 
I 
interpret 
the 
"76"
> in 
"SHELX-76" 
correctly).

He is one of the brightest people you'll find on this planet. His set
of programs still is a de-facto standard, used to solve the vast majority
of (small molecule) crystal structures. To know what that means,
consider that without his work, you wouldn't be sitting in front of
*that* computer. It would be a different computer in a different world.

> Others, 
like 
the 
OP, 
want 
to 
freeze 
Python 
and 
request 
that 
updated
> versions 
be 
considered 
an 
internal 
fork 
in 
the 
project 
rather 
than
> evolutionary[1] 
progress 
whenever 
their 
inconvenience 
tolerance 
(which
> is 
clearly 
high 
in 
the 
OP's 
case, 
let's 
not 
belittle 
that!) 
is
> exceeded.

It remains to be seen if your idea of "evolution" will still be
remembered in 30 years. From all I've seen, arbitrarily introducing
hardships in the name of progress doesn't work out in practice.
Look around. C++ is dirty but got big because it never seriously broke
with the C heritage. Microsoft is the biggest software company in the world
even though the OS is dirty, but my DOS Turbo C compiler from 1990
probably still runs on a Windows XP system. Apple's market share
in the OS market is still tiny in comparison, I'd argue to a significant
degree because they were constantly "innovating".

I'm not at all convinced Python 3 will succeed, although I'm hoping
for it. Introducing a break like that without at the same time
introducing new technologies is in stark violation of
"practicality beats purity."

My personal battle is to not see my own work die a slow death because
everybody in my field thinks I must be crazy to base my work on something
that is one day this and the next day something incompatible. If you
give it a new name at least, it will be much easier for me to explain
and stand my ground. Also, look at the sqlite experience:

http://mail.python.org/pipermail/python-3000/2008-February/011991.html

You really need to be able to have both the old and the new in the same
environment indefinitely, and it has to be easily predictable what you get
when you run "python".

Ralf




From stephen at xemacs.org  Wed Feb  6 07:39:50 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 06 Feb 2008 15:39:50 +0900
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <865955.18383.qm@web31106.mail.mud.yahoo.com>
References: <865955.18383.qm@web31106.mail.mud.yahoo.com>
Message-ID: <87lk5ytwp5.fsf@uwakimon.sk.tsukuba.ac.jp>

Ralf W. Grosse-Kunstleve writes:

 > Stephen J. Turnbull wrote:

[really butchered quotes omitted]

 > [Prof. George Sheldrick] is one of the brightest people you'll find
 > on this planet.

Ad hominem argument.  My point is simply that he knows what he needs
and said so.  He doesn't need a language able to evolve with the
times.  Good for him.

 > It remains to be seen if your idea of "evolution" will still be
 > remembered in 30 years.

It will, if only because it's not mine, but rather Stephen Jay
Gould's.

 > From all I've seen, arbitrarily introducing hardships in the name
 > of progress doesn't work out in practice.

Who's introducing hardships?  The hardships are inherent in progress
in a language.  All I've argued is that the way you want to deal with
these hardships is fundamentally broken in the face of a rapidly
evolving language.  The right way is for the app to declare the API it
needs, and for the system to point to a version that provides that
API.

A quick and dirty but basically right solution is available for
systems that understand the shebang: put the fully-versioned name in
the shebang, and make links from the "best" version available to all
versions it can support.

 > My personal battle is to not see my own work die a slow death because
 > everybody in my field thinks I must be crazy to base my work on something
 > that is one day this and the next day something incompatible.

If so, I feel for you.  Nevertheless, even if they do,
that's not Python's problem to solve.

 > If you give it a new name at least, it will be much easier for me
 > to explain and stand my ground.

What's wrong with "Python 2.4" (or whatever)?  It's not like the
vendor is going to go out of business or strategically stop
distributing it.  It's not like all the programmers familiar with
Python 2 idioms are going to get their brains wiped.  It's not like
all the familiar modules will suddenly stop working with Python 2.
It's not like the community will suddenly refuse to answer your
questions because you're working with Python 2.

And it's not like Python 2 will stop being a language far more suited
to rapid evolution of your application than FORTRAN.

 > You really need to be able to have both the old and the new in the
 > same environment indefinitely, and it has to be easily predictable
 > what you get when you run "python".

If you think you depend on predictability of "what you get when you
run 'python'", I hope you're wrong, because it's not dependable unless
you "own" the OS distribution that your users run on and all of their
other applications, too.


From ncoghlan at gmail.com  Wed Feb  6 12:09:11 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Feb 2008 21:09:11 +1000
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <20080205112232.34bc5c80@mbook-fbsd>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<47A87A31.8070208@gmail.com>
	<20080205112232.34bc5c80@mbook-fbsd>
Message-ID: <47A99557.2000303@gmail.com>

Mike Meyer wrote:
> On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:
>> While I'm +1 on the idea of making altinstall the default for setup.py 
>> and the Windows installer (forcing the administrator to explicitly 
>> request that the new Python version be made the default Python 
>> installation for the machine), I'm a pretty strong -1 on officially 
>> blessing the idea of a python3 alias or a special py3 extension on 
>> Windows (end users are obviously free to create those if they want, of 
>> course).
> 
> I like the python3 alias (and python2 in 2.X), because it moves us
> from a single default python to having a default python2 and python3,
> which means scripts that care can ask for that without breaking the
> ability to upgrade python2 or python3 without breaking those scripts,
> while still allowing the default python to be either python2 or
> python3.

The main objection I have is that the executable name is only part of 
the story - all of the PYTHON* environment variables may matter as well 
(there are 9 listed in python -h, and I would have to check the code to 
say with complete confidence that there aren't any others).

That said, while as a matter of purity I still think this is a platform 
problem (one which all current platforms I know of handle poorly ;), as 
a matter of practicality I'm seeing some benefits allowing a machine to 
have full fledged "Python 2" and "Python 3" setups side by side.

If Guido does decide to go the route of changing the interpreter binary 
to python3 (ala sqlite vs sqlite3), I would also suggest that we go the 
whole way and insert a '3' into all of the environment variable names 
that python3 looks for (PYTHON3PATH, PYTHON3STARTUP, etc). I'd even 
concede the utility of accepting the py3 extension on modules as a 
concession to extension based dispatching on Windows.

I still think it would be a bit of an ugly hack, but if it was followed 
through to the point of being able to define separate Python 2 & 3 
environments without the use of launch scripts (i.e. by changing the 
environment variable names as described above), maybe it would still be 
a worthwhile thing to do.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mwm at mired.org  Wed Feb  6 07:24:35 2008
From: mwm at mired.org (Mike Meyer)
Date: Wed, 6 Feb 2008 01:24:35 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of	those!
In-Reply-To: <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080206012435.44bd22b7@bhuda.mired.org>

On Wed, 06 Feb 2008 10:32:53 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Mike Meyer writes:
>  > On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
>  > > Mike Meyer writes:
>  > >  > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
>  > >  > Wrong. The recommend "make install" overwrites the "python" link.
>  > > Irrelevant.  That "python" link is *not involved* in the shebang
>  > > quoted above, and it is not part of any Python installation, precisely
>  > The python link is part of the python install on every system I've
>  > dealt with - including from the source tarballs - as they all install
>  > it, and the vast majority of software that needs python to run expect
>  > to find it there.
> And your point might be?  Did you snip the part where I explained that
> that is irrelevant to my point before reading it?  If not, please

It's not irrelevant, because you're basic assumption is wrong. Or
maybe you're just engaged in wishful thinking. The default python
install includes the python link, and overwrites, much as you might
wish it otherwise.

> To paraphrase the good Samuel Clemens, I think the reports of
> impending Python irrelvancy are vastly overblown. 

So do I. That doesn't mean the situation can't be improved.

> Finally, I have been arguing that the right way is precisely to leave
> it up to the packagers because they have the knowledge and the
> technology to deal with dependencies.

You mean the people you described this way:

> GNUbies *will* ignore you.  "GNU maintainers" do not care, by
> Stallman's definition, about the "right way" for any system but the
> GNU System.  Unfortunately, there is no such thing as The GNU System,
> so each packager feels free to define what's right for the GNU System
> in terms of their own practices.

Right - they all do it however they feel like, and there's not much we
can do about it. That's not *quite* the same as nothing we can do
about it.

> Python (the project) does not.

Correct. On the other hand, the collective of people here certainly
have more experience at dealing with multiple python installations on
one system than any other group in the world. And you suggest that the
advice from that collective experience that setup.py should offer
packagers on how to deal with this situation is - none?

> As Nick Coghlan put it, "you're pushing for a language level solution
> to a system level problem: how do we reliably associate files with an
> application that understands them?"

I'm not suggesting any changes to the language. Like you, I'm
suggesting a change to the default install. I happen to believe it
will make most peoples lives easier. 

> This is paradigmatic for why "python3" is a nonsolution.

Just because it doesn't solve your solutions doesn't mean it doesn't
solve anyones. And it's sure as hell better than doing nothing.

>  > So you want the default behavior after installing python to be that
>  > the "python" command not be found?
> Maybe.  It wouldn't be hard to detect that there is no "python" on
> $PATH and finish either the configuration stage or the install stage
> with the warning,
>     **** There is no 'python' on $PATH. ****
>     If you want this version of Python to become your default "python",
>     run "setup.py installdefault" now.
>     (You may need Administrator or root privileges to do so.)

Yup. So what's the problem with doing the same thing for
python<MAJORVERSIONNUMBER> at the same time?

Personally, I'd rather it just check the install directory and not
$PATH, because you pretty much have to do that if you need different
installs of the same X.Y version. I'd also favor more automation, and
have it offer to run that script rather than just suggest it.

>  > > Sure, both are good things.  But having setup.py provide them is not.
>  > > If the user and/or the distro is providing the link, setup.py should
>  > > assume that they know what they are doing.
>  > If the users are using a packaging system, then what setup.py does -
>  > or does not do - is largely irrelevant. Users will get what the
>  > packagers want them to have, and there's not jack that the people
>  > actually writing the software can do about it.
> No.  If the user is root installing Python as support for system
> software, then what setup.py does can mess up the packaging system,
> and this is the most likely result of having "python" and/or "python2"
> and/or "python3" links made by default.

Sure, there are *lots* of ways the user can screw up the system as
root. Doing a default python install from source is *not* one of them,
no matter what it does with those symlinks. If you know of a system
where things are otherwise, please warn me about it now.

>  > If the users get what setup.py provides, then they built from source,
>  > so they at least know how to install and deal with a build
>  > environment, and it's reasonable to assume that they may know a little
>  > bit more than that.
> That's exactly what I just wrote.  I don't understand why you draw the
> conclusion you do, that a "python3" link somehow becomes a good idea

I draw that conclusion based on a decade and a half of installing,
maintaining and upgrading python software.

> (rather than a crock pandering to lazy developers downstream).

Damn straight I'm lazy. Computers - all of them - suck bad enough as
it is. Anything I can do to make my life easier is a win.

>  > We can either determine a good way to deal with the issues this
>  > raises,
> Nick's point, and my point, is that all the good ways require either
> downstream developer discipline in use of shebangs that are adequate,
> or system-level support.

True. But you act like making it *easy* for people is somehow
criminal.

>  > and nudge package builders and people doing installs from
>  > source in that direction by making those things the default, or at
>  > least easy;
> Although we disagree on what the semantics should be, what could be
> easier than "setup.py installdefault", and still retain some semblance
> of providing minimal opportunity for exercise of responsibility by the
> sysadmin?

I don't really have a problem with that; I just think we should be
looking beyond the end of our noses - uh, 3.0 - and trying to set
things up to make things easier for the system administrator.

>  > or we can do what was done in the 1->2 transition and let
>  > them all work it out for themselves,
> The whole problem is that the RightThang[tm] has nothing to do with
> Python package builders and people who install Python from source.
> The problem is entirely downstream from Python: it is that app
> developers target a specific feature set but are unwilling to make the
> miniscule effort to announce it with a properly specific shebang.

But of course they don't. Your "properly specific shebang" will break
on every version of python *after* the one they target, even though
the script would otherwise work. No developer in their right mind
wants to deal with the extra support work that would generate.

> If in fact the Python 2.x for x in 0, ..., 6 are nearly always
> backward compatible, then a better way (but still not perfect) for the
> system or user to solve the problem is for the system to create a
> "python2.y" alias for python2.x, for y in 0, ..., x-1. 

Personally, I think *this* is a crock. But if you're going to advocate
that people lie about what versions they have installed, then please
advocate having setup.py do something to encourage people to create
systems that look like this, so that app developers have a reason to
use the shebang line you want them to.

> If you detect a bug (eg, the one that came in with the introduction
> of boolean True), then you can simply override the alias with an
> installation of the needed version.  *This doesn't work with
> "python2" shebangs.*

Right. It's not identical, so you have to do something *different* to
solve this problem. With python2, you set the python2 link back to the
previous version. Yeah, you may have to fix the apps that require the
new version, but oddly enough, the more of those scripts there are,
the less likely it is that one of your old apps will barf on that new
version.

Personally, I think that's a *lot* easier than running around creating
or re-creating a bunch of symlinks every time I install or upgrade
python, much less trying to figure out when I really can declare that
I no longer need some version under those conditions. On the other
hand, if setup.py made setting up the symlinks easy, maybe I'd change
my mind.

>  > and risk once again having some popular distro on which "python" is
>  > obsolete for the next N years.
> Huh?  Python 1.5.2 was then, and remains now, a better language for my
> purposes than any version of awk, FORTRAN, or Perl I've ever used. :-)
> I only wrote about 10 Python scripts on Red Hat systems, but none of
> them barfed because "python" was "obsolete."  Red Hat had *lots* of
> other problems, which is why I abandoned it.  Having to type
> "python2.2" occasionally was the least of my complaints about RHL
> (which I have been pronouncing "are hell" since the last millennium: I
> was shocked, SHOCKED, I tell you, when they chose to make that
> pronunciation the official name of their flagship certified product!)

Much as RHEL may work to remind you that GNU means GNUs Not Unix, it's
still one of the most popular distributions around, and hence liable
to be many people's first exposure to python in a Unix-oid
environment. The more we can do to make that part of their experience
not suck, the more likely they are to consider doing more work in
Python. Not to mention making life better for those of us who have
clients that won't let us include the depenguinator in our list of
required software packages.

	 <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From nate at binkert.org  Wed Feb  6 18:40:25 2008
From: nate at binkert.org (nathan binkert)
Date: Wed, 6 Feb 2008 09:40:25 -0800
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of those!
In-Reply-To: <20080206012435.44bd22b7@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012435.44bd22b7@bhuda.mired.org>
Message-ID: <217accd40802060940t627548e2i2b2a32b0a7b42656@mail.gmail.com>

This is very similar to the #! idea and is really just an extension.

Python 2 clearly cannot execute python 3 code, but how hard would it
be to require something like a
__pyversion__ = 3 statement at the top of the file.  If the line does
not exist, the python3 binary would dlopen libpython2.x and execute
the code.  I understand that the syntax of the two versions are not
compatible, but couldn't the tokenizers be compatible?  Imagine the
python3 binary tokenizing the code, searching for the __pyversion__
line and if it does not exist, doing the dlopen and passing the
tokenized code to the python2.x binary (with some potential
translation to fix any token incompatibility).  if the libpython2.x.so
doesn't exist, then a VersionError could be raised.  If the initial
file does have a __pyversion__ and subsequent files do, then a
VersionError exception could be raised.  This solution could help deal
with code that requires new features.  For example, say I want to use
a with clause, I could require 2.6 or 3.0.

Sorry to prolong an already long conversation.

  Nate

From lists at cheimes.de  Wed Feb  6 19:37:47 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 06 Feb 2008 19:37:47 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<20080204185642.73659254@bhuda.mired.org>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.co m>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
Message-ID: <47A9FE7B.7090405@cheimes.de>

Phillip J. Eby wrote:
> FYI, setuptools uses and supports #! lines on Windows, with the 
> executable path in quotes if it contains spaces.  For a given script 
> "foo", it generates two files:
> 
>      foo.exe
>      foo-script.py
> 
> And the .exe is just a standardized .exe file that looks in an 
> adjacent -script.py file of the same name for the #! line.

Nice, it's an elegant and easy solution to the problem. I like to get a
quite similar solution into the core but I propose a slightly different
path.

Like your launcher.c, spam.exe looks for spam.py. It additionally looks
for spam.pyw first giving .pyw files a higher priority.

The first line of the file is read and parsed. The line must start with
a shebang (#!). Contrary to your script it doesn't look for a complete
path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5"
both work.

Then the launcher executable queries the installation path of Python X.Y
from the registry (first HKCU, then HKLM). At last the script is
executed with the Python binary: python.exe for .py and pythonw.exe for
.pyw. The working directory is the absolute path of spam.exe. This way
we neither have to change the paths for Windows scripts nor quote the
path name.

Comments?

Christian

From lists at cheimes.de  Wed Feb  6 19:38:49 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 06 Feb 2008 19:38:49 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<20080204185642.73659254@bhuda.mired.org>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.co m>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
Message-ID: <47A9FEB9.70701@cheimes.de>

Phillip J. Eby wrote:
> FYI, setuptools uses and supports #! lines on Windows, with the 
> executable path in quotes if it contains spaces.  For a given script 
> "foo", it generates two files:
> 
>      foo.exe
>      foo-script.py
> 
> And the .exe is just a standardized .exe file that looks in an 
> adjacent -script.py file of the same name for the #! line.

Nice, it's an elegant and easy solution to the problem. I like to get a
quite similar solution into the core but I propose a slightly different
path.

Like your launcher.c, spam.exe looks for spam.py. It additionally looks
for spam.pyw first giving .pyw files a higher priority.

The first line of the file is read and parsed. The line must start with
a shebang (#!). Contrary to your script it doesn't look for a complete
path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5"
both work.

Then the launcher executable queries the installation path of Python X.Y
from the registry (first HKCU, then HKLM). At last the script is
executed with the Python binary: python.exe for .py and pythonw.exe for
.pyw. The working directory is the absolute path of spam.exe. This way
we neither have to change the paths for Windows scripts nor quote the
path name.

Comments?

Christian


From martin at v.loewis.de  Wed Feb  6 20:20:13 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Feb 2008 20:20:13 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <foaspo$tbc$1@ger.gmane.org>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>
	<foaspo$tbc$1@ger.gmane.org>
Message-ID: <47AA086D.60904@v.loewis.de>

> My new
> sys._compact_freelists() deallocates the float and int blocks which do
> not contain a referenced object.

I wonder whether such a thing should run as a side effect of a full 
garbage collection, also. GC has a chance to free up arenas for good,
and allocated-but-unused blocks on arenas prevent Python from returning
them to the OS.

Regards,
Martin

From theller at ctypes.org  Wed Feb  6 21:04:16 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 06 Feb 2008 21:04:16 +0100
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A9FE7B.7090405@cheimes.de>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<20080204185642.73659254@bhuda.mired.org>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.co
	m>	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
Message-ID: <fod3s0$lm8$1@ger.gmane.org>

Christian Heimes schrieb:
> Phillip J. Eby wrote:
>> FYI, setuptools uses and supports #! lines on Windows, with the 
>> executable path in quotes if it contains spaces.  For a given script 
>> "foo", it generates two files:
>> 
>>      foo.exe
>>      foo-script.py
>> 
>> And the .exe is just a standardized .exe file that looks in an 
>> adjacent -script.py file of the same name for the #! line.
> 
> Nice, it's an elegant and easy solution to the problem. I like to get a
> quite similar solution into the core but I propose a slightly different
> path.
> 
> Like your launcher.c, spam.exe looks for spam.py. It additionally looks
> for spam.pyw first giving .pyw files a higher priority.

Why give a higher priority to .pyw if the .py also exists?  Shouldn't there
be a way to select one or the other, maybe with a command line flag or whatever?

And, while we're at it:  What IMO needs to be fixed is that there is no way
for a .pyw file to show exceptions to the user.  Maybe the console window
is not such a bad idea.  Maybe it could be opened on demand, as soon as there
is something printed to stderr?
 
> The first line of the file is read and parsed. The line must start with
> a shebang (#!). Contrary to your script it doesn't look for a complete
> path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5"
> both work.

You mean you parse this string to get the version number?
 
> Then the launcher executable queries the installation path of Python X.Y
> from the registry (first HKCU, then HKLM). At last the script is
> executed with the Python binary: python.exe for .py and pythonw.exe for
> .pyw. The working directory is the absolute path of spam.exe. This way
> we neither have to change the paths for Windows scripts nor quote the
> path name.

What if there are no registry entries (svn checkout)?

Thomas


From pje at telecommunity.com  Wed Feb  6 21:28:25 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 06 Feb 2008 15:28:25 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A9FEB9.70701@cheimes.de>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.co m>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FEB9.70701@cheimes.de>
Message-ID: <20080206202817.2950F3A40AF@sparrow.telecommunity.com>

At 07:38 PM 2/6/2008 +0100, Christian Heimes wrote:
>The first line of the file is read and parsed. The line must start with
>a shebang (#!). Contrary to your script it doesn't look for a complete
>path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5"
>both work.
>
>Then the launcher executable queries the installation path of Python X.Y
>from the registry (first HKCU, then HKLM). At last the script is
>executed with the Python binary: python.exe for .py and pythonw.exe for
>.pyw. The working directory is the absolute path of spam.exe. This way
>we neither have to change the paths for Windows scripts nor quote the
>path name.
>
>Comments?

I intentionally didn't do it that way because it won't work right for 
Python applications that bundle their own not-in-the-registry Python. 


From lists at cheimes.de  Wed Feb  6 22:35:28 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 06 Feb 2008 22:35:28 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47AA086D.60904@v.loewis.de>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>	<foaspo$tbc$1@ger.gmane.org>
	<47AA086D.60904@v.loewis.de>
Message-ID: <fod971$eq5$1@ger.gmane.org>

Martin v. L?wis wrote:
> I wonder whether such a thing should run as a side effect of a full 
> garbage collection, also. GC has a chance to free up arenas for good,
> and allocated-but-unused blocks on arenas prevent Python from returning
> them to the OS.

Under which condition does a full garbage collection take place? Does
the interpreter run a full gc once in a while or is it started by
gc.collect()?

I'm more concerned about the int and float memory blocks. They should
get freed when more than a couple of hundred blocks are allocated. The
other free lists are much smaller. But memory fragmentation is an issue
for them, too.

Christian


From python at rcn.com  Wed Feb  6 23:29:39 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  6 Feb 2008 17:29:39 -0500 (EST)
Subject: [Python-3000] ABC method mismatch
Message-ID: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>

The tuple class is registered to the collections.Sequence ABC, but it does not support count() and index() as required by the ABC.

Do you want to drop those methods from the ABC or add them to tuple?


Raymond

From guido at python.org  Wed Feb  6 23:40:39 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Feb 2008 14:40:39 -0800
Subject: [Python-3000] ABC method mismatch
In-Reply-To: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>

On Feb 6, 2008 2:29 PM, Raymond Hettinger <python at rcn.com> wrote:
> The tuple class is registered to the collections.Sequence ABC, but it does not support count() and index() as required by the ABC.

Good catch!

> Do you want to drop those methods from the ABC or add them to tuple?

I'm not sure, but I'm tempted to add them to the ABC. What's your
hunch? Do others have an opinion? They shouldn't be a burden for
implementers who use real inheritance from Sequence, as they are
concrete methods there. And it doesn't make sense to move them to
MutableSequence, because there's nothing in them that requires the
sequence to be mutable.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Feb  6 23:55:36 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  6 Feb 2008 17:55:36 -0500 (EST)
Subject: [Python-3000] ABC method mismatch
Message-ID: <20080206175536.AGL69941@ms19.lnh.mail.rcn.net>

>> Do you want to drop those methods from the ABC or add them to tuple?

> I'm not sure, but I'm tempted to add them to the ABC.
> What's your hunch?

My hunch is that adding count() and index() to tuple is at
harmless at worst and sometimes helpful at best (I've
occasionally needed them and felt mild annoyance at having
coerce tuples to lists just to get them).

Also, if you add them now, you won't spend the rest of your
life turning down requests to add them as people periodically
discover the mismatch with strings and lists.


Raymond

From fdrake at acm.org  Wed Feb  6 23:58:24 2008
From: fdrake at acm.org (Fred Drake)
Date: Wed, 6 Feb 2008 17:58:24 -0500
Subject: [Python-3000] ABC method mismatch
In-Reply-To: <ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>
References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
	<ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>
Message-ID: <0F448ACD-8670-40E2-B823-638143B75E49@acm.org>

On Feb 6, 2008 2:29 PM, Raymond Hettinger <python at rcn.com> wrote:
> Do you want to drop those methods from the ABC or add them to tuple?


On Feb 6, 2008, at 5:40 PM, Guido van Rossum wrote:
> I'm not sure, but I'm tempted to add them to the ABC. What's your
>


Is Guido having a bad typing day?  ;-)

I'd be happy seeing these methods added to tuple; there's no reason  
that they would only be useful on mutable sequences.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From guido at python.org  Thu Feb  7 00:01:41 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Feb 2008 15:01:41 -0800
Subject: [Python-3000] ABC method mismatch
In-Reply-To: <0F448ACD-8670-40E2-B823-638143B75E49@acm.org>
References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
	<ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>
	<0F448ACD-8670-40E2-B823-638143B75E49@acm.org>
Message-ID: <ca471dc20802061501u3029dad1j750eae0b41800d13@mail.gmail.com>

On Feb 6, 2008 2:58 PM, Fred Drake <fdrake at acm.org> wrote:
> On Feb 6, 2008 2:29 PM, Raymond Hettinger <python at rcn.com> wrote:
> > Do you want to drop those methods from the ABC or add them to tuple?
>
>
> On Feb 6, 2008, at 5:40 PM, Guido van Rossum wrote:
> > I'm not sure, but I'm tempted to add them to the ABC. What's your

> Is Guido having a bad typing day?  ;-)

Ever since I turned 50, every day seems to be a bad typing day. :-(

> I'd be happy seeing these methods added to tuple; there's no reason
> that they would only be useful on mutable sequences.

Sounds like this is the consensus. Go for it, Raymond!

--Guido

>    -Fred
>
> --
> Fred Drake   <fdrake at acm.org>
>
>
>
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Feb  7 00:04:01 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Feb 2008 00:04:01 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <fod971$eq5$1@ger.gmane.org>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>	<foaspo$tbc$1@ger.gmane.org>	<47AA086D.60904@v.loewis.de>
	<fod971$eq5$1@ger.gmane.org>
Message-ID: <47AA3CE1.40501@v.loewis.de>

Christian Heimes wrote:
> Martin v. L?wis wrote:
>> I wonder whether such a thing should run as a side effect of a full 
>> garbage collection, also. GC has a chance to free up arenas for good,
>> and allocated-but-unused blocks on arenas prevent Python from returning
>> them to the OS.
> 
> Under which condition does a full garbage collection take place? Does
> the interpreter run a full gc once in a while or is it started by
> gc.collect()?

Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens
after a*b*c object allocations (roughly; early object deallocations
are deducted from that number). Currently, this product is 70000.

> I'm more concerned about the int and float memory blocks. They should
> get freed when more than a couple of hundred blocks are allocated. The
> other free lists are much smaller. But memory fragmentation is an issue
> for them, too.

Ain't we talking about Python 3? There aren't any int memory blocks 
around. As for float objects - sure, we should do something about them.
I propose to change them into a freelist as well, and allocate float
objects through pymalloc.

I'd be curious whether any real-world application allocates tons of
float objects temporarily, and whether such an application would improve
by using the Numeric extensions.

Regards,
Martin

From python at rcn.com  Thu Feb  7 00:05:29 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  6 Feb 2008 18:05:29 -0500 (EST)
Subject: [Python-3000] ABC method mismatch
Message-ID: <20080206180529.AGL71523@ms19.lnh.mail.rcn.net>

[Fred Drake]
> I'd be happy seeing these methods added to tuple; there's
> no reason that they would only be useful on mutable sequences.

That reminds me, I've always thought Sequence.__contains__().
should always be accompanied by Sequence.index().  Their
use cases are strongly related:

  Good news: I searched your house and found your keys
  Bad news: I don't have method to tell you where I found them


Raymond

From lists at cheimes.de  Thu Feb  7 00:46:02 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 07 Feb 2008 00:46:02 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47AA3CE1.40501@v.loewis.de>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>	<foaspo$tbc$1@ger.gmane.org>	<47AA086D.60904@v.loewis.de>	<fod971$eq5$1@ger.gmane.org>
	<47AA3CE1.40501@v.loewis.de>
Message-ID: <47AA46BA.6080900@cheimes.de>

Martin v. L?wis wrote:
> Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens
> after a*b*c object allocations (roughly; early object deallocations
> are deducted from that number). Currently, this product is 70000.

I see. 70k is a large number. Do you have some real world experience how
often a a full gc takes place?

> Ain't we talking about Python 3? There aren't any int memory blocks 
> around. As for float objects - sure, we should do something about them.
> I propose to change them into a freelist as well, and allocate float
> objects through pymalloc.

I was referring to Python 2.x as well. Your proposed solution would make
it easier to maintain a small free list but it may also make allocation
slower. How old are the int and float blocks? Where they implemented
before Python got pymalloc?

I found another place that may gain some speedup from free lists.
Currently list and dict aren't using a real free list. The code
pre-allocates 80 objects each. I've modified the code to free list
without pre-allocated objects. I also added some debug code to measure
how many objects are reused or allocated:

./python setup build

List allocations: 1046
List reuse through freelist: 15746
Dict allocations: 770
Dict reuse through freelist: 1307

A simple startup of Python

List allocations: 59
List reuse through freelist: 644
Dict allocations: 239
Dict reuse through freelist: 223

Christian

From python at rcn.com  Thu Feb  7 02:17:12 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  6 Feb 2008 20:17:12 -0500 (EST)
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
 warnings
Message-ID: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net>

It's interesting that tuples could be registered as Sequences without having all the requisite methods.  Should the register() method to some sort of validation that all public methods in the ABC are in the class/type being registered?


Raymond

From aahz at pythoncraft.com  Thu Feb  7 02:21:16 2008
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 6 Feb 2008 17:21:16 -0800
Subject: [Python-3000] ABC method mismatch
In-Reply-To: <ca471dc20802061501u3029dad1j750eae0b41800d13@mail.gmail.com>
References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
	<ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>
	<0F448ACD-8670-40E2-B823-638143B75E49@acm.org>
	<ca471dc20802061501u3029dad1j750eae0b41800d13@mail.gmail.com>
Message-ID: <20080207012116.GB11221@panix.com>

On Wed, Feb 06, 2008, Guido van Rossum wrote:
> On Feb 6, 2008 2:58 PM, Fred Drake <fdrake at acm.org> wrote:
>> 
>> I'd be happy seeing these methods added to tuple; there's no reason
>> that they would only be useful on mutable sequences.
> 
> Sounds like this is the consensus. Go for it, Raymond!

Far be it from me to act the grinch (and I honestly don't care), but I'm
curious what caused you to change your mind at this point.  You have
argued against this for years:

http://www.python.org/search/hypermail/python-1992/0285.html

More recently, your argument has revolved around the fact that you don't
want tuples used as a "poor man's list" -- they are intended to be used
for heterogeneous data.  (Assuming I'm rephrasing you correctly.)

(My purpose in poking you is to get better at channeling you rather than
because this particular decision has much significance.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From guido at python.org  Thu Feb  7 03:28:05 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Feb 2008 18:28:05 -0800
Subject: [Python-3000] ABC method mismatch
In-Reply-To: <20080207012116.GB11221@panix.com>
References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net>
	<ca471dc20802061440v2f943be1n89a09bbd5c904104@mail.gmail.com>
	<0F448ACD-8670-40E2-B823-638143B75E49@acm.org>
	<ca471dc20802061501u3029dad1j750eae0b41800d13@mail.gmail.com>
	<20080207012116.GB11221@panix.com>
Message-ID: <ca471dc20802061828r5265c2c4mb499eac39062fccd@mail.gmail.com>

On Feb 6, 2008 5:21 PM, Aahz <aahz at pythoncraft.com> wrote:
> On Wed, Feb 06, 2008, Guido van Rossum wrote:
> > On Feb 6, 2008 2:58 PM, Fred Drake <fdrake at acm.org> wrote:
> >>
> >> I'd be happy seeing these methods added to tuple; there's no reason
> >> that they would only be useful on mutable sequences.
> >
> > Sounds like this is the consensus. Go for it, Raymond!
>
> Far be it from me to act the grinch (and I honestly don't care), but I'm
> curious what caused you to change your mind at this point.  You have
> argued against this for years:
>
> http://www.python.org/search/hypermail/python-1992/0285.html
>
> More recently, your argument has revolved around the fact that you don't
> want tuples used as a "poor man's list" -- they are intended to be used
> for heterogeneous data.  (Assuming I'm rephrasing you correctly.)
>
> (My purpose in poking you is to get better at channeling you rather than
> because this particular decision has much significance.)

Well, I guess I changed my mind when I made tuple a virtual subclass
of Sequence. That meant basically accepting it is *also* usable as an
immutable sequence. It is accepting what everybody has been saying for
a long time.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Feb  7 03:29:59 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Feb 2008 18:29:59 -0800
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
	warnings
In-Reply-To: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net>
References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802061829t2dc2923ex666a54e0a1f824f3@mail.gmail.com>

On Feb 6, 2008 5:17 PM, Raymond Hettinger <python at rcn.com> wrote:
> It's interesting that tuples could be registered as Sequences without having all the requisite methods.  Should the register() method to some sort of validation that all public methods in the ABC are in the class/type being registered?

I think the Zope/Twisted interfaces have gone here before. I believe
they have a separate validation call which is typically only invoked
in unittests. The validation can be pretty expensive, so running the
validation each time the interpreter is started would just slow things
down. I've just not yet written the validation code. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Feb  7 03:35:37 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  6 Feb 2008 21:35:37 -0500 (EST)
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
 warnings
Message-ID: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net>

> I think the Zope/Twisted interfaces have gone here before.
> I believe they have a separate validation call which is 
> typically only invoked in unittests. 

+1 That seems like a good idea.



Raymond

From stephen at xemacs.org  Thu Feb  7 03:48:08 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 07 Feb 2008 11:48:08 +0900
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of	those!
In-Reply-To: <20080206012025.00a73dff@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012025.00a73dff@bhuda.mired.org>
Message-ID: <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Meyer writes:

 > Personally, I think *this* [proposal to link 2.x to 2.y for y < x]
 > is a crock. But if you're going to advocate that people lie about
 > what versions they have installed,

If you insist on that interpretation, consider that it's no different
from the common cc -> gcc link.  Anyway, your favored solution of
"python2" tells exactly the same "lie" ("Python 2" is not installed!),
but in a way that cannot be interpreted satisfactorily for as many
apps that have specific version preferences without changing the apps.
Also, it doesn't tell what version of Python is actually recommended
by the developers, which is a potential support issue.  Finally, the
developers who use "python2" shebangs *are* lying (no quotes!) unless
they use only features present in Python 2.0.

You asked whether I oppose making things easy.  Not everything, but I
sure do oppose making such inaccuracy easy!

As I interpret it, in my proposal you aren't telling anyone that you
have that version of Python installed; you are telling applications
that you have installed *some* version of Python that can handle that
version of the API.  Eg, an admin can determine which Python version
will deal with "python2.2" simply by doing "ls -l `which python2.2`".

Again, it's not correct to interpret "python2" that way; "python2" is
the sysadmin's preferred version of Python 2 at the moment, which in
my experience is unlikly to be 2.x.0 for any x, and might be
substantially older if the mission-critical apps run satisfactorily on
that older version.  Implying that "'python2' can handle any Python 2
script" is likely to remain a lie until the release of 2.x.1 at least,
and possibly indefinitely.

 > With python2, [if there's a problem with 2.x's initial release] you
 > set the python2 link back to the previous version. Yeah, you may
 > have to fix the apps that require the new version,

Which is (a) unnecessary in my scheme and (b) a long-term maintenance
burden when upgrading the apps that require the new version.  The
second is a serious annoyance in my experience.

 > Personally, I think that's a *lot* easier than running around creating
 > or re-creating a bunch of symlinks every time I install or upgrade
 > python,

Huh?  Eminently automatable.  Why do you think it would be hard?

Also, do you really install or upgrade Python more often than you
install or upgrade apps that run on Python, each time risking annoying
breakage that requires hacking the app to fix?  And since the app may
not even tell you which versions you already have available that might
be satisfactory, you have to test them yourself, and possibly install
and uninstall several versions!

Finally, do you really uninstall old versions that often?  It seems
likely to me that there probably would be no need at all to fiddle
with symlinks based on upgrading Python 2.4.3 to Python 2.4.4, and no
*pressing* need to update the symlinks upon installing Python 2.5.1
alongside Python 2.4.4.  Rather you'd do that at a later time, after
Python 2.5 "shakes down" and you've tested all your important apps
with 2.5.

Yes, it's a matter of balance, and YMMV.  But between the theoretical
deficiencies of "python2" and the degree to which I believe you're
exaggerating the frequency (or lack thereof) that you would encounter
the relevant situations under the two plans, I really don't see a good
argument for "python2".

 > much less trying to figure out when I really can declare that
 > I no longer need some version under those conditions.

This is a real issue with the scheme I proposed, although modern
versions of file will tell you the whole shebang, I think.  It's also
a problem that modern PMSes will help you with, as well as being one
faced only by those who run an unusually tight ship in my experience.
Again, I would imagine it's not hard to automate.

 > On the other hand, if setup.py made setting up the symlinks easy,
 > maybe I'd change my mind.

I think this should be a separate tool, but either way I'd not object
to making it easy to do, as long as it was not default.


From martin at v.loewis.de  Thu Feb  7 06:04:12 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Feb 2008 06:04:12 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47AA46BA.6080900@cheimes.de>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>	<foaspo$tbc$1@ger.gmane.org>	<47AA086D.60904@v.loewis.de>	<fod971$eq5$1@ger.gmane.org>
	<47AA3CE1.40501@v.loewis.de> <47AA46BA.6080900@cheimes.de>
Message-ID: <47AA914C.8020402@v.loewis.de>

>> Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens
>> after a*b*c object allocations (roughly; early object deallocations
>> are deducted from that number). Currently, this product is 70000.
> 
> I see. 70k is a large number. Do you have some real world experience how
> often a a full gc takes place?

Not from a real world application, no. If you run the program below with

python /tmp/a.py 2>&1|grep "generation 2"

you'll see it does one "regular" generation-2-collection; on my machine,
it takes 0.23s for that to happen. Notice that this application doesn't
use reference counting at all (i.e. objects are released here only
through GC, not because their refcount goes zero before the GC
triggers).

What are you aiming at? Is it a good thing or a bad thing that 70k is
a large number?

> I was referring to Python 2.x as well. Your proposed solution would make
> it easier to maintain a small free list but it may also make allocation
> slower. How old are the int and float blocks? Where they implemented
> before Python got pymalloc?

I don't remember; try to find out for yourself.

I think it will be very difficult to demonstrate that releasing the
freelists has a significant effect when it's run along with GC. The
time for running the GC should dominate the time spend in releasing
the freelists, and any application for which the freelists help (i.e.
which periodically allocate many similar object and then return them)
will rebuild the freelist to the required size quickly.

Regards,
Martin


From martin at v.loewis.de  Thu Feb  7 06:44:12 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Feb 2008 06:44:12 +0100
Subject: [Python-3000] Free list for small longs
In-Reply-To: <47AA914C.8020402@v.loewis.de>
References: <47A8C149.1020407@cheimes.de>	<18344.50025.175774.601181@montanaro-dyndns-org.local>	<bbaeab100802051223i3c10fe7ep9505ef63473594df@mail.gmail.com>	<47A8E376.8000308@v.loewis.de>	<bbaeab100802051505l37d23a87p5d8af58c317db63c@mail.gmail.com>	<foaspo$tbc$1@ger.gmane.org>	<47AA086D.60904@v.loewis.de>	<fod971$eq5$1@ger.gmane.org>	<47AA3CE1.40501@v.loewis.de>
	<47AA46BA.6080900@cheimes.de> <47AA914C.8020402@v.loewis.de>
Message-ID: <47AA9AAC.8000106@v.loewis.de>

> Not from a real world application, no. If you run the program below with

And now for the program below...

Regards,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a.py
Type: text/x-python
Size: 211 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20080207/1023dad9/attachment.py 

From p.f.moore at gmail.com  Thu Feb  7 10:15:16 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 Feb 2008 09:15:16 +0000
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
	idea -- let's do more of those!
In-Reply-To: <47A9FE7B.7090405@cheimes.de>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
Message-ID: <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>

On 06/02/2008, Christian Heimes <lists at cheimes.de> wrote:
> Nice, it's an elegant and easy solution to the problem. I like to get a
> quite similar solution into the core but I propose a slightly different
> path.
>
> Like your launcher.c, spam.exe looks for spam.py. It additionally looks
> for spam.pyw first giving .pyw files a higher priority.
>
> The first line of the file is read and parsed. The line must start with
> a shebang (#!). Contrary to your script it doesn't look for a complete
> path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5"
> both work.
>
> Then the launcher executable queries the installation path of Python X.Y
> from the registry (first HKCU, then HKLM). At last the script is
> executed with the Python binary: python.exe for .py and pythonw.exe for
> .pyw. The working directory is the absolute path of spam.exe. This way
> we neither have to change the paths for Windows scripts nor quote the
> path name.
>
> Comments?

One major issue: The same launcher cannot be used for py and pyw
files. The launcher for py files needs to be compiled as a console
app, whereas the launcher for pyw (GUI) files needs to be compiled as
a GUI app. On Windows, GUI vs Console is a property of the EXE file,
and the differing behaviour is built into the OS loader. You may not
like the fact (at times, I certainly don't) but you can't fight it.

Trying to avoid this is what makes GUI apps with consoles that flash
up quickly before disappearing, and other similar nastiness.

Also, be aware that Python does not *have* to register itself in the
registry (AIUI). But other things (like bdist_wininst IIRC) expect it,
so as long as you don't fail too horribly if there's no registry
entries, you should be OK.

Finally, you don't want to change the current directory. Suppose I had
a simple program listdir.py that displayed the contents of the current
directory. Using your wrapper would break it. Why do you think
changing directory is needed, as opposed to making sys.path look like
you want - oh, hang on, you're running python.exe so you have no
access to sys.path. Hmm, your proposal may have a more serious problem
than I realised.

Paul.

From alexander.belopolsky at gmail.com  Thu Feb  7 16:18:05 2008
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 7 Feb 2008 15:18:05 +0000 (UTC)
Subject: [Python-3000] ABC method mismatch
References: <20080206180529.AGL71523@ms19.lnh.mail.rcn.net>
Message-ID: <loom.20080207T145831-870@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:

> 
> [Fred Drake]
> > I'd be happy seeing these methods added to tuple; there's
> > no reason that they would only be useful on mutable sequences.
> 

Note that adding index and count to tuple will conflict with using these
names as attributes in classes generated by collections.namedtuple.
There is certain elegance in tuples and numbers having no non-
special methods.

What is the advantage of having count and index in Sequence ABC?
These methods are easily implementable as functions that take an
iterable (in fact, itertools.count already exists) and having them in
Sequence ABC puts an extra burden on implementors of concrete classes.


From lists at cheimes.de  Thu Feb  7 17:27:42 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 07 Feb 2008 17:27:42 +0100
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
	warnings
In-Reply-To: <ca471dc20802061829t2dc2923ex666a54e0a1f824f3@mail.gmail.com>
References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net>
	<ca471dc20802061829t2dc2923ex666a54e0a1f824f3@mail.gmail.com>
Message-ID: <fofbhu$kq4$1@ger.gmane.org>

Guido van Rossum wrote:
> I think the Zope/Twisted interfaces have gone here before. I believe
> they have a separate validation call which is typically only invoked
> in unittests. The validation can be pretty expensive, so running the
> validation each time the interpreter is started would just slow things
> down. I've just not yet written the validation code. :-)

You are correct. Normally interfaces are only validated in unit tests.
The validation function of zope.interface checks if a class implements
or object provides the promised methods and attributes. It also compares
the method signatures.

Christian


From fdrake at acm.org  Thu Feb  7 17:36:51 2008
From: fdrake at acm.org (Fred Drake)
Date: Thu, 7 Feb 2008 11:36:51 -0500
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
	warnings
In-Reply-To: <fofbhu$kq4$1@ger.gmane.org>
References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net>
	<ca471dc20802061829t2dc2923ex666a54e0a1f824f3@mail.gmail.com>
	<fofbhu$kq4$1@ger.gmane.org>
Message-ID: <0E557A86-41B2-495F-B610-68338F9FE47D@acm.org>

On Feb 7, 2008, at 11:27 AM, Christian Heimes wrote:
> You are correct. Normally interfaces are only validated in unit tests.
> The validation function of zope.interface checks if a class implements
> or object provides the promised methods and attributes. It also  
> compares
> the method signatures.


Note also that checking the "class implements" is frequently  
problematic since the verification doesn't know what the factory does  
to the object (where the factory is the constructor provided by the  
class in most cases).  We frequently test instances this way in unit  
tests.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From greg.ewing at canterbury.ac.nz  Thu Feb  7 21:34:01 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 08 Feb 2008 09:34:01 +1300
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
 warnings
In-Reply-To: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net>
References: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net>
Message-ID: <47AB6B39.4090300@canterbury.ac.nz>

Raymond Hettinger wrote:
>>I think the Zope/Twisted interfaces have gone here before.
>>I believe they have a separate validation call which is 
>>typically only invoked in unittests. 
> 
> +1 That seems like a good idea.

Or maybe this is a task for pylint?

--
Greg

From fdrake at acm.org  Thu Feb  7 21:46:34 2008
From: fdrake at acm.org (Fred Drake)
Date: Thu, 7 Feb 2008 15:46:34 -0500
Subject: [Python-3000] Should ABCMeta.register() do validation and emit
	warnings
In-Reply-To: <47AB6B39.4090300@canterbury.ac.nz>
References: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net>
	<47AB6B39.4090300@canterbury.ac.nz>
Message-ID: <842C9510-415B-4959-A036-5C08C31BA4BC@acm.org>

On Feb 7, 2008, at 3:34 PM, Greg Ewing wrote:
> Or maybe this is a task for pylint?


Maybe a lint tool could help, but it's not unusual for this to be  
relevant only for specific instances, not necessarily all instances.   
Certainly with zope.interface, you can create objects that provide a  
particular interface (and are recognized as providing it) without it  
being tied to their class.

This is an important strength of the zope.interface module.  Since  
it's entirely dependent on the dynamic behavior of the application, a  
lint tool will probably have a hard time with that.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From greg.ewing at canterbury.ac.nz  Thu Feb  7 21:52:18 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 08 Feb 2008 09:52:18 +1300
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
Message-ID: <47AB6F82.4070308@canterbury.ac.nz>

Paul Moore wrote:
> On Windows, GUI vs Console is a property of the EXE file,
> and the differing behaviour is built into the OS loader.

Would it be feasible for it to always start up as a gui
app, and then create its own console window (a fake one
if necessary) if it decides it needs one?

Doing things this way might make it possible to be a
bit smarter and not have the console window go away
the moment the script ends. That tends to make
console-mode scripts rather useless...

Also, it would be very handy if in all cases it would
display a traceback somehow in the event of an uncaught
exception. It's frustrating when a gui app fails to start
up and just vanishes into a black hole leaving no clue
what went wrong.

--
Greg

From pje at telecommunity.com  Thu Feb  7 23:10:36 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 07 Feb 2008 17:10:36 -0500
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <47AB6F82.4070308@canterbury.ac.nz>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
Message-ID: <20080207221027.261D33A40D7@sparrow.telecommunity.com>

At 09:52 AM 2/8/2008 +1300, Greg Ewing wrote:
>Would it be feasible for it to always start up as a gui
>app, and then create its own console window (a fake one
>if necessary) if it decides it needs one?

No, because that would make it impossible to write a sane 
command-line app.  Imagine if running 'ls' always created a *new* 
console window into which its results would be written...  following 
which that console window would immediately close.

That's what happens if a Windows GUI app creates its own console later. 


From p.f.moore at gmail.com  Thu Feb  7 23:21:37 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 7 Feb 2008 22:21:37 +0000
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <20080207221027.261D33A40D7@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
Message-ID: <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>

On 07/02/2008, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:52 AM 2/8/2008 +1300, Greg Ewing wrote:
> >Would it be feasible for it to always start up as a gui
> >app, and then create its own console window (a fake one
> >if necessary) if it decides it needs one?
>
> No, because that would make it impossible to write a sane
> command-line app.  Imagine if running 'ls' always created a *new*
> console window into which its results would be written...  following
> which that console window would immediately close.
>
> That's what happens if a Windows GUI app creates its own console later.

As Phillip says. Windows console and GUI executables are completely
different in behaviour. Don't try to fight it. You need two distinct
EXEs.

(I can argue the pros and cons of Unix vs Windows behaviour all night.
I'll take whichever side you prefer. The only thing that I am sure of
is that trying to ignore the differences is always going to give you
problems.)

Paul.

From greg.ewing at canterbury.ac.nz  Fri Feb  8 03:59:23 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 08 Feb 2008 15:59:23 +1300
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
Message-ID: <47ABC58B.7010608@canterbury.ac.nz>

Paul Moore wrote:
> As Phillip says. Windows console and GUI executables are completely
> different in behaviour. Don't try to fight it. You need two distinct
> EXEs.

Hmmm. So maybe what's needed is a *third* kind of exe that
gets launched when you double-click a .py file, that keeps
its console open after the script finishes?

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From agthorr at barsoom.org  Fri Feb  8 04:19:09 2008
From: agthorr at barsoom.org (Daniel Stutzbach)
Date: Thu, 7 Feb 2008 21:19:09 -0600
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <47ABC58B.7010608@canterbury.ac.nz>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
Message-ID: <eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>

On Feb 7, 2008 8:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Paul Moore wrote:
> > As Phillip says. Windows console and GUI executables are completely
> > different in behaviour. Don't try to fight it. You need two distinct
> > EXEs.
>
> Hmmm. So maybe what's needed is a *third* kind of exe that
> gets launched when you double-click a .py file, that keeps
> its console open after the script finishes?
>

Picture command-line usage of python.  You're sitting at your prompt, and
you run a python script.  It pops up a *new* window and you have to interact
with that.  Not a great user experience.  Also, piping data to or from the
script is impossible because stdin/stdout are tied to the new window.  (I
use Python for windows and pipe data to scripts everyday, so this is pretty
important to me :-) )

I agree with Paul and Phillip that having two EXEs is necessary.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080207/b8e5c46d/attachment-0001.htm 

From pje at telecommunity.com  Fri Feb  8 05:19:24 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 07 Feb 2008 23:19:24 -0500
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <47ABC58B.7010608@canterbury.ac.nz>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
Message-ID: <20080208041917.4390C3A4076@sparrow.telecommunity.com>

At 03:59 PM 2/8/2008 +1300, Greg Ewing wrote:
>Paul Moore wrote:
> > As Phillip says. Windows console and GUI executables are completely
> > different in behaviour. Don't try to fight it. You need two distinct
> > EXEs.
>
>Hmmm. So maybe what's needed is a *third* kind of exe that
>gets launched when you double-click a .py file, that keeps
>its console open after the script finishes?

That's actually a separate problem from the one Paul and I are 
talking about, one that I'm not even going to *try* to solve.  (Being 
as I already have, and failed miserably.)  Hell, I'm not sure Windows 
even has a way to *tell* whether your app was launched from the 
command line or otherwise.  I couldn't find one for Win2K, anyway; 
there might be a way to do it in XP and Vista.


From facundobatista at gmail.com  Fri Feb  8 13:02:55 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 8 Feb 2008 10:02:55 -0200
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
	<eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
Message-ID: <e04bdf310802080402l5424ba95rfe0cdb2c39efc014@mail.gmail.com>

2008/2/8, Daniel Stutzbach <agthorr at barsoom.org>:

> Picture command-line usage of python.  You're sitting at your prompt, and
> you run a python script.  It pops up a *new* window and you have to interact
> with that.  Not a great user experience.  Also, piping data to or from the
> script is impossible because stdin/stdout are tied to the new window.  (I
> use Python for windows and pipe data to scripts everyday, so this is pretty
> important to me :-) )

Not really following this discussion, but a detail regarding
stdin/stdout in windows.

When the process is issued as a Service, it don't even HAS
stdin/stdout (not that is redirected to NUL). It don't even exist.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From mwm at mired.org  Wed Feb  6 20:07:43 2008
From: mwm at mired.org (Mike Meyer)
Date: Wed, 6 Feb 2008 14:07:43 -0500
Subject: [Python-3000] [Python-ideas] Namespaces are one honking great
 idea -- let's do more of those!
In-Reply-To: <47A99557.2000303@gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<47A87A31.8070208@gmail.com> <20080205112232.34bc5c80@mbook-fbsd>
	<47A99557.2000303@gmail.com>
Message-ID: <20080206140743.7aafa84b@mbook-fbsd>

On Wed, 06 Feb 2008 21:09:11 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Mike Meyer wrote:
> > On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> While I'm +1 on the idea of making altinstall the default for setup.py 
> >> and the Windows installer (forcing the administrator to explicitly 
> >> request that the new Python version be made the default Python 
> >> installation for the machine), I'm a pretty strong -1 on officially 
> >> blessing the idea of a python3 alias or a special py3 extension on 
> >> Windows (end users are obviously free to create those if they want, of 
> >> course).
> > I like the python3 alias (and python2 in 2.X), because it moves us
> > from a single default python to having a default python2 and python3,
> > which means scripts that care can ask for that without breaking the
> > ability to upgrade python2 or python3 without breaking those scripts,
> > while still allowing the default python to be either python2 or
> > python3.
> The main objection I have is that the executable name is only part of 
> the story - all of the PYTHON* environment variables may matter as well 
> (there are 9 listed in python -h, and I would have to check the code to 
> say with complete confidence that there aren't any others).

True, there are 9 of them, but most of them are version-neutral, in
that they change the behavior of python in some way, but *not* what
scripts it finds. While you may want different versions of them for
different python versions, 2 vs. 3 doesn't seem to be a factor (unless
it ignores them and uses others):

PYTHONOPTIMIZE, PYTHONDEBUG, PYTHONINSPECT, PYTHONUNBUFFERED, PYTHONVERBOSE
and PYTHONY2K

For that matter, all but PYTHON2K are an alternative to a command line
option. I think we can safely ignore all of them. That leaves:

PYTHONHOME - this tells python where to find it's standard
libraries. Setting it is bad for your health and sanity unless you
*really* know what you're doing. I think we can safely ignore it.

PYTHONSTARTUP - generally only set per user, and only for interactive
pythons. Anyone with the chops to set this up can make it work for
both by checking the version and importing an appropriate
version-specific module.

PYTHONPATH - this one is the real issue. It's used by people who can't
write to python library to point add third party modules to the
standard path. Personally, I tend to install Python (after all, if I
can install the modules, I can almost certainly install python) in
that case, but that doesn't help them. If they have both installed on
the system and can't write to their libraries, something has to be
done to deal with it.

> That said, while as a matter of purity I still think this is a platform 
> problem (one which all current platforms I know of handle poorly ;), as 
> a matter of practicality I'm seeing some benefits allowing a machine to 
> have full fledged "Python 2" and "Python 3" setups side by side.
> 
> If Guido does decide to go the route of changing the interpreter binary 
> to python3 (ala sqlite vs sqlite3), I would also suggest that we go the 
> whole way and insert a '3' into all of the environment variable names 
> that python3 looks for (PYTHON3PATH, PYTHON3STARTUP, etc). I'd even 
> concede the utility of accepting the py3 extension on modules as a 
> concession to extension based dispatching on Windows.

Actually, you can get that kind of behavior by using shell scripts
instead of symlinks for  python2 and python3 commands:

#!/bin/sh
PYTHONPATH=${PYTHON2PATH:-$PYTHONPATH} /usr/bin/python2.6

and similarly

#!/bin/sh
PYTHONPATH=${PYTHON3PATH:-$PYTHONPATH} /usr/bin/python3.0

Except you can't use #!/usr/bin/python2 in this case; you have to use
#!/usr/bin/env python2, which might be a problem.

> I still think it would be a bit of an ugly hack, but if it was followed 
> through to the point of being able to define separate Python 2 & 3 
> environments without the use of launch scripts (i.e. by changing the 
> environment variable names as described above), maybe it would still be 
> a worthwhile thing to do.

How about if we hide the "launch scripts" in the python2 and python3
commands, as above?

	  <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From mwm at mired.org  Thu Feb  7 07:30:54 2008
From: mwm at mired.org (Mike Meyer)
Date: Thu, 7 Feb 2008 01:30:54 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of	those!
In-Reply-To: <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012025.00a73dff@bhuda.mired.org>
	<87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20080207013054.459233bc@bhuda.mired.org>

On Thu, 07 Feb 2008 11:48:08 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Mike Meyer writes:
>  > On the other hand, if setup.py made setting up the symlinks easy,
>  > maybe I'd change my mind.
> I think this should be a separate tool, but either way I'd not object
> to making it easy to do, as long as it was not default.

And this is the real issue with your approach. If you're doing things
on systems you control, or where you're bundling in the python install
you're going to use, then we can do it however you want, and things
will be fine.

People building an app for distribution are going to want it to run in
an many places as possible. If the only name that python makes
available by default is "python", that's what they're going to use.

Removing the default for "python" is a step backwards. Most packagers
and installers will add it back anyway, so many app developers will
use it. Systems that fail to add it will appear broken. Apps that use
a more specific shebang will, at the *very* best, break on exactly the
same set of systems they'd break on using "python", and more likely
break on a lot more.

Leaving things as they are leaves things as they are. If we're going
to break backwards comparability in the language, I think we should
try and improve this situation.

Adding an install target or tool to create version-specific links (of
any flavor) suggests a solution, but I suspect it'll be ignored by a
large percentage of platforms. So packagers for platform that don't
ignore it can patch scripts to take advantage of it, but they tend to
handle multiple versions cleanly in any case, so it's not that big a
win for them. Developers wanting to distribute cross-platform scripts
are still pretty much where they are now.

Making the default install provide those links gives those developers
an expectation that they can use them. If some packagers breaks that -
well, the problem is that python on their system is broken. Yeah, some
developers will ignore the links and keep using "python". But some -
me, for instance - will provide a patch to users and tell them to
complain to the maintainers of that package for their system.

	 <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From lists at cheimes.de  Fri Feb  8 18:25:54 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 08 Feb 2008 18:25:54 +0100
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <e04bdf310802080402l5424ba95rfe0cdb2c39efc014@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>	<47A9FE7B.7090405@cheimes.de>	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>	<47AB6F82.4070308@canterbury.ac.nz>	<20080207221027.261D33A40D7@sparrow.telecommunity.com>	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>	<47ABC58B.7010608@canterbury.ac.nz>	<eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
	<e04bdf310802080402l5424ba95rfe0cdb2c39efc014@mail.gmail.com>
Message-ID: <foi3b2$bis$1@ger.gmane.org>

Facundo Batista wrote:
> When the process is issued as a Service, it don't even HAS
> stdin/stdout (not that is redirected to NUL). It don't even exist.

GUI apps don't have standard streams stdin, stdout and stderr, too. The
variables are defined when the code is compiled but during runtime the
streams are not connected fileno(stdout) is -1.

Christian


From greg.ewing at canterbury.ac.nz  Sat Feb  9 00:45:45 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 09 Feb 2008 12:45:45 +1300
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
	<eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
Message-ID: <47ACE9A9.10305@canterbury.ac.nz>

Daniel Stutzbach wrote:
>     Hmmm. So maybe what's needed is a *third* kind of exe that
>     gets launched when you double-click a .py file, that keeps
>     its console open after the script finishes?
> 
> Picture command-line usage of python.  You're sitting at your prompt, 
> and you run a python script.  It pops up a *new* window and you have to 
> interact with that.

That's why I just suggested a *third* exe. It wouldn't be
used as the command line interpreter -- it would only get
launched by double-clicking a .py file in the gui.

The current behaviour of having the gui launch the normal
command line interpreter for .py files is almost useless,
because the window disappears before you can see the output.

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Feb  9 00:48:19 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 09 Feb 2008 12:48:19 +1300
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <20080208041917.4390C3A4076@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
	<20080208041917.4390C3A4076@sparrow.telecommunity.com>
Message-ID: <47ACEA43.5050907@canterbury.ac.nz>

Phillip J. Eby wrote:
> Hell, I'm not sure Windows even 
> has a way to *tell* whether your app was launched from the command line 
> or otherwise.

It wouldn't have to, if the .py extension were bound to
a different executable from the standard command line
interpreter, in the same way that .pyw is bound to the
pythonw executable.

--
Greg

From stephen at xemacs.org  Sat Feb  9 01:44:41 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 09 Feb 2008 09:44:41 +0900
Subject: [Python-3000] Namespaces are one honking great idea --
	let's	do	more of	those!
In-Reply-To: <20080207013054.459233bc@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<aac2c7cb0802012308i4d173c5s82ee93e313645a3@mail.gmail.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012025.00a73dff@bhuda.mired.org>
	<87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080207013054.459233bc@bhuda.mired.org>
Message-ID: <878x1vnekm.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Meyer writes:
 > On Thu, 07 Feb 2008 11:48:08 +0900 "Stephen J. Turnbull" <stephen at xemacs.org> wrote:

 > > I think this should be a separate tool, but either way I'd not object
 > > to making it easy to do, as long as it was not default.
 > 
 > And this is the real issue with your approach. If you're doing things
 > on systems you control, or where you're bundling in the python install
 > you're going to use, then we can do it however you want, and things
 > will be fine.

Please, give me some credit.  I'm talking about a default for systems
I don't control, but do want (a) to stop giving bad advice to and (b)
to stop abetting their bad practices.

So don't tell me that I'm mistaking my personal practices for a global
optimum.  I'm not.  I've thought pretty carefully about how this
applies in the context of presently widespread practices.

But now that you've put that fallacy on the table, let me point out
that it seems to apply to *you*.  It's just that lots of other people
have adopted the same global suboptimum, which makes it workable.  As
Opus the penguin says, "If a million people do the same stupid thing,
it is still a stupid thing."  Now, the "python" link is not *stupid*
in the context of the economics of networks, but we can do better, and
it won't cost much.

 > People building an app for distribution are going to want it to run in
 > an many places as possible. If the only name that python makes
 > available by default is "python", that's what they're going to use.

First, you're wrong, as you immediately pointed out yourself.  It's
not *Python-provided* defaults that matter.  It's the defaults
provided by common distro vendors, which for the market in question
dwarf installs from source as distributed by Python.

Second, my point is to change that default *and the bug-prone behavior
enabled by it*, so I'm not at all bothered by the fact that I'm
changing the default and thus am incompatible with bug-prone behavior.

 > Leaving things as they are leaves things as they are. If we're going
 > to break backwards comparability in the language, I think we should
 > try and improve this situation.

AFAICS my proposal *is* a technical improvement, in the sense that it
will work substantially better and at lower total cost in breakage,
debugging, and special-casing within communities that adopt it.  You
have made no argument against it except that it is a change and you
don't like change, and then you vastly exaggerate the likely annoyance
the removal of the "python" link creation step will cause, as well as
the potential for actual breakage.  True, it may get no uptake, and
then the small annoyances that will occasionally happen are for
nothing.  But the upside is quite large compared to the small
annoyances.

 > Adding an install target or tool to create version-specific links (of
 > any flavor) suggests a solution, but I suspect it'll be ignored by a
 > large percentage of platforms.

I doubt they'll ignore it.  They may decide not to implement it for
the reasons you gave (stipulated and omitted).  More likely, platforms
like Debian with its /etc/alternatives mechanism will find a way to
fit it into their existing alternatives mechanism.  Either way, if
Python adopts it for from-source installs, my proposal is compatible
with such systems.  If it works well, and as you suggest below app
vendors who use it press the platform maintainers (directly or through
customers as you suggest), platforms will adopt it -- we're talking a
couple dozen people who do watch each other and diffuse innovations
rapidly in the open source sector, and the proprietary vendors are
rapidly becoming more nimble (and watching the open source sector)
themselves.

 > Making the default install provide those links gives those developers
 > an expectation that they can use them. If some packagers breaks that -
 > well, the problem is that python on their system is broken. Yeah, some
 > developers will ignore the links and keep using "python". But some -
 > me, for instance - will provide a patch to users and tell them to
 > complain to the maintainers of that package for their system.

That's the way it's supposed to work.  I don't claim that my proposal
will be entirely painless, but it won't be very painful.  Application
developers who *do* use it can get benefits *without* any cooperation
from installed systems as long as users can install appropriate links
(or wrappers on Windows) on their executable search PATH.  The only
problem you'll face is if the systems are doing something really
insane like "ln -s perl5.8 python2.2".<wink>  And of course Python or
third parties can provide a link analysis tool for determining what
versions of Python are available and where the binaries live (eg,
borrow Zope's).

From python at rcn.com  Sat Feb  9 01:51:43 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri,  8 Feb 2008 19:51:43 -0500 (EST)
Subject: [Python-3000] Nix dict.copy()
Message-ID: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>

I recommend dropping the dict.copy() method from Py3.0.

* We can already write:  newd = copy.copy(d).
* We can also write:     newd = dict(d)
* Both of those approaches also work for most other containers.
* The collections.Mapping ABC does not support copy().
* copy() is not a universal feature of mapping like objects
* Lists do not have a copy() method.
* If we drop dict.copy(), I'll can also drop set.copy() which is unneeded.

Let's make the basic APIs as clean and parallel as possible.

Raymond

From guido at python.org  Sat Feb  9 02:03:13 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Feb 2008 17:03:13 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802081703w1a73e0catbd1aaecbcf56fb28@mail.gmail.com>

On Feb 8, 2008 4:51 PM, Raymond Hettinger <python at rcn.com> wrote:
> I recommend dropping the dict.copy() method from Py3.0.
>
> * We can already write:  newd = copy.copy(d).
> * We can also write:     newd = dict(d)
> * Both of those approaches also work for most other containers.
> * The collections.Mapping ABC does not support copy().
> * copy() is not a universal feature of mapping like objects
> * Lists do not have a copy() method.
> * If we drop dict.copy(), I'll can also drop set.copy() which is unneeded.
>
> Let's make the basic APIs as clean and parallel as possible.

Makes sense.  I request that you also implement the transitional code:
either a fixer for 2to3 or (perhaps more realistically) a warning to
dict.copy() when -3 is given.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Feb  9 02:35:57 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri,  8 Feb 2008 20:35:57 -0500 (EST)
Subject: [Python-3000] Thoughts on collections.Container and
	collections.Iterable
Message-ID: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>

I'm wondering if collections.Iterable should inherit from collections.Container"?

In Python, anything that supports __iter__ automatically supports tests using the "in" operator:

   class A:
       def __iter__(self):
           for i in (1,2,3):
                yield i
   >>> 2 in A()
   True

The two concepts are deeply related.  Anywhere we can write "for x in s: ..." we can also write "if x in s: ...".

The new definition of Iterable would look like this:

    class Iterable(Container):
        __metaclass__ = ABCMeta

        @abstractmethod
        def __iter__(self):
            while False:
                yield None

        @classmethod
        def __subclasshook__(cls, C):
            if cls is Iterable:
                if any("__iter__" in B.__dict__ for B in C.__mro__):
                    return True
            return NotImplemented

        def __contains__(self, value):
            for v in self:
                if v == value:
                    return True
            return False


Raymond

From guido at python.org  Sat Feb  9 02:42:37 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Feb 2008 17:42:37 -0800
Subject: [Python-3000] Thoughts on collections.Container and
	collections.Iterable
In-Reply-To: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>
References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>

I would need to think more about this. I'm tempted not to do this, and
let these ABCs denote the *explicit* presence of __contains__ and
__iter__, respectively. Something that's iterable but doesn't
implement __contains__ supports the 'in' operator very inefficiently
(through linear search) which we might not want to encourage.

--Guido

On Feb 8, 2008 5:35 PM, Raymond Hettinger <python at rcn.com> wrote:
> I'm wondering if collections.Iterable should inherit from collections.Container"?
>
> In Python, anything that supports __iter__ automatically supports tests using the "in" operator:
>
>    class A:
>        def __iter__(self):
>            for i in (1,2,3):
>                 yield i
>    >>> 2 in A()
>    True
>
> The two concepts are deeply related.  Anywhere we can write "for x in s: ..." we can also write "if x in s: ...".
>
> The new definition of Iterable would look like this:
>
>     class Iterable(Container):
>         __metaclass__ = ABCMeta
>
>         @abstractmethod
>         def __iter__(self):
>             while False:
>                 yield None
>
>         @classmethod
>         def __subclasshook__(cls, C):
>             if cls is Iterable:
>                 if any("__iter__" in B.__dict__ for B in C.__mro__):
>                     return True
>             return NotImplemented
>
>         def __contains__(self, value):
>             for v in self:
>                 if v == value:
>                     return True
>             return False
>
>
> Raymond
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Sat Feb  9 02:59:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Feb 2008 11:59:05 +1000
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <47ACE9A9.10305@canterbury.ac.nz>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<20080204185642.73659254@bhuda.mired.org>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>	<47A9FE7B.7090405@cheimes.de>	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>	<47AB6F82.4070308@canterbury.ac.nz>	<20080207221027.261D33A40D7@sparrow.telecommunity.com>	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>	<47ABC58B.7010608@canterbury.ac.nz>	<eae285400802071919t7f2831e1w1fc7d06d766e79ee@mail.gmail.com>
	<47ACE9A9.10305@canterbury.ac.nz>
Message-ID: <47AD08E9.2010806@gmail.com>

Greg Ewing wrote:
> Daniel Stutzbach wrote:
>>     Hmmm. So maybe what's needed is a *third* kind of exe that
>>     gets launched when you double-click a .py file, that keeps
>>     its console open after the script finishes?
>>
>> Picture command-line usage of python.  You're sitting at your prompt, 
>> and you run a python script.  It pops up a *new* window and you have to 
>> interact with that.
> 
> That's why I just suggested a *third* exe. It wouldn't be
> used as the command line interpreter -- it would only get
> launched by double-clicking a .py file in the gui.
> 
> The current behaviour of having the gui launch the normal
> command line interpreter for .py files is almost useless,
> because the window disappears before you can see the output.

As far as I know, double-clicking in the GUI and typing the script name 
at a command prompt use the same mechanism to determine the file 
association. I'm not sure it is possible to separate them.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Feb  9 03:23:24 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Feb 2008 12:23:24 +1000
Subject: [Python-3000] Thoughts on collections.Container
	and	collections.Iterable
In-Reply-To: <ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>
	<ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
Message-ID: <47AD0E9C.4080805@gmail.com>

Guido van Rossum wrote:
> I would need to think more about this. I'm tempted not to do this, and
> let these ABCs denote the *explicit* presence of __contains__ and
> __iter__, respectively. Something that's iterable but doesn't
> implement __contains__ supports the 'in' operator very inefficiently
> (through linear search) which we might not want to encourage.

I think a bigger conceptual problem with doing containment tests on 
arbitrary iterables is that you may end up consuming an iterator in the 
process of checking whether it contains the item you want (i.e. if 
Iterable defines __contains__, then Iterator will acquire it by 
inheritance).

So "assert (x in obj) == (x in obj)" may well fail in some cases if we 
adopt the idea that being iterable implies support for containment tests.

Random thought inspired by the __contains__ code in Raymond's post: does 
anyone else think a 'key' parameter on any() and all() would be useful?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at rcn.com  Sat Feb  9 03:31:28 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri,  8 Feb 2008 21:31:28 -0500 (EST)
Subject: [Python-3000] Thoughts on collections.Container
 and	collections.Iterable
Message-ID: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net>

[Nick]
>  does anyone else think a 'key' parameter on any() and all() 
> would be useful?

Not really.  The key= function is useful whenever you need to
keep the original value around for some reason (i.e. sorting
it or getting the min/max value).  But with any() and all()
you don't keep anything.

Also, if you take examples of any() and all() and try them
with the key= syntax, you'll find that they become less readable.

Besides, for the "gurus who are in the know", the min() and max()
functions can often be substituted directly for any() and all()
and then you get the key= syntax for free ;-)

   any(x%2==0 for x in s)
   max(x%2==0 for x in s)

Raymond

From digitalxero at gmail.com  Sat Feb  9 03:45:35 2008
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Fri, 8 Feb 2008 18:45:35 -0800
Subject: [Python-3000] Namespaces are one honking great idea -- let's do
	more of those!
In-Reply-To: <20080206012435.44bd22b7@bhuda.mired.org>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012435.44bd22b7@bhuda.mired.org>
Message-ID: <e9764b730802081845r173ec350y34c8df55bd41eb61@mail.gmail.com>

On Feb 5, 2008 10:24 PM, Mike Meyer <mwm at mired.org> wrote:
> Sure, there are *lots* of ways the user can screw up the system as
> root. Doing a default python install from source is *not* one of them,
> no matter what it does with those symlinks. If you know of a system
> where things are otherwise, please warn me about it now.

Ubuntu will screw up if you login as root and build a newer python
version then 2.2. You wont be able to use it's GUI package installer
and other issues.

From fdrake at acm.org  Sat Feb  9 03:52:06 2008
From: fdrake at acm.org (Fred Drake)
Date: Fri, 8 Feb 2008 21:52:06 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
Message-ID: <D6C2AD40-CF20-4AC5-B510-47582F718C71@acm.org>

On Feb 8, 2008, at 7:51 PM, Raymond Hettinger wrote:
> I recommend dropping the dict.copy() method from Py3.0.


+1


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From ntoronto at cs.byu.edu  Sat Feb  9 06:04:17 2008
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 08 Feb 2008 22:04:17 -0700
Subject: [Python-3000] Thoughts on collections.Container
	and	collections.Iterable
In-Reply-To: <ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>
	<ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
Message-ID: <47AD3451.6000609@cs.byu.edu>

Guido van Rossum wrote:
> I would need to think more about this. I'm tempted not to do this, and
> let these ABCs denote the *explicit* presence of __contains__ and
> __iter__, respectively. Something that's iterable but doesn't
> implement __contains__ supports the 'in' operator very inefficiently
> (through linear search) which we might not want to encourage.
It could be worse. Is a container necessarily finite?

Neil


From ntoronto at cs.byu.edu  Sat Feb  9 06:30:08 2008
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 08 Feb 2008 22:30:08 -0700
Subject: [Python-3000] Thoughts on
	collections.Container	and	collections.Iterable
In-Reply-To: <47AD3451.6000609@cs.byu.edu>
References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>	<ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
	<47AD3451.6000609@cs.byu.edu>
Message-ID: <47AD3A60.40404@cs.byu.edu>

Neil Toronto wrote:
> Guido van Rossum wrote:
>> I would need to think more about this. I'm tempted not to do this, and
>> let these ABCs denote the *explicit* presence of __contains__ and
>> __iter__, respectively. Something that's iterable but doesn't
>> implement __contains__ supports the 'in' operator very inefficiently
>> (through linear search) which we might not want to encourage.
> It could be worse. Is a container necessarily finite?

I meant an iterable, of course, not a container.

Neil

From mwm at mired.org  Sat Feb  9 07:35:13 2008
From: mwm at mired.org (Mike Meyer)
Date: Sat, 9 Feb 2008 01:35:13 -0500
Subject: [Python-3000] Namespaces are one honking great idea -- let's
	do	more of those!
In-Reply-To: <e9764b730802081845r173ec350y34c8df55bd41eb61@mail.gmail.com>
References: <994938.11662.qm@web31111.mail.mud.yahoo.com>
	<47A4F1D5.5080405@canterbury.ac.nz>
	<87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1202119219.21638.4.camel@qrnik>
	<87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204123057.1004e485@mbook-fbsd>
	<87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080204183807.412b14d8@bhuda.mired.org>
	<87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20080206012435.44bd22b7@bhuda.mired.org>
	<e9764b730802081845r173ec350y34c8df55bd41eb61@mail.gmail.com>
Message-ID: <20080209013513.63a5a621@bhuda.mired.org>

On Fri, 8 Feb 2008 18:45:35 -0800 "Dj Gilcrease" <digitalxero at gmail.com> wrote:
> On Feb 5, 2008 10:24 PM, Mike Meyer <mwm at mired.org> wrote:
> > Sure, there are *lots* of ways the user can screw up the system as
> > root. Doing a default python install from source is *not* one of them,
> > no matter what it does with those symlinks. If you know of a system
> > where things are otherwise, please warn me about it now.
> Ubuntu will screw up if you login as root and build a newer python
> version then 2.2. You wont be able to use it's GUI package installer
> and other issues.

Are you sure? The default Python install prefix is /usr/local.  Ubuntu
(at least on the admittedly aged system I have here) puts the system
python in /usr/bin. So that default python install shouldn't break the
system tools, providing they run the right python. For that matter,
even on this relatively old Ubuntu, they've got a 2.4 version
installed.

Of course, once you've got more than one python installed on the
system, chances are good that some - and possibly all - of your
scripts will break if you run them on the wrong python. But even if
the scripts run the default python (and system scripts shouldn't), the
above can be fixed by tweaking the environment - and root's
environment on my box is right.

    Thanks,
    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

From ncoghlan at gmail.com  Sat Feb  9 14:08:07 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Feb 2008 23:08:07 +1000
Subject: [Python-3000] Thoughts on collections.Container
	and	collections.Iterable
In-Reply-To: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net>
References: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net>
Message-ID: <47ADA5B7.9000408@gmail.com>

Raymond Hettinger wrote:
> Besides, for the "gurus who are in the know", the min() and max()
> functions can often be substituted directly for any() and all()
> and then you get the key= syntax for free ;-)
> 
>    any(x%2==0 for x in s)
>    max(x%2==0 for x in s)

Except that you lose the short-circuiting behaviour by using the latter 
form.

Anyway, for some reason the whole 'these work really well with generator 
expressions' aspect of any/all completely slipped my mind when I asked 
that question about a key argument. Sorry for the pointless noise...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From suraj at barkale.com  Fri Feb  8 18:38:52 2008
From: suraj at barkale.com (Suraj Barkale)
Date: Fri, 8 Feb 2008 17:38:52 +0000 (UTC)
Subject: [Python-3000] Windows gui vs. console
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
Message-ID: <loom.20080208T173715-184@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> Paul Moore wrote:
> > On Windows, GUI vs Console is a property of the EXE file,
> > and the differing behaviour is built into the OS loader.
> 
> Would it be feasible for it to always start up as a gui
> app, and then create its own console window (a fake one
> if necessary) if it decides it needs one?
> 
This is not feasible if you are running default cmd.exe as your shell. See this
blog post for details
http://blogs.adobe.com/simplicity/2007/04/window_and_consolefriendly_win_1.html

Regards,
Suraj


From python at rcn.com  Sat Feb  9 22:33:57 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 9 Feb 2008 13:33:57 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
Message-ID: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>

In the process of getting UserDict updated to subclass from MutableMapping, it ended-up in the collections module during the 
transistion.  Probably, all three should end-up in the same place, either in the collections module or in their own modules.  What 
do you guys think?  Do you like them in collections or as separate modules?

Also, I'm wondering if it is time for UserString to bite the dust. It isn't really useful in the same way a UserDict or UserList 
which have two principal use cases. One, for testing to make sure that simulated dicts and lists are substitutable for real lists 
and dicts.  Two, for convenience in subclassing  where it's easier to write "self.data[k]" than "dict.__getitem__(self, k)" 
throughout.

It seems that wherever people could have worked with UserString for a mutable string, in practice they prefer to work with lists and 
follow-up with a ''.join() afterwards.


Raymond 

From steven.bethard at gmail.com  Sat Feb  9 22:52:44 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 9 Feb 2008 14:52:44 -0700
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
Message-ID: <d11dcfba0802091352i3e461d3av6560a1f5ff7ecd12@mail.gmail.com>

On Feb 9, 2008 2:33 PM, Raymond Hettinger <python at rcn.com> wrote:
> In the process of getting UserDict updated to subclass from MutableMapping,
> it ended-up in the collections module during the transistion.  Probably, all three
> should end-up in the same place, either in the collections module or in their own
> modules.  What do you guys think?  Do you like them in collections or as
> separate modules?

+1 for putting them in the collections module.

> Also, I'm wondering if it is time for UserString to bite the dust. It isn't really
> useful in the same way a UserDict or UserList which have two principal use
> cases. One, for testing to make sure that simulated dicts and lists are
> substitutable for real lists and dicts.

I thought this was the point of DictMixin, not UserDict, no?

> Two, for convenience in subclassing  where it's easier to write "self.data[k]"
> than "dict.__getitem__(self, k)" throughout.

Interesting.  I'd never seen this before, but a Google code search
reveals that it is reasonably common.  Too bad.  I was hoping that the
new ABCs would eliminated the need for UserDict and UserList
entirely...

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From brett at python.org  Sun Feb 10 02:27:31 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 9 Feb 2008 17:27:31 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802081703w1a73e0catbd1aaecbcf56fb28@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<ca471dc20802081703w1a73e0catbd1aaecbcf56fb28@mail.gmail.com>
Message-ID: <bbaeab100802091727t61d095cdx50300e4e0ccbc255@mail.gmail.com>

On Feb 8, 2008 5:03 PM, Guido van Rossum <guido at python.org> wrote:
> On Feb 8, 2008 4:51 PM, Raymond Hettinger <python at rcn.com> wrote:
> > I recommend dropping the dict.copy() method from Py3.0.
> >
> > * We can already write:  newd = copy.copy(d).
> > * We can also write:     newd = dict(d)
> > * Both of those approaches also work for most other containers.
> > * The collections.Mapping ABC does not support copy().
> > * copy() is not a universal feature of mapping like objects
> > * Lists do not have a copy() method.
> > * If we drop dict.copy(), I'll can also drop set.copy() which is unneeded.
> >
> > Let's make the basic APIs as clean and parallel as possible.
>
> Makes sense.  I request that you also implement the transitional code:
> either a fixer for 2to3 or (perhaps more realistically) a warning to
> dict.copy() when -3 is given.

+1 on the warning over the fixer. Since 2to3 can't do it perfectly I
think it would be better to let users deal with it directly.

-Brett

From brett at python.org  Sun Feb 10 02:32:12 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 9 Feb 2008 17:32:12 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
Message-ID: <bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>

On Feb 9, 2008 1:33 PM, Raymond Hettinger <python at rcn.com> wrote:
> In the process of getting UserDict updated to subclass from MutableMapping, it ended-up in the collections module during the
> transistion.  Probably, all three should end-up in the same place, either in the collections module or in their own modules.  What
> do you guys think?  Do you like them in collections or as separate modules?
>

+1 on collections.

-Brett

From python at rcn.com  Sun Feb 10 02:43:04 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 9 Feb 2008 17:43:04 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
Message-ID: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>

[Brett] 
> +1 on collections.

[Steven Bethard]
> +1 for putting them in the collections module.

How about UserString?
Is there a reason for it to continue to exist?


Raymond

From brett at python.org  Sun Feb 10 02:48:41 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 9 Feb 2008 17:48:41 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
Message-ID: <bbaeab100802091748n5c6f6d33u19affd4fb32197af@mail.gmail.com>

On Feb 9, 2008 5:43 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Brett]
> > +1 on collections.
>
> [Steven Bethard]
> > +1 for putting them in the collections module.
>
> How about UserString?
> Is there a reason for it to continue to exist?

I have never used it so I don't have an opinion on it. But you know
me, I am happy to trim the stdlib down. =)

-Brett

From foom at fuhm.net  Sun Feb 10 02:52:41 2008
From: foom at fuhm.net (James Y Knight)
Date: Sat, 9 Feb 2008 20:52:41 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <bbaeab100802091727t61d095cdx50300e4e0ccbc255@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<ca471dc20802081703w1a73e0catbd1aaecbcf56fb28@mail.gmail.com>
	<bbaeab100802091727t61d095cdx50300e4e0ccbc255@mail.gmail.com>
Message-ID: <39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net>


On Feb 9, 2008, at 8:27 PM, Brett Cannon wrote:

> On Feb 8, 2008 5:03 PM, Guido van Rossum <guido at python.org> wrote:
>> On Feb 8, 2008 4:51 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> I recommend dropping the dict.copy() method from Py3.0.
>>>
>>> * We can already write:  newd = copy.copy(d).
>>> * We can also write:     newd = dict(d)
>>> * Both of those approaches also work for most other containers.
>>> * The collections.Mapping ABC does not support copy().
>>> * copy() is not a universal feature of mapping like objects
>>> * Lists do not have a copy() method.
>>> * If we drop dict.copy(), I'll can also drop set.copy() which is  
>>> unneeded.
>>>
>>> Let's make the basic APIs as clean and parallel as possible.
>>
>> Makes sense.  I request that you also implement the transitional  
>> code:
>> either a fixer for 2to3 or (perhaps more realistically) a warning to
>> dict.copy() when -3 is given.
>
> +1 on the warning over the fixer. Since 2to3 can't do it perfectly I
> think it would be better to let users deal with it directly.

Has anyone thought of making a execution-informed converter? If you  
have a comprehensive test suite, it seems that it ought be possible to  
use the actual execution of the test suite, under a trace hook  
perhaps, or something of that sort, to inform the converter and allow  
it to correctly make changes like this.

James

From guido at python.org  Sun Feb 10 03:35:51 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 9 Feb 2008 18:35:51 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<ca471dc20802081703w1a73e0catbd1aaecbcf56fb28@mail.gmail.com>
	<bbaeab100802091727t61d095cdx50300e4e0ccbc255@mail.gmail.com>
	<39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net>
Message-ID: <ca471dc20802091835ve3aab8eg2e9cf5c728360042@mail.gmail.com>

On Feb 9, 2008 5:52 PM, James Y Knight <foom at fuhm.net> wrote:
> Has anyone thought of making a execution-informed converter? If you
> have a comprehensive test suite, it seems that it ought be possible to
> use the actual execution of the test suite, under a trace hook
> perhaps, or something of that sort, to inform the converter and allow
> it to correctly make changes like this.

Yes, but it seems an overwhelmingly complex task given the available
number of volunteers.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Sun Feb 10 03:51:42 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 9 Feb 2008 19:51:42 -0700
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
Message-ID: <d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>

On Feb 9, 2008 6:43 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Brett]
> > +1 on collections.
>
> [Steven Bethard]
> > +1 for putting them in the collections module.
>
> How about UserString?
> Is there a reason for it to continue to exist?

Like Brett, I've never found a use for it, so I'm happy to see it go.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Sun Feb 10 05:58:35 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 09 Feb 2008 23:58:35 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
Message-ID: <20080210045826.651DD3A40B0@sparrow.telecommunity.com>

At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote:
>I recommend dropping the dict.copy() method from Py3.0.
>
>* We can already write:  newd = copy.copy(d).
>* We can also write:     newd = dict(d)
>* Both of those approaches also work for most other containers.

I'm not fond of this idea.  dict.copy() is polymorphic -- but dict(d) isn't.

copy.copy() is polymorphic, but requires an import, which means the 
path of least resistance will be to use dict(d)...  which will then 
turn non-dictionaries into dictionaries.

If copy.copy() were a builtin and the "one obvious way" to copy 
containers, +1.  Otherwise, -0.5.


From solipsis at pitrou.net  Sun Feb 10 13:53:53 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 10 Feb 2008 12:53:53 +0000 (UTC)
Subject: [Python-3000] Allocation of unicode objects
Message-ID: <loom.20080210T124758-937@post.gmane.org>

Hi,

Since there are discussions going on on the topic of allocation algorithms for
various built-in types, I thought I'd mention there's a patch for turning
unicode objects into variable-sized objects (rather than using a
separately-allocated buffer). The aim is to make allocation of those objects
lighter, and relieve cache and memory pressure a bit.

http://bugs.python.org/issue1943

Marc-Andr? Lemburg expressed skepticism, based on the fact that it made
subclassing unicode objects as part of C extensions more difficult.

And here is a microbenchmark of the thing:

Splitting a small string:
./python -m timeit -s "s=open('INTBENCH', 'r').read()" "s.split()"
-> Unpatched py3k: 26.4 usec per loop
-> PyVarObject patch: 20.2 usec per loop

Splitting a medium-sized string:
./python -m timeit -s "s=open('LICENSE', 'r').read()" "s.split()"
-> Unpatched py3k: 458 usec per loop
-> PyVarObject patch: 316 usec per loop

Splitting a long string:
./python -m timeit -s "s=open('Misc/HISTORY', 'r').read()" "s.split()"
-> Unpatched py3k: 31.3 msec per loop
-> PyVarObject patch: 17.8 msec per loop

Even if the patch is rejected, I think it is important to remember that
implementation characteristics of the unicode type will be crucial for Py3k
performance :-)

Regards

Antoine.



From shiblon at gmail.com  Sun Feb 10 16:54:35 2008
From: shiblon at gmail.com (Chris Monson)
Date: Sun, 10 Feb 2008 10:54:35 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080210045826.651DD3A40B0@sparrow.telecommunity.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
Message-ID: <da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>

- Original message -
I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is...

Speaking of polymorphic, it seems like it would make sense to allow
classes to specify their own __copy__ method for cases where
specialized handling of copy is needed.  copy.copy is sometimes
overzealous, and therefore less efficient than necessary.

Copy is a pretty fundamental idea, and I routinely run up against the
fact that not only is there more than one way to do it (via
constructors, using copy.copy, or when supported, thing.copy), the
only "standard" approach requires an import and is weirdly one size
fits all.

Having a __copy__ slot and a builtin copy function with current
copy.copy behavior as a fallback would be great.

Just my accurately-valued 2 cents.

- C

Sent from Gmail for mobile

On 2/9/08, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote:
> >I recommend dropping the dict.copy() method from Py3.0.
> >
> >* We can already write:  newd = copy.copy(d).
> >* We can also write:     newd = dict(d)
> >* Both of those approaches also work for most other containers.
>
> I'm not fond of this idea.  dict.copy() is polymorphic -- but dict(d) isn't.
>
> copy.copy() is polymorphic, but requires an import, which means the
> path of least resistance will be to use dict(d)...  which will then
> turn non-dictionaries into dictionaries.
>
> If copy.copy() were a builtin and the "one obvious way" to copy
> containers, +1.  Otherwise, -0.5.
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/shiblon%40gmail.com
>

-- 
Sent from Gmail for mobile | mobile.google.com

From guido at python.org  Sun Feb 10 17:22:38 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 10 Feb 2008 08:22:38 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080210045826.651DD3A40B0@sparrow.telecommunity.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
Message-ID: <ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>

On Feb 9, 2008 8:58 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote:
> >I recommend dropping the dict.copy() method from Py3.0.
> >
> >* We can already write:  newd = copy.copy(d).
> >* We can also write:     newd = dict(d)
> >* Both of those approaches also work for most other containers.
>
> I'm not fond of this idea.  dict.copy() is polymorphic -- but dict(d) isn't.
>
> copy.copy() is polymorphic, but requires an import, which means the
> path of least resistance will be to use dict(d)...  which will then
> turn non-dictionaries into dictionaries.
>
> If copy.copy() were a builtin and the "one obvious way" to copy
> containers, +1.  Otherwise, -0.5.

On second thought, I'm with Phillip here, for exactly the reasons he
states. Let's leave copy() alone.

I'm of two minds about adding it to the MutableMapping and MutableSet
ABCs; requiring it might constrain implementations unnecessary, and
there isn't an easy way to provide a generic implementation. Now I
wish we had "optional" abstract methods. Maybe those can be added to
the ABC PEP yet?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Feb 10 17:24:32 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 10 Feb 2008 08:24:32 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
Message-ID: <ca471dc20802100824t57b6eebcl296dcc79c4c5a59a@mail.gmail.com>

On Feb 10, 2008 7:54 AM, Chris Monson <shiblon at gmail.com> wrote:
> Speaking of polymorphic, it seems like it would make sense to allow
> classes to specify their own __copy__ method for cases where
> specialized handling of copy is needed.  copy.copy is sometimes
> overzealous, and therefore less efficient than necessary.

Good idea! I've fired up the time machine, and fixed the world.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Feb 10 17:27:52 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 10 Feb 2008 08:27:52 -0800
Subject: [Python-3000] Allocation of unicode objects
In-Reply-To: <loom.20080210T124758-937@post.gmane.org>
References: <loom.20080210T124758-937@post.gmane.org>
Message-ID: <ca471dc20802100827i33eadfa0o36e76a1ab07b90dc@mail.gmail.com>

On Feb 10, 2008 4:53 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Since there are discussions going on on the topic of allocation algorithms for
> various built-in types, I thought I'd mention there's a patch for turning
> unicode objects into variable-sized objects (rather than using a
> separately-allocated buffer). The aim is to make allocation of those objects
> lighter, and relieve cache and memory pressure a bit.
>
> http://bugs.python.org/issue1943
>
> Marc-Andr? Lemburg expressed skepticism, based on the fact that it made
> subclassing unicode objects as part of C extensions more difficult.

Has anybody ever tried that? The same would apply to PyString and I've
never heard this complaint. I think that given the relative importance
of fast strings in Py3k vs. the convenience of subclassing PyUnicode,
the latter may have to suffer.

> And here is a microbenchmark of the thing:
>
> Splitting a small string:
> ./python -m timeit -s "s=open('INTBENCH', 'r').read()" "s.split()"
> -> Unpatched py3k: 26.4 usec per loop
> -> PyVarObject patch: 20.2 usec per loop
>
> Splitting a medium-sized string:
> ./python -m timeit -s "s=open('LICENSE', 'r').read()" "s.split()"
> -> Unpatched py3k: 458 usec per loop
> -> PyVarObject patch: 316 usec per loop
>
> Splitting a long string:
> ./python -m timeit -s "s=open('Misc/HISTORY', 'r').read()" "s.split()"
> -> Unpatched py3k: 31.3 msec per loop
> -> PyVarObject patch: 17.8 msec per loop
>
> Even if the patch is rejected, I think it is important to remember that
> implementation characteristics of the unicode type will be crucial for Py3k
> performance :-)

Right. I haven't had enough time to review this (or any other patch),
but the idea is very appealing.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Sun Feb 10 18:33:21 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 10 Feb 2008 10:33:21 -0700
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
Message-ID: <d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>

On Feb 10, 2008 9:22 AM, Guido van Rossum <guido at python.org> wrote:
> On Feb 9, 2008 8:58 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote:
> > >I recommend dropping the dict.copy() method from Py3.0.
> > >
> > >* We can already write:  newd = copy.copy(d).
> > >* We can also write:     newd = dict(d)
> > >* Both of those approaches also work for most other containers.
> >
> > I'm not fond of this idea.  dict.copy() is polymorphic -- but dict(d) isn't.
> >
> > copy.copy() is polymorphic, but requires an import, which means the
> > path of least resistance will be to use dict(d)...  which will then
> > turn non-dictionaries into dictionaries.
> >
> > If copy.copy() were a builtin and the "one obvious way" to copy
> > containers, +1.  Otherwise, -0.5.
>
> On second thought, I'm with Phillip here, for exactly the reasons he
> states. Let's leave copy() alone.

Maybe copy.copy() could become __builtin__.copy()?  I know everyone
hates adding builtins, but it is commonly needed functionality, it has
__special_method__ support, and putting it in builtins would allow us
to simplify the set() and dict() APIs...

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From skip at pobox.com  Sun Feb 10 20:42:45 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 10 Feb 2008 13:42:45 -0600
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
Message-ID: <18351.21429.600838.877694@montanaro-dyndns-org.local>


    Steven> Maybe copy.copy() could become __builtin__.copy()?  

And what of copy.depcopy()?  Doesn't seem to be much else public defined in
the copy module:

    % python -c 'import copy ; print copy.__all__'
    ['Error', 'copy', 'deepcopy']

If copy.copy() migrates to builtin it should raise something other than
copy.Error as well.

Skip

From python at rcn.com  Sun Feb 10 20:55:45 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 10 Feb 2008 11:55:45 -0800
Subject: [Python-3000] Nix dict.copy()
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
Message-ID: <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>

[Phillip Eby]
> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is...

Can't say dict.copy() is really polymorphic if only one other class defines the method.

The __copy__ method is more common.  The only wrinkle is that the copy() function isn't a builtin.


Raymond

From larry at hastings.org  Sun Feb 10 21:11:00 2008
From: larry at hastings.org (Larry Hastings)
Date: Sun, 10 Feb 2008 12:11:00 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
Message-ID: <47AF5A54.1020701@hastings.org>

Steven Bethard wrote:
> Maybe copy.copy() could become __builtin__.copy()?
+1 for exactly the reasons cited.  I think copy() and deepcopy() should 
both be "essential" built-in functions.


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/1be73f4b/attachment.htm 

From lists at cheimes.de  Sun Feb 10 21:49:03 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 10 Feb 2008 21:49:03 +0100
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <47AF5A54.1020701@hastings.org>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org>
Message-ID: <fonnvv$dl0$1@ger.gmane.org>

Larry Hastings wrote:
> +1 for exactly the reasons cited.  I think copy() and deepcopy() should
> both be "essential" built-in functions.

I'm -0 on copy and -1 on deepcopy.

If you need a copy or a deepcopy of an object (except dicts, lists and
sets) you are most certainly using the wrong approach. Back when I was
learning Python over 5 years ago I was using copy and deepcopy often.
Nowadays I don't need the copy module anymore because I've learned how
to design software without the need for copies.

In my humble opinion the addition of copy to builtins leaves a bad mark.

Christian


From martin at v.loewis.de  Sun Feb 10 21:55:09 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Feb 2008 21:55:09 +0100
Subject: [Python-3000] Allocation of unicode objects
In-Reply-To: <loom.20080210T124758-937@post.gmane.org>
References: <loom.20080210T124758-937@post.gmane.org>
Message-ID: <47AF64AD.3040204@v.loewis.de>

> Since there are discussions going on on the topic of allocation algorithms for
> various built-in types, I thought I'd mention there's a patch for turning
> unicode objects into variable-sized objects (rather than using a
> separately-allocated buffer). The aim is to make allocation of those objects
> lighter, and relieve cache and memory pressure a bit.

I find that fairly reasonable, so I'm +1 for this approach (haven't 
studied the patch, though). I always wondered why Unicode objects
didn't work the way string objects work, to begin with.

Regards,
Martin

From martin at v.loewis.de  Sun Feb 10 22:01:42 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 10 Feb 2008 22:01:42 +0100
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
Message-ID: <47AF6636.70104@v.loewis.de>

Raymond Hettinger wrote:
> [Phillip Eby]
>> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is...
> 
> Can't say dict.copy() is really polymorphic if only one other class defines the method.

Why do you say it's only one? I found atleast UserDict.copy, 
os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy,
and test.test_userdict.SeqDict. Notice that UserDict.copy is also
polymorphic in the sense that it may return instances of subclasses,
by using copy.copy.

Regards,
Martin


From python at rcn.com  Sun Feb 10 22:10:48 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 10 Feb 2008 13:10:48 -0800
Subject: [Python-3000] Nix dict.copy()
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
Message-ID: <00e701c86c29$69745630$6800a8c0@RaymondLaptop1>

>> [Phillip Eby]
>>> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is...

[Raymond]
>> Can't say dict.copy() is really polymorphic if only one other class defines the method.

[Martin]
> Why do you say it's only one? I found atleast UserDict.copy, os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy,

It would have been better for me to say that copy() seems to appear only in variants of dicts, but __copy__() is a long-standing 
defined protocol that is supported across non-mapping types as well.

Underneath the hood, __copy__() and copy() have the same implementation, so it's really a question of whether we want two ways to 
say the same thing (with one of those ways being polymorphic across many objects and the other being used mainly for dict variants).

Had copy() been a builtin, we probably wouldn't be having this discussion.


Raymond

From python at rcn.com  Sun Feb 10 22:01:00 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 10 Feb 2008 13:01:00 -0800
Subject: [Python-3000] __reduce__() and __reduce_ex__()
Message-ID: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>

In Py3.0, would it be worth it to simplify this protocol by combining these two methods?  Always pass in a proto argument and the 
method can ignore it if it wants.


Raymond 

From rhamph at gmail.com  Sun Feb 10 22:39:22 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 10 Feb 2008 14:39:22 -0700
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <fonnvv$dl0$1@ger.gmane.org>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
Message-ID: <aac2c7cb0802101339p66bc7851r8dde5cdc69a5083b@mail.gmail.com>

On Feb 10, 2008 1:49 PM, Christian Heimes <lists at cheimes.de> wrote:
> Larry Hastings wrote:
> > +1 for exactly the reasons cited.  I think copy() and deepcopy() should
> > both be "essential" built-in functions.
>
> I'm -0 on copy and -1 on deepcopy.
>
> If you need a copy or a deepcopy of an object (except dicts, lists and
> sets) you are most certainly using the wrong approach. Back when I was
> learning Python over 5 years ago I was using copy and deepcopy often.
> Nowadays I don't need the copy module anymore because I've learned how
> to design software without the need for copies.
>
> In my humble opinion the addition of copy to builtins leaves a bad mark.

-1 on both.

If a dict is wrong for some input, it's probably wrong for all inputs.
 If you're keeping this new dict for your own uses then you clearly
want to keep it as your preferred type, and if you're returning it to
your caller the preference for builtins is clearly want a single
standardized return type (even filter does this in 3.0).

I'm having a hard time imagining cases where you *really* want to
return a copy of whatever the original type was, and this makes me
think we shouldn't encourage that coding style.

dict(iterable) for the one, and only one, obvious way to do it.

-- 
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Sun Feb 10 22:51:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 10 Feb 2008 22:51:44 +0100
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
Message-ID: <47AF71F0.4080301@v.loewis.de>

>>>> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) 
>>>> is...
> 
> [Raymond]
>>> Can't say dict.copy() is really polymorphic if only one other class 
>>> defines the method.
> 
> [Martin]
>> Why do you say it's only one? I found atleast UserDict.copy, 
>> os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy,
> 
> It would have been better for me to say that copy() seems to appear only 
> in variants of dicts, but __copy__() is a long-standing defined protocol 
> that is supported across non-mapping types as well.

But that's Phillip's point: dict(d) is not polymorphic, yet d.copy()
is. That's important in applications where you want to copy a dict-like
object polymorphically. So indeed, your claim that .copy "is [not]
really polymorphic" is quite puzzling - we now seem to agree that it
*is* polymorphic.

So going back to your original message, you should remove dict(d)
from the list of alternatives for dict.copy() (or remark that this
will be a copy-to-basetype, so not quite the same thing).

> Had copy() been a builtin, we probably wouldn't be having this discussion.

Interestingly enough, dict.copy was added *after* copy.copy (in r8135),
which supported __copy__ right from the beginning (since r4555). Even
more interesting, it was added along with dict.absorb, which then
got renamed dict.update a week later. No rationale is given in the
checkin message on the purpose of the copy method.

Regards,
Martin

From python at rcn.com  Sun Feb 10 23:14:51 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 10 Feb 2008 14:14:51 -0800
Subject: [Python-3000] Nix dict.copy()
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
Message-ID: <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>

[Martin]
> So indeed, your claim that .copy "is [not]
> really polymorphic" is quite puzzling - we now seem to agree that it
> *is* polymorphic.

Yes, of course.  Every method used by dict imitators could be considered polymophic.  I was just observing copy.copy(obj) applies to 
many, many more object types than obj.copy().

The former works with lists, tuples, deques, strings, and all kinds of things.  The latter shows-up mainly in dict imitators.


> So going back to your original message, you should remove dict(d)
> from the list of alternatives for dict.copy() (or remark that this
> will be a copy-to-basetype, so not quite the same thing).

That makes sense.  You can never be sure of a constructor's signature.  It works most of the time but not all of the time.


>> Had copy() been a builtin, we probably wouldn't be having this discussion.
>
> Interestingly enough, dict.copy was added *after* copy.copy (in r8135),
> which supported __copy__ right from the beginning (since r4555). Even
> more interesting, it was added along with dict.absorb, which then
> got renamed dict.update a week later. No rationale is given in the
> checkin message on the purpose of the copy method.

Interesting history.

Now, we have one and only one chance to slim-down the mapping API, leaving the copy() function as the one, universal, preferred way 
to do it.  I don't think this chance will come again.


Raymond 

From p.f.moore at gmail.com  Mon Feb 11 00:46:25 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 10 Feb 2008 23:46:25 +0000
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
	<000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
Message-ID: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>

On 10/02/2008, Raymond Hettinger <python at rcn.com> wrote:
> Now, we have one and only one chance to slim-down the mapping API, leaving the copy()
> function as the one, universal, preferred way to do it.  I don't think this chance will come
> again.

This discussion confused me (largely because of the subtle differences
between the various ways of copying a dictionary).

If I want a builtin dictionary from an arbitrary mapping d, I'd use
dict(d.items()) - even if I knew d was a real dictionary. I wouldn't
use d.copy() as that would not guarantee I had a builtin dict, should
someone later pass me a user defined type.

If I want a copy of an arbitrary mapping (or any object) and I want to
preserve type, I would use copy.copy(). This is just as polymorphic as
a copy method (interesting that it is Phillip arguing for methods
being the polymorphic choice, when copy.copy is a generic function,
which he has previously argued as being better...) I'm not sure how
often I'd want to preserve type like this. It's never come up for me
in real code. So I don't mind importing the copy module to get the
functionality, it reminds me I'm doing something a little unusual. (On
the other hand, (shallow) copy seems fundamental enough to me to
qualify as a builtin, should someone care enough to argue for that).

Given the above, I see no case where I'd want to use dict.copy(). So
I'm in favour of removing it.

On the other hand, I'd imagine it would be difficult to write a good
fixer for this. I'm not sure how much of a showstopper that might be.

Paul.

From guido at python.org  Mon Feb 11 01:18:09 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 10 Feb 2008 16:18:09 -0800
Subject: [Python-3000] __reduce__() and __reduce_ex__()
In-Reply-To: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>
References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802101618l24d50139gd0a914c3f877f672@mail.gmail.com>

On Feb 10, 2008 1:01 PM, Raymond Hettinger <python at rcn.com> wrote:
> In Py3.0, would it be worth it to simplify this protocol by combining these two methods?  Always pass in a proto argument and the
> method can ignore it if it wants.

Yeah -- the _ex suffix on the name suggests that that's the API we
wanted but we'd already released a version with a suboptimal API...
:-( There are other uses of 'ex' or 'Ex' on APIs too I believe (maybe
in the C API). It may well be a convention I borrowed from Microsoft.
Any of these that you find are likely cleanup candidates.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Mon Feb 11 01:24:00 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 10 Feb 2008 16:24:00 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <fonnvv$dl0$1@ger.gmane.org>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
Message-ID: <20080211002400.GA2720@panix.com>

On Sun, Feb 10, 2008, Christian Heimes wrote:
> Larry Hastings wrote:
>>
>> +1 for exactly the reasons cited.  I think copy() and deepcopy() should
>> both be "essential" built-in functions.
> 
> I'm -0 on copy and -1 on deepcopy.
> 
> If you need a copy or a deepcopy of an object (except dicts, lists and
> sets) you are most certainly using the wrong approach. Back when I was
> learning Python over 5 years ago I was using copy and deepcopy often.
> Nowadays I don't need the copy module anymore because I've learned how
> to design software without the need for copies.

Let's suppose you have an object that represents an order.  And let's
suppose that this object needs to be copied to create a re-order (letting
the customer preserve the information from the original order).  How do
you suggest that the re-order be created without copy(), and especially
deepcopy()?  What if an attribute of orders can be a list or dict?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From digitalxero at gmail.com  Mon Feb 11 01:29:12 2008
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Sun, 10 Feb 2008 16:29:12 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
	<000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
	<79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
Message-ID: <e9764b730802101629q46290cc9mdf96a9fcb8beb188@mail.gmail.com>

On Feb 10, 2008 3:46 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On the other hand, I'd imagine it would be difficult to write a good
> fixer for this. I'm not sure how much of a showstopper that might be.

Cant you just flag it as depreciated and have it actually call
copy.copy() when it is run. That way you dont need to write a fixer
for it and can remove it in 3.1

From greg.ewing at canterbury.ac.nz  Mon Feb 11 01:54:21 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Feb 2008 13:54:21 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
	<000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
	<79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
Message-ID: <47AF9CBD.1050504@canterbury.ac.nz>

Paul Moore wrote:

> Given the above, I see no case where I'd want to use dict.copy(). So
> I'm in favour of removing it.

Personally I don't think I would even notice if the entire
contents of the copy module disappeared. I've never used
either form of generic copying. In the rare cases when I
do want to copy something, I know what type I'm dealing
with and have specific ideas on how to copy it, so I
write code accordingly.

Generic copying in Python seems like an anti-pattern to
me, and not something that should be encouraged.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Mon Feb 11 01:57:18 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Feb 2008 13:57:18 +1300
Subject: [Python-3000] __reduce__() and __reduce_ex__()
In-Reply-To: <ca471dc20802101618l24d50139gd0a914c3f877f672@mail.gmail.com>
References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>
	<ca471dc20802101618l24d50139gd0a914c3f877f672@mail.gmail.com>
Message-ID: <47AF9D6E.5080709@canterbury.ac.nz>

Guido van Rossum wrote:
> Yeah -- the _ex suffix on the name suggests that that's the API we
> wanted but we'd already released a version with a suboptimal API...

Hmmm, that convention isn't very scalable. What happens
when you update the API again -- do you use an _ex_ex
suffix? :-)

Also it tends to make things look unpleasanty Microsoftish. :-(

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Mon Feb 11 02:01:15 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Feb 2008 14:01:15 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080211002400.GA2720@panix.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
	<20080211002400.GA2720@panix.com>
Message-ID: <47AF9E5B.8010902@canterbury.ac.nz>

Aahz wrote:

> Let's suppose you have an object that represents an order.  And let's
> suppose that this object needs to be copied to create a re-order (letting
> the customer preserve the information from the original order).

In that case, I would give my Order class a copy() method (or
re_order() or whatever name is appropriate) that does the right
thing for orders in the context of that application.

There are bound to be things that you *don't* want to copy from
the original order, e.g. the order ID, the date...

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Mon Feb 11 02:22:01 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 10 Feb 2008 17:22:01 -0800
Subject: [Python-3000] __reduce__() and __reduce_ex__()
In-Reply-To: <47AF9D6E.5080709@canterbury.ac.nz>
References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>
	<ca471dc20802101618l24d50139gd0a914c3f877f672@mail.gmail.com>
	<47AF9D6E.5080709@canterbury.ac.nz>
Message-ID: <ca471dc20802101722j441d4aa1h9ba75e7cea0da861@mail.gmail.com>

On Feb 10, 2008 4:57 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
> > Yeah -- the _ex suffix on the name suggests that that's the API we
> > wanted but we'd already released a version with a suboptimal API...
>
> Hmmm, that convention isn't very scalable. What happens
> when you update the API again -- do you use an _ex_ex
> suffix? :-)
>
> Also it tends to make things look unpleasanty Microsoftish. :-(

Whine, whine.

Do you have a better idea?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Feb 11 03:02:22 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 10 Feb 2008 21:02:22 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.co
 m>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
	<000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
	<79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
Message-ID: <20080211020218.E32A23A4075@sparrow.telecommunity.com>

At 11:46 PM 2/10/2008 +0000, Paul Moore wrote:
>If I want a copy of an arbitrary mapping (or any object) and I want to
>preserve type, I would use copy.copy(). This is just as polymorphic as
>a copy method (interesting that it is Phillip arguing for methods
>being the polymorphic choice, when copy.copy is a generic function,
>which he has previously argued as being better...)

Actually, the issue here is the import, not methods vs. 
functions.  Otherwise, I'd be arguing for ob.len().  :)

And if we were starting from square one with no existing code, I'd 
probably be okay with needing to import copy, as long as that was the 
publicized One Obvious Way to copy all things.

However, since existing code has to be migrated, and lots of things 
have copy() methods, and 2to3 isn't going to be able to tell, 
practicality (IMO) seems to favor keeping the existing method.


From larry at hastings.org  Mon Feb 11 03:24:56 2008
From: larry at hastings.org (Larry Hastings)
Date: Sun, 10 Feb 2008 18:24:56 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <fonnvv$dl0$1@ger.gmane.org>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>	<47AF5A54.1020701@hastings.org>
	<fonnvv$dl0$1@ger.gmane.org>
Message-ID: <47AFB1F8.8050803@hastings.org>

Christian Heimes wrote:
> Larry Hastings wrote:
>   
>> +1 for exactly the reasons cited.  I think copy() and deepcopy() should
>> both be "essential" built-in functions.
>>     
>
> I'm -0 on copy and -1 on deepcopy.
>
> If you need a copy or a deepcopy of an object (except dicts, lists and
> sets) you are most certainly using the wrong approach.

My +1 isn't based primarily on how often I would use copy/deepcopy, as I 
don't need them very often.  (Although I do use it *ever*, unlike many 
builtins--complex(), divmod(), and reversed() just to name three.)  
Instead, it's based on the observation that copy and deepcopy seem to be 
a first-class part of the language.  The following built-in classes 
implement __copy__; those that also implement __deepcopy__ are marked 
with an asterisk:
    array.array*
    collections.defaultdict
    collections.deque
    decimal.Decimal
    xml.etree.ElementTree*
    itertools.tee
    re.match*
    re.pattern*
All of those are implemented in C except for decimal.Decimal.  This list 
was compiled from Python 3.0a2.

I pass no judgment on programmers using copy/deepcopy.  Though I note 
that, if you are passed a merely dict-like / list-like / set-like 
object, I would feel more confident in using copy.copy() to copy it than 
using dict() / list() / set() to copy it; enforcing the type this way 
seems so un-duck-typing to me.  I suppose I could go fishing in the 
object to see if it implements a copy() method (dicts and sets do, lists 
don't), but then the deprecation/removal of .copy() methods in favor of 
copy.copy was how this whole thread got started.

And, needless to say, there is simply no substitute for copy.deepcopy(),


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/25f1d4f7/attachment.htm 

From daniel at stutzbachenterprises.com  Mon Feb 11 03:38:34 2008
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 10 Feb 2008 20:38:34 -0600
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080211020218.E32A23A4075@sparrow.telecommunity.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<da3f900e0802100754r5a3d1fccx1ca1c9a5c3062a6b@mail.gmail.com>
	<00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1>
	<47AF6636.70104@v.loewis.de>
	<00e701c86c29$69745630$6800a8c0@RaymondLaptop1>
	<47AF71F0.4080301@v.loewis.de>
	<000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1>
	<79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com>
	<20080211020218.E32A23A4075@sparrow.telecommunity.com>
Message-ID: <eae285400802101838gdc4341bp15c34bd4fa7a8f5d@mail.gmail.com>

On Feb 10, 2008 8:02 PM, Phillip J. Eby <pje at telecommunity.com> wrote:

> However, since existing code has to be migrated, and lots of things
> have copy() methods, and 2to3 isn't going to be able to tell,
> practicality (IMO) seems to favor keeping the existing method.
>

If it only converts dict and set .copy() methods, 2to3 can't tell, but it
wouldn't be too hard to respell all instances of foo.copy() as copy(foo).
Obviously, this is a bit more radical and has good and bad points.  It'd be
giving every class's .copy method a promotion to __copy__, basically.

Suggested 2to3 operation:

foo.copy() => copy(foo)
def copy(self):    =>  def __copy__(self):     (if within a class
definition)

If __copy__(self) is just "self.copy()", remove it.
If copy(self) is just "self.__copy__()" or "copy.copy(self)", remove it
If a class still defines both __copy__ and copy(), print a warning/error
(this should be rare).

If copy is called with arguments or defined with more than the "self"
argument, make no change.
If copy is defined with more than the "self" argument, but all the arguments
are optional, print a warning/error.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/f4ff421b/attachment-0001.htm 

From lists at cheimes.de  Mon Feb 11 02:14:19 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 11 Feb 2008 02:14:19 +0100
Subject: [Python-3000] __reduce__() and __reduce_ex__()
In-Reply-To: <47AF9D6E.5080709@canterbury.ac.nz>
References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1>	<ca471dc20802101618l24d50139gd0a914c3f877f672@mail.gmail.com>
	<47AF9D6E.5080709@canterbury.ac.nz>
Message-ID: <foo9hv$sn7$1@ger.gmane.org>

Greg Ewing wrote:
> Hmmm, that convention isn't very scalable. What happens
> when you update the API again -- do you use an _ex_ex
> suffix? :-)

How about a _ng suffix (next generation)? *scnr*

Christian


From aahz at pythoncraft.com  Mon Feb 11 04:44:19 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 10 Feb 2008 19:44:19 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <47AF9E5B.8010902@canterbury.ac.nz>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
	<20080211002400.GA2720@panix.com>
	<47AF9E5B.8010902@canterbury.ac.nz>
Message-ID: <20080211034419.GA10737@panix.com>

On Mon, Feb 11, 2008, Greg Ewing wrote:
> Aahz wrote:
>> 
>> Let's suppose you have an object that represents an order.  And let's
>> suppose that this object needs to be copied to create a re-order (letting
>> the customer preserve the information from the original order).
> 
> In that case, I would give my Order class a copy() method (or
> re_order() or whatever name is appropriate) that does the right
> thing for orders in the context of that application.
> 
> There are bound to be things that you *don't* want to copy from
> the original order, e.g. the order ID, the date...

Certainly -- that's why __copy__() exists, right?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From greg.ewing at canterbury.ac.nz  Mon Feb 11 05:23:26 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Feb 2008 17:23:26 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <20080211034419.GA10737@panix.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
	<20080211002400.GA2720@panix.com> <47AF9E5B.8010902@canterbury.ac.nz>
	<20080211034419.GA10737@panix.com>
Message-ID: <47AFCDBE.50804@canterbury.ac.nz>

Aahz wrote:
> On Mon, Feb 11, 2008, Greg Ewing wrote:
> 
>>There are bound to be things that you *don't* want to copy from
>>the original order, e.g. the order ID, the date...
> 
> Certainly -- that's why __copy__() exists, right?

Yes, but then I don't see the advantage over just
giving the object a copy() method and calling it
directly.

In other words, I see little benefit in having
copy() be a generic function.

-- 
Greg



From larry at hastings.org  Mon Feb 11 08:55:43 2008
From: larry at hastings.org (Larry Hastings)
Date: Sun, 10 Feb 2008 23:55:43 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <47AFCDBE.50804@canterbury.ac.nz>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>	<47AF5A54.1020701@hastings.org>
	<fonnvv$dl0$1@ger.gmane.org>	<20080211002400.GA2720@panix.com>
	<47AF9E5B.8010902@canterbury.ac.nz>	<20080211034419.GA10737@panix.com>
	<47AFCDBE.50804@canterbury.ac.nz>
Message-ID: <47AFFF7F.6090803@hastings.org>

Greg Ewing wrote:
> Yes, but then I don't see the advantage over just giving the object a copy() method and calling it directly.
>
> In other words, I see little benefit in having copy() be a generic function.
>   

So true!  Other candidates for not being generic functions: len(), 
repr(), str(), int(), complex(), dict(), list(), set()...

So what decides whether something should be a generic function or a 
commonly-defined member function?  Personally I find "historical 
accident" an unsatisfying answer.


/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/ae20c536/attachment.htm 

From pje at telecommunity.com  Mon Feb 11 18:56:15 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 11 Feb 2008 12:56:15 -0500
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <47AFCDBE.50804@canterbury.ac.nz>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<47AF5A54.1020701@hastings.org> <fonnvv$dl0$1@ger.gmane.org>
	<20080211002400.GA2720@panix.com>
	<47AF9E5B.8010902@canterbury.ac.nz>
	<20080211034419.GA10737@panix.com>
	<47AFCDBE.50804@canterbury.ac.nz>
Message-ID: <20080211175606.4AE1C3A4075@sparrow.telecommunity.com>

At 05:23 PM 2/11/2008 +1300, Greg Ewing wrote:
>Aahz wrote:
> > On Mon, Feb 11, 2008, Greg Ewing wrote:
> >
> >>There are bound to be things that you *don't* want to copy from
> >>the original order, e.g. the order ID, the date...
> >
> > Certainly -- that's why __copy__() exists, right?
>
>Yes, but then I don't see the advantage over just
>giving the object a copy() method and calling it
>directly.
>
>In other words, I see little benefit in having
>copy() be a generic function.

Well, in the current circumstance, it helps because third parties can 
implement copying for things that didn't implement it themselves.


From dalcinl at gmail.com  Tue Feb 12 00:19:54 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 11 Feb 2008 20:19:54 -0300
Subject: [Python-3000] Thoughts on collections.Container and
	collections.Iterable
In-Reply-To: <47AD3A60.40404@cs.byu.edu>
References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net>
	<ca471dc20802081742y3c6c6790wa5cc9a0f61868dd3@mail.gmail.com>
	<47AD3451.6000609@cs.byu.edu> <47AD3A60.40404@cs.byu.edu>
Message-ID: <e7ba66e40802111519g9517305p12e99e65123122bd@mail.gmail.com>

itertools.count() maybe ? Well not really infinite, it stops with
OverflowError...

But then (don't try this at home)...

>>> 4 in itertools.cycle(range(3)) # ups!

I seems that implicitely supporting containment testing for any
iterable is not always good idea.

On 2/9/08, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> Neil Toronto wrote:
> > Guido van Rossum wrote:
> >> I would need to think more about this. I'm tempted not to do this, and
> >> let these ABCs denote the *explicit* presence of __contains__ and
> >> __iter__, respectively. Something that's iterable but doesn't
> >> implement __contains__ supports the 'in' operator very inefficiently
> >> (through linear search) which we might not want to encourage.
> > It could be worse. Is a container necessarily finite?
>
> I meant an iterable, of course, not a container.
>
> Neil
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/dalcinl%40gmail.com
>


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From python at rcn.com  Tue Feb 12 01:43:54 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 19:43:54 -0500 (EST)
Subject: [Python-3000] ABC for Inequalities
Message-ID: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net>

Was some thought given to providing a mixin for boolean inequalities in total orderings (define __le__ and get the rest for free)?

One of the motivating examples in the ABC pep was that the presence of __getitem__ was insufficient to distinguish between a sequence and a mapping.  By registering one of the collections ABCs, a class can make an affirmative declaration that it is either a mapping or a sequence.

It seems that there is a similar issue with inequalities.  If something defines __le__, you don't necessarily know whether it returns a boolean (it could return an array of bools for a vector comparison).  Even if a bool is returned, it is not clear whether it implies sortable ordering or whether is something with a completely different meaning (i.e. set.issubset).


Raymond

From guido at python.org  Tue Feb 12 02:02:27 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 17:02:27 -0800
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net>
References: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802111702q300db03bief4d5d29ad13a313@mail.gmail.com>

On Feb 11, 2008 4:43 PM, Raymond Hettinger <python at rcn.com> wrote:
> Was some thought given to providing a mixin for boolean inequalities in total orderings (define __le__ and get the rest for free)?

I think it was briefly mentioned but we didn't get beyond the stage of
"you can do this with a tiny metaclass". There should be a way to
avoid getting the rest for free too though, as the example of sets
shows. There was also the issue of what to do if a subclass defines
only __le__ and a superclass already defined all 6 operators. And
there's also the issue that because of the way it's mapped to a single
tp_richcompare slot in C, 'object' defines all 6 operators, but 4 of
these raise exceptions. IOW it's messy.

> One of the motivating examples in the ABC pep was that the presence of __getitem__ was insufficient to distinguish between a sequence and a mapping.  By registering one of the collections ABCs, a class can make an affirmative declaration that it is either a mapping or a sequence.
>
> It seems that there is a similar issue with inequalities.  If something defines __le__, you don't necessarily know whether it returns a boolean (it could return an array of bools for a vector comparison).  Even if a bool is returned, it is not clear whether it implies sortable ordering or whether is something with a completely different meaning (i.e. set.issubset).

I'm not sure I follow the connection with the above query, but I
believe that for now the best approach here is to document the
requirements for  the comparisons. E.g. for Real numbers we require a
total ordering based on the standard ordering for mathematical reals
(insofar as practical in the light of inexactness and NaNs, of
course), for Sequences I believe we specify that it should order
item-wise, and for Sets we redefine the comparisons as set inclusion
operators.

A class whose __le__ returns an array or results should probably not
derive directly from one of those standard ABCs. (Possibly the numpy
folks might be interested in defining a different ABC for arrays whose
'+' operator does elementwise addition instead of concatenation; but
that's up to them.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Feb 12 02:45:01 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 20:45:01 -0500 (EST)
Subject: [Python-3000] ABC for Inequalities
Message-ID: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>

>> Was some thought given to providing a mixin for boolean 
>> inequalities in total orderings (define __le__ and get 
>> the rest for free)

[GvR]
> IOW it's messy. 

Would it make sense to do something in numbers.py modeled
after Exact/InExact?  Those don't have any behavior; they
just make a statement about semantics.


Raymond


---------------------------
class TotalOrdering(object):
    '''Equality and inequality operations on this type are guaranteed to
    return boolean values and to imply a total ordering where the relations
    are transitive (a<b and b<c implies a<c) and the operators have the
    usual interrelationships:  (a<b) == (b>a) == (not a>=b) == (not b<=a).
    '''
    @abstractmethod
    def __eq__(self, other): pass

    @abstractmethod
    def __ne__(self, other): pass
    
    @abstractmethod
    def __lt__(self, other): pass

    @abstractmethod
    def __le__(self, other): pass

    @abstractmethod
    def __gt__(self, other): pass

    @abstractmethod
    def __ge__(self, other): pass

From python at rcn.com  Tue Feb 12 03:58:27 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 21:58:27 -0500 (EST)
Subject: [Python-3000] What is numbers.Integral used for?
Message-ID: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net>

During the discussion of PEP 3141, what use cases were envisioned for the Integral type?  Were there some examples of concrete implementations other than <type 'int'> or <type 'long'??

I'm asking because I'm unclear about what a class is stating about itself when it registers as Integral.  I'm not sure that most finite fields would fit into the structure since they all Integrals support logical operations, are convertible into ints via __index__, andy the must support all of the operations defined for Reals.

I looked at the reference for Scheme's numeric tower to find guidance, but it seemed that the notion expressed solely through the "integer?" predicate which behaves differently than our inheritance tower.  For example, the spec says (rational? 6/10) ==> #t and (integer? 8/4) ==> #t but in our current implementation Rational(8,4) is not Integral like it is for Scheme.

Also, Scheme seems apply implement the notions of Exact/Inexact much differently
than we do: "A number is exact if it was written as an exact constant or was derived from exact numbers using only exact operations."  So, while all integers are exact, some floats and decimal constants could also be exact.  This matches how exactness is used in the decimal module -- all constants start as exact and operations can return either exact or inexact (and the module signals when this occurs).  Our module says Integrals are Exact and makes no further assertions.


Raymond


From guido at python.org  Tue Feb 12 04:08:58 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 19:08:58 -0800
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>

Hold on. We went down that road quite a bit and then discovered it
wasn't going to work. The problem is that if you have two instance x,
y such that isinstance(x, TotalOrdering) and isinstance(y,
TotalOrdering) you still don't know if x < y is defined. (For example,
if x is a string and y is a number.) We ended up deciding that such an
ABC was not useful. There's a useful definition in there somewhere but
it's elusive. We'll eventually figure it out.

On Feb 11, 2008 5:45 PM, Raymond Hettinger <python at rcn.com> wrote:
> >> Was some thought given to providing a mixin for boolean
> >> inequalities in total orderings (define __le__ and get
> >> the rest for free)
>
> [GvR]
> > IOW it's messy.
>
> Would it make sense to do something in numbers.py modeled
> after Exact/InExact?  Those don't have any behavior; they
> just make a statement about semantics.
>
>
> Raymond
>
>
> ---------------------------
> class TotalOrdering(object):
>     '''Equality and inequality operations on this type are guaranteed to
>     return boolean values and to imply a total ordering where the relations
>     are transitive (a<b and b<c implies a<c) and the operators have the
>     usual interrelationships:  (a<b) == (b>a) == (not a>=b) == (not b<=a).
>     '''
>     @abstractmethod
>     def __eq__(self, other): pass
>
>     @abstractmethod
>     def __ne__(self, other): pass
>
>     @abstractmethod
>     def __lt__(self, other): pass
>
>     @abstractmethod
>     def __le__(self, other): pass
>
>     @abstractmethod
>     def __gt__(self, other): pass
>
>     @abstractmethod
>     def __ge__(self, other): pass
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 04:13:40 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 19:13:40 -0800
Subject: [Python-3000] What is numbers.Integral used for?
In-Reply-To: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net>
References: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802111913r7b2bbfb5we12badfbcae16547@mail.gmail.com>

On Feb 11, 2008 6:58 PM, Raymond Hettinger <python at rcn.com> wrote:
> During the discussion of PEP 3141, what use cases were envisioned for the Integral type?  Were there some examples of concrete implementations other than <type 'int'> or <type 'long'??

I believe numpy defines a whole slew of these of different sizes.

> I'm asking because I'm unclear about what a class is stating about itself when it registers as Integral.  I'm not sure that most finite fields would fit into the structure since they all Integrals support logical operations, are convertible into ints via __index__, andy the must support all of the operations defined for Reals.

An earlier version of the PEP talked about fields and all sort of
stuff, but we found nobody cared.

> I looked at the reference for Scheme's numeric tower to find guidance, but it seemed that the notion expressed solely through the "integer?" predicate which behaves differently than our inheritance tower.  For example, the spec says (rational? 6/10) ==> #t and (integer? 8/4) ==> #t but in our current implementation Rational(8,4) is not Integral like it is for Scheme.
>
> Also, Scheme seems apply implement the notions of Exact/Inexact much differently
> than we do: "A number is exact if it was written as an exact constant or was derived from exact numbers using only exact operations."  So, while all integers are exact, some floats and decimal constants could also be exact.  This matches how exactness is used in the decimal module -- all constants start as exact and operations can return either exact or inexact (and the module signals when this occurs).  Our module says Integrals are Exact and makes no further assertions.

I expect that there are separate useful notions of exactness and
inexactness, one solely based on the type, and one based on value.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Feb 12 04:31:14 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 19:31:14 -0800
Subject: [Python-3000] ABC for Inequalities
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
Message-ID: <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>

[GvR]
> Hold on. We went down that road quite a bit and then discovered it
> wasn't going to work. 

I figured that it was missing for a reason.

> The problem is that if you have two instance x,
> y such that isinstance(x, TotalOrdering) and isinstance(y,
> TotalOrdering) you still don't know if x < y is defined. (For example,
> if x is a string and y is a number.) 

So, ordering only makes sense when "other" is known to be
of the same type as "self".

> There's a useful definition in there somewhere but
> it's elusive. We'll eventually figure it out.

I hope so. I'm less interested in the mixin and more interested in
a way for a class to have a way to tell you whether it is sortable.
Right now, the presence or absence of __le__ doesn't tell
you much.


Raymond

From python at rcn.com  Tue Feb 12 04:58:55 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 19:58:55 -0800
Subject: [Python-3000] Nix dict.copy()
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
Message-ID: <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>

Where did we end-up on dict.copy()?

Is the copy(obj) function sufficiently universal to replace the obj.copy() method used in sets and in dict imitators?  Or does the 
need for import preclude it from consideration?

To me it seems that relationship between __copy__ and the copy() function is just as universal as the relationship between __len__ 
and len(). Am I missing something that makes d.copy() preferable to copy(d)?

I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for 
dict.__copy__ and dict.copy share the same implementation).


Raymond

From python at rcn.com  Tue Feb 12 05:10:29 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 20:10:29 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
	<d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>
Message-ID: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>

Was the concensus on this to move UserList into the collections module and to remove UserString?
Any objections?


Raymond



---------------------------------------------------
>> [Brett]
>> > +1 on collections.
>>
>> [Steven Bethard]
>> > +1 for putting them in the collections module.
>>
>> How about UserString?
>> Is there a reason for it to continue to exist?
> 
> Like Brett, I've never found a use for it, so I'm happy to see it go.
> 
> Steve
> -- 
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>        --- Bucky Katt, Get Fuzzy

From guido at python.org  Tue Feb 12 05:41:55 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 20:41:55 -0800
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>

On Feb 11, 2008 7:31 PM, Raymond Hettinger <python at rcn.com> wrote:
> [GvR]
> > Hold on. We went down that road quite a bit and then discovered it
> > wasn't going to work.
>
> I figured that it was missing for a reason.
>
> > The problem is that if you have two instance x,
> > y such that isinstance(x, TotalOrdering) and isinstance(y,
> > TotalOrdering) you still don't know if x < y is defined. (For example,
> > if x is a string and y is a number.)
>
> So, ordering only makes sense when "other" is known to be
> of the same type as "self".

No, that's too far in the other direction. E.g. two instances of Real
but of different implementations should still be comparable. (Though I
expect there's some higher math involved to prove this for all reals.
:-)

> > There's a useful definition in there somewhere but
> > it's elusive. We'll eventually figure it out.
>
> I hope so. I'm less interested in the mixin and more interested in
> a way for a class to have a way to tell you whether it is sortable.
> Right now, the presence or absence of __le__ doesn't tell
> you much.

Yeah, this is one area where a leap of faith (a.k.a. duck typing :-)
may still be the best approach.

I think you can probably get something that is reasonable given the
kind of assumptions that prevail in an ABC world by having a
"Sortable" metaclass (itself a subclass of ABCMeta) with the implied
semantics that objects x and y can be sorted if the nearest common
ancestor of x.__class__ and y.__class__ is an instance of Sortable.
E.g.

class S(metaclass=Sortable): ...

class S0(S):
  def __lt__(self, other): ...
  def __le__(self, other): ...
  ...etc...

class S1(S0): ...

class S2(S0): ...

x = S1()
y = S2()

Now x<y should work: the nearest common ancestor of S1 and S2 is S0,
which is a subclass of S, which is an instance of Sortable. (If S is
an instance of Sortable, so is S0 -- that's how metaclasses are
inherited.)

But I'm not sure this is worth it; we don't have an easy way to add
this to an argument annotation, since the argument annotations are
(usually) assumed to be type expressions so that f(x:T) means
isinstance(x,T) -- but here we want to express isinstance(x.__class__,
Sortable). The best I can think of is to say that if the annotation is
a lambda (not just any callable, since classes are callable) it must
evaluate to true when called on the actual argument: then we'd write
f(x: (lambda x: isinstance(x.__class__, Sortable))). Very ugly, but
could of course be abbreviated with a suitable helper function to f(x:
metaclass_is(Sortable)).

PS. "Sortable" is perhaps not the best name, as one would perhaps
think that a list of numbers is sortable because it has a .sort()
method... Maybe Orderable or Ordered or TotallyOrdered?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 05:47:10 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 20:47:10 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>

On Feb 11, 2008 7:58 PM, Raymond Hettinger <python at rcn.com> wrote:
> Where did we end-up on dict.copy()?
>
> Is the copy(obj) function sufficiently universal to replace the obj.copy() method used in sets and in dict imitators?  Or does the
> need for import preclude it from consideration?
>
> To me it seems that relationship between __copy__ and the copy() function is just as universal as the relationship between __len__
> and len(). Am I missing something that makes d.copy() preferable to copy(d)?
>
> I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for
> dict.__copy__ and dict.copy share the same implementation).

I find the use of a copy operation that purports to copy any object
suspect (regardless whether it's a module or a builtin), and that I'd
rather have a copy() method on the MutableMapping API.

I think my objection against the fully generic copy operation is that
it's type is useless -- all we know that it is supposed to return the
same type as its argument, but our type system can't express that. The
type of the more constrained (but still generic in the sense of PEP
3124) copy operation on mutable sequences is better defined -- it
returns another mutable sequence, with the understanding that it
typically returns something of the same type as its input, though not
always -- e.g. if I subclass dict but don't override .copy(), the
inherited .copy() method returns a dict, not my subclass.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 05:48:27 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Feb 2008 20:48:27 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
	<d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>
	<008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802112048n25895961hae0ecd1638a8dfe2@mail.gmail.com>

I don't know about consensus, but personally I'm +1 on this.

On Feb 11, 2008 8:10 PM, Raymond Hettinger <python at rcn.com> wrote:
> Was the concensus on this to move UserList into the collections module and to remove UserString?
> Any objections?
>
>
> Raymond
>
>
>
> ---------------------------------------------------
>
> >> [Brett]
> >> > +1 on collections.
> >>
> >> [Steven Bethard]
> >> > +1 for putting them in the collections module.
> >>
> >> How about UserString?
> >> Is there a reason for it to continue to exist?
> >
> > Like Brett, I've never found a use for it, so I'm happy to see it go.
> >
> > Steve
> > --
> > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> > tiny blip on the distant coast of sanity.
> >        --- Bucky Katt, Get Fuzzy
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at cheimes.de  Tue Feb 12 06:41:36 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 12 Feb 2008 06:41:36 +0100
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>	<d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>
	<008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
Message-ID: <forbig$7b4$1@ger.gmane.org>

Raymond Hettinger wrote:
> Was the concensus on this to move UserList into the collections module and to remove UserString?
> Any objections?

You have my +1 as well.

Christian


From brett at python.org  Tue Feb 12 07:10:34 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 11 Feb 2008 22:10:34 -0800
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
	<d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>
	<008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
Message-ID: <bbaeab100802112210v6123d0adh7ec91d898cdf03c2@mail.gmail.com>

On Feb 11, 2008 8:10 PM, Raymond Hettinger <python at rcn.com> wrote:
> Was the concensus on this to move UserList into the collections module and to remove UserString?
> Any objections?

I ain't going to object.

-Brett

From python at rcn.com  Tue Feb 12 07:20:31 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 22:20:31 -0800
Subject: [Python-3000] Nix dict.copy()
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
Message-ID: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>

>> I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for
>> dict.__copy__ and dict.copy share the same implementation).
> 
> I find the use of a copy operation that purports to copy any object
> suspect (regardless whether it's a module or a builtin), and that I'd
> rather have a copy() method on the MutableMapping API.

Should I revert the -3 warning on copy() methods that I checked-in a few days ago?



> I think my objection against the fully generic copy operation is that
> it's type is useless -- all we know that it is supposed to return the
> same type as its argument, but our type system can't express that.

I'm not sure I follow you here.  The definition of Iterator.__iter__
is to return self, so it doesn't seem like much of a leap to express 
a signature with a return type of type(self).

Or, were you saying that the actual problem is that we can't know
in advnace the signature of  type(self).__init__()?

> The
> type of the more constrained (but still generic in the sense of PEP
> 3124) copy operation on mutable sequences is better defined -- it
> returns another mutable sequence, with the understanding that it
> typically returns something of the same type as its input, though not
> always -- e.g. if I subclass dict but don't override .copy(), the
> inherited .copy() method returns a dict, not my subclass.

The part I'm not following in this paragraph is how a d.copy()
method is any different from the d.__copy__() method that
is called by the copy() function.  Both share the same underlying
implementation and both have the same effects when overridden.

If all we want a guaranteed dict, why not write dict(obj)?  
This already works with all the dict lookalikes because they
define obj.keys() and obj.__getitem__().


Raymond

From python at rcn.com  Tue Feb 12 07:38:58 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 11 Feb 2008 22:38:58 -0800
Subject: [Python-3000] ABC for Inequalities
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
	<ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
Message-ID: <00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1>

[GvR]
> I think you can probably get something that is reasonable given the
> kind of assumptions that prevail in an ABC world by having a
> "Sortable" metaclass (itself a subclass of ABCMeta) with the implied
> semantics that objects x and y can be sorted if the nearest common
> ancestor of x.__class__ and y.__class__ is an instance of Sortable.
> E.g.
> 
> class S(metaclass=Sortable): ...
> 
> class S0(S):
>  def __lt__(self, other): ...
>  def __le__(self, other): ...
>  ...etc...
> 
> class S1(S0): ...
> 
> class S2(S0): ...
> 
> x = S1()
> y = S2()
> 
> Now x<y should work: the nearest common ancestor of S1 and S2 is S0,
> which is a subclass of S, which is an instance of Sortable. (If S is
> an instance of Sortable, so is S0 -- that's how metaclasses are
> inherited.)
> 
> But I'm not sure this is worth it; we don't have an easy way to add
> this to an argument annotation, since the argument annotations are
> (usually) assumed to be type expressions so that f(x:T) means
> isinstance(x,T) -- but here we want to express isinstance(x.__class__,
> Sortable). The best I can think of is to say that if the annotation is
> a lambda (not just any callable, since classes are callable) it must
> evaluate to true when called on the actual argument: then we'd write
> f(x: (lambda x: isinstance(x.__class__, Sortable))). Very ugly, but
> could of course be abbreviated with a suitable helper function to f(x:
> metaclass_is(Sortable)).

Wow!

That's a lot of firepower for such a little problem ;-)



Raymond

From ntoronto at cs.byu.edu  Tue Feb 12 07:40:35 2008
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 11 Feb 2008 23:40:35 -0700
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
	<ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
Message-ID: <47B13F63.9010302@cs.byu.edu>

Guido van Rossum wrote:
> On Feb 11, 2008 7:31 PM, Raymond Hettinger <python at rcn.com> wrote:
>> [GvR]
>>> The problem is that if you have two instance x,
>>> y such that isinstance(x, TotalOrdering) and isinstance(y,
>>> TotalOrdering) you still don't know if x < y is defined. (For example,
>>> if x is a string and y is a number.)
>> So, ordering only makes sense when "other" is known to be
>> of the same type as "self".
> 
> No, that's too far in the other direction. E.g. two instances of Real
> but of different implementations should still be comparable. (Though I
> expect there's some higher math involved to prove this for all reals.
> :-)

The problem is that, like all the binary operators, "orderable" is only 
defined for pairs of types. A single declaration of orderable-ness 
necessarily lacks information needed for type safety.

>>> There's a useful definition in there somewhere but
>>> it's elusive. We'll eventually figure it out.
>> I hope so. I'm less interested in the mixin and more interested in
>> a way for a class to have a way to tell you whether it is sortable.
>> Right now, the presence or absence of __le__ doesn't tell
>> you much.
> 
> Yeah, this is one area where a leap of faith (a.k.a. duck typing :-)
> may still be the best approach.
> 
> I think you can probably get something that is reasonable given the
> kind of assumptions that prevail in an ABC world by having a
> "Sortable" metaclass (itself a subclass of ABCMeta) with the implied
> semantics that objects x and y can be sorted if the nearest common
> ancestor of x.__class__ and y.__class__ is an instance of Sortable.

Besides probably not being "worth it" (as you say), forcing the classes 
into an inheritance hierarchy doesn't seem Pythonic. (I may not have the 
Python-fu to make such declarations, but I'll call 'em as I see 'em.) 
Also, x objects and y objects may be orderable homogeneously, but not 
heterogeneously. I could impose a total order on sets of circles and 
another on sets of squares without imposing an order on sets of both 
circles and squares, and not be able to spell this under the inheritance 
approach.

To do it properly beyond duck typing, there should be a way to ask 
whether (and declare that) a *pair* of types implements an interface 
such as Orderable. This is analogous to how the binary operators 
currently work, though those are decentralized: the op/rop methods 
jointly decide what's legal.

A little voice in my head is shouting "icky!" at the prospects.

IIRC, one major motivation for ABCs was to avoid calling potentially 
destructive methods by allowing interface queries first. Does anyone 
expect comparisons to be destructive?

Where would an "Orderable" query happen? Not before sorting a list - 
you'd have to scan the list to see what types are in it. (And declaring 
orderability for just pairs of types may not be enough then. Is 
orderability transitive?) Before each comparison? If that's the case, 
the only reason I can see for checking for Orderable is to keep people 
from doing dumb things.

Neil

From guido at python.org  Tue Feb 12 18:56:58 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 09:56:58 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>

On Feb 11, 2008 10:20 PM, Raymond Hettinger <python at rcn.com> wrote:
> >> I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for
> >> dict.__copy__ and dict.copy share the same implementation).
> >
> > I find the use of a copy operation that purports to copy any object
> > suspect (regardless whether it's a module or a builtin), and that I'd
> > rather have a copy() method on the MutableMapping API.
>
> Should I revert the -3 warning on copy() methods that I checked-in a few days ago?

Yes, I think we jumped the gun here a bit.

> > I think my objection against the fully generic copy operation is that
> > it's type is useless -- all we know that it is supposed to return the
> > same type as its argument, but our type system can't express that.
>
> I'm not sure I follow you here.  The definition of Iterator.__iter__
> is to return self, so it doesn't seem like much of a leap to express
> a signature with a return type of type(self).

Actually, __iter__ is defined on any Iterable, and it returns an
Iterator; so after x = y.__iter__() (or x = iter(y) of course) you
know that x is an Iterator, even if you know nothing about y's type
(though if y is not an Iterable, the expresion may raise an
exception).

OTOH if you see x = copy(y), you don't know anything about x unless
you know something about y; without knowing anything about y, all you
know is that x is an object.

Here's another way to look at it, maybe it'll help. My hunch is that
whenever someone writes x = copy(y), they actually know more about the
type of y than just that it is an object! Because if they knew
*nothing* about it, they wouldn't know whether copying was (a)
pointless (as it is for immutable objects), (b) impossible (e.g. for
sockets), or (c) useful (e.g. for lists).

When thinking about these kind of things I often place myself in the
position of a static type checker that happens to know something about
the types of some variables and is trying to gather more information.
Seeing x=iter(y) can provide new information about y (it must be an
Iterable). Seeing x=copy(y) provides no new information about y, since
all (well, most) objects can be copied. OTOH seeing x=y.copy() *does*
provide new information about y; it must be a type that has a .copy()
method. Now seeing x=len(y) *does* provide new information about y
(since we know which types support .__len__()).

I realize I'm doing a lousy job of explaining me. Please trust me that
there's something here. I will eventually get to it (maybe not today).

> Or, were you saying that the actual problem is that we can't know
> in advance the signature of  type(self).__init__()?

No, the __copy__ GF takes care of that concern.

> > The
> > type of the more constrained (but still generic in the sense of PEP
> > 3124) copy operation on mutable sequences is better defined -- it
> > returns another mutable sequence, with the understanding that it
> > typically returns something of the same type as its input, though not
> > always -- e.g. if I subclass dict but don't override .copy(), the
> > inherited .copy() method returns a dict, not my subclass.
>
> The part I'm not following in this paragraph is how a d.copy()
> method is any different from the d.__copy__() method that
> is called by the copy() function.  Both share the same underlying
> implementation and both have the same effects when overridden.

The difference is that *every* object has a .__copy__() method (it is
inherited from object, at least conceptually), but not every object
has a .copy() method.

> If all we want a guaranteed dict, why not write dict(obj)?
> This already works with all the dict lookalikes because they
> define obj.keys() and obj.__getitem__().

Well, the whole point of this discussion that dict(obj) is what you
should write if you want a guaranteed dict, and obj.copy() is what you
should write if you want a guaranteed mutable mapping of the same type
as obj. Just like you write list(obj) if you want a list and obj[:] if
you want a sequence of the same type as obj.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 18:57:45 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 09:57:45 -0800
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
	<ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
	<00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1>
Message-ID: <ca471dc20802120957r3e39ab71u21e3b14c6f778ef5@mail.gmail.com>

On Feb 11, 2008 10:38 PM, Raymond Hettinger <python at rcn.com> wrote:
> That's a lot of firepower for such a little problem ;-)

Indeed, which is why it was not adopted. ;-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 19:02:25 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 10:02:25 -0800
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <47B13F63.9010302@cs.byu.edu>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
	<ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
	<47B13F63.9010302@cs.byu.edu>
Message-ID: <ca471dc20802121002v5ee24227g667546de4b748c4c@mail.gmail.com>

On Feb 11, 2008 10:40 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> The problem is that, like all the binary operators, "orderable" is only
> defined for pairs of types. A single declaration of orderable-ness
> necessarily lacks information needed for type safety.

Actually, lots of languages have ways to express that the values of a
given type are orderable. E.g. in C++ you can create a template that
takes a single type argument which must define a total ordering using
'<'.

> Besides probably not being "worth it" (as you say), forcing the classes
> into an inheritance hierarchy doesn't seem Pythonic. (I may not have the
> Python-fu to make such declarations, but I'll call 'em as I see 'em.)

You're forgetting about virtual inheritance.

> Also, x objects and y objects may be orderable homogeneously, but not
> heterogeneously. I could impose a total order on sets of circles and
> another on sets of squares without imposing an order on sets of both
> circles and squares, and not be able to spell this under the inheritance
> approach.

Yes, that was what I explained at length.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Feb 12 19:10:53 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 10:10:53 -0800
Subject: [Python-3000] Preventing merges into Py3k
Message-ID: <ca471dc20802121010i3bddd22bq2ed73537d67c443@mail.gmail.com>

On Feb 12, 2008 9:44 AM, thomas.heller <python-3000-checkins at python.org> wrote:
> Author: thomas.heller
> Date: Tue Feb 12 18:44:23 2008
> New Revision: 60746
>
> Modified:
>    python/branches/py3k/Modules/_ctypes/_ctypes.c
> Log:
> Revert the last svnmerge (r60681) from trunk to _ctypes.c, it should
> not have been merged as was noticed in the commit message.

There is a way to prevent merging a particular revision by instructing
svnmerge properly. I believe the syntax is svnmerge block (svnmerge
help block will explain you more).

Christian has been using this -- Christian, care to post a detailed example?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Tue Feb 12 19:30:30 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 12 Feb 2008 11:30:30 -0700
Subject: [Python-3000] Location of UserDict, UserList, and UserString
In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1>
	<bbaeab100802091732i1f81fbb5g6bbbb335ad1b2a9d@mail.gmail.com>
	<000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1>
	<d11dcfba0802091851n4bc575d7tdf692f3dd4bce86c@mail.gmail.com>
	<008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1>
Message-ID: <d11dcfba0802121030u24e85257q723adcc018797c6d@mail.gmail.com>

On Feb 11, 2008 9:10 PM, Raymond Hettinger <python at rcn.com> wrote:
> Was the concensus on this to move UserList into the collections module and to remove UserString?
> Any objections?

No objections here either.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From lists at cheimes.de  Tue Feb 12 20:41:43 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 12 Feb 2008 20:41:43 +0100
Subject: [Python-3000] Preventing merges into Py3k
In-Reply-To: <ca471dc20802121010i3bddd22bq2ed73537d67c443@mail.gmail.com>
References: <ca471dc20802121010i3bddd22bq2ed73537d67c443@mail.gmail.com>
Message-ID: <fosspr$k4k$1@ger.gmane.org>

Guido van Rossum wrote:
> There is a way to prevent merging a particular revision by instructing
> svnmerge properly. I believe the syntax is svnmerge block (svnmerge
> help block will explain you more).
> 
> Christian has been using this -- Christian, care to post a detailed example?

Yes, svnmerge.py block is correct. In order to prevent a merge you have
to use svnmerge.py in a local copy of the py3k branch. Don't forget to
submit the change! ;)

cd .../py3k
svnmerge.py block -r12345
svn ci -m "block r12345"

Christian


From greg.ewing at canterbury.ac.nz  Tue Feb 12 23:05:10 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Feb 2008 11:05:10 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
Message-ID: <47B21816.7050505@canterbury.ac.nz>

Raymond Hettinger wrote:
> The part I'm not following in this paragraph is how a d.copy()
> method is any different from the d.__copy__() method that
> is called by the copy() function.  Both share the same underlying
> implementation and both have the same effects when overridden.

Having it a function/special method pair appears to make a
strong statement that it's a very important operation that
a large number of types ought to implement. As Guido said,
it's not really a generic enough operation to warrant that.

--
Greg

From greg.ewing at canterbury.ac.nz  Tue Feb 12 23:20:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Feb 2008 11:20:54 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
Message-ID: <47B21BC6.9060309@canterbury.ac.nz>

Guido van Rossum wrote:

> Here's another way to look at it, maybe it'll help. My hunch is that
> whenever someone writes x = copy(y), they actually know more about the
> type of y than just that it is an object!

I don't see how this is much different than the situation
with __iter__. If you know that y is a sequence of items
of type Y, then you know that iter(y) probably produces
an object with a next() method that returns objects of
type Y.

If you're talking about programmers doing type analysis
in their heads, I think the type system they use is quite
a sophisticated one, certainly a parametric one that has
no trouble coping with things like this.

> I realize I'm doing a lousy job of explaining me.

What's confusing is that you're talking about static
typing issues in a language that has no static type
system. I'm having trouble seeing the relevance.

--
Greg


From guido at python.org  Tue Feb 12 23:32:30 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 14:32:30 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <47B21BC6.9060309@canterbury.ac.nz>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
	<47B21BC6.9060309@canterbury.ac.nz>
Message-ID: <ca471dc20802121432r72165f2ek1047a4e32ab01420@mail.gmail.com>

On Feb 12, 2008 2:20 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
> > Here's another way to look at it, maybe it'll help. My hunch is that
> > whenever someone writes x = copy(y), they actually know more about the
> > type of y than just that it is an object!
>
> I don't see how this is much different than the situation
> with __iter__. If you know that y is a sequence of items
> of type Y, then you know that iter(y) probably produces
> an object with a next() method that returns objects of
> type Y.
>
> If you're talking about programmers doing type analysis
> in their heads, I think the type system they use is quite
> a sophisticated one, certainly a parametric one that has
> no trouble coping with things like this.

It depends. I used Google Code Search to find examples of copy.copy().
Many were hard to understand because there was no clue about the
argument passed it, and the copy() call doesn't help me infer its type
either. Contrast this with example code using iter() -- it tells me
that the argument is an iterable. That rules out lots of
possibilities, like numbers.

> > I realize I'm doing a lousy job of explaining me.
>
> What's confusing is that you're talking about static
> typing issues in a language that has no static type
> system. I'm having trouble seeing the relevance.

Trust me, there is a relevance to it in my head! It has to do with
Python's ancestry from ABC, which did static type inference.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From daniel at stutzbachenterprises.com  Tue Feb 12 23:40:10 2008
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 12 Feb 2008 16:40:10 -0600
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
Message-ID: <eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>

On Feb 12, 2008 11:56 AM, Guido van Rossum <guido at python.org> wrote:

> Here's another way to look at it, maybe it'll help. My hunch is that
> whenever someone writes x = copy(y), they actually know more about the
> type of y than just that it is an object! Because if they knew
> *nothing* about it, they wouldn't know whether copying was (a)
> pointless (as it is for immutable objects), (b) impossible (e.g. for
> sockets), or (c) useful (e.g. for lists).


Based on the same reasoning, when we see x = copy(y) we can tentatively
assume that y is a type that can usefully be copied.  Unless we don't trust
the author of the code to be sensible.


> Seeing x=copy(y) provides no new information about y, since
> all (well, most) objects can be copied. OTOH seeing x=y.copy() *does*
> provide new information about y; it must be a type that has a .copy()
> method.


What useful information do we get by knowing that a type has a .copy()
method?

If I took the last sentence out of context, I would assume that having a
.copy() method informs you that the type is copyable.  Except you've just
said that all (well, most) objects in Python can be copied--so I'm not sure
new information a .copy() method conveys.

Normally I might assume that it does a special kind of copying that doesn't
fit the definition of copy.copy() nor copy.deepcopy(), except that for dict
and set types, .copy() and .__copy__() are identical.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080212/525d4a0e/attachment.htm 

From greg.ewing at canterbury.ac.nz  Tue Feb 12 23:39:13 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Feb 2008 11:39:13 +1300
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802121432r72165f2ek1047a4e32ab01420@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
	<47B21BC6.9060309@canterbury.ac.nz>
	<ca471dc20802121432r72165f2ek1047a4e32ab01420@mail.gmail.com>
Message-ID: <47B22011.7070906@canterbury.ac.nz>

Guido van Rossum wrote:
> I used Google Code Search to find examples of copy.copy().
> Many were hard to understand because there was no clue about the
> argument passed it, and the copy() call doesn't help me infer its type
> either. Contrast this with example code using iter() -- it tells me
> that the argument is an iterable. That rules out lots of
> possibilities, like numbers.

Hmmm. Personally I'm not sure that such ad-hoc piecemeal
type inference is all that useful. Just knowing that
something is an iterable still doesn't tell me anything
about the type of the items it contains, which is
likely to be of much more interest to me.

Similarly, just knowing that something has a copy()
method doesn't narrow things down all that much. It
might be a dict, or a set, or something else the user
has come up with, which could be just about anything.

--
Greg


From guido at python.org  Tue Feb 12 23:52:04 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 14:52:04 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
	<eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>
Message-ID: <ca471dc20802121452n76a55ab1g1e07bb22c1383753@mail.gmail.com>

On Feb 12, 2008 2:40 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> > Based on the same reasoning, when
we see x = copy(y) we can tentatively
> assume that y is a type that can usefully be copied.  Unless we don't trust
> the author of the code to be sensible.

My point is that (almost) any type is copyable so we can't tell what
the author is assuming. Maybe they believe they had either a list or a
dict -- or maybe they know they have an arbitrary class instance.

> > Seeing x=copy(y) provides no new information about y, since
> > all (well, most) objects can be copied. OTOH seeing x=y.copy() *does*
> > provide new information about y; it must be a type that has a .copy()
> > method.
>
> What useful information do we get by knowing that a type has a .copy()
> method?

It rules out all classes that don't have one. That's nearly all types.

> If I took the last sentence out of context, I would assume that having a
> .copy() method informs you that the type is copyable.  Except you've just
> said that all (well, most) objects in Python can be copied--so I'm not sure
> new information a .copy() method conveys.

Only dict and a few mapping-like types (and a vanishingly small number
of user-defined classes) define copy().

> Normally I might assume that it does a special kind of copying that doesn't
> fit the definition of copy.copy() nor copy.deepcopy(), except that for dict
> and set types, .copy() and .__copy__() are identical.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From daniel at stutzbachenterprises.com  Wed Feb 13 00:11:09 2008
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 12 Feb 2008 17:11:09 -0600
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <ca471dc20802121452n76a55ab1g1e07bb22c1383753@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
	<eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>
	<ca471dc20802121452n76a55ab1g1e07bb22c1383753@mail.gmail.com>
Message-ID: <eae285400802121511r6872ca26hd4c45843688f0997@mail.gmail.com>

On Feb 12, 2008 4:52 PM, Guido van Rossum <guido at python.org> wrote:

> > What useful information do we get by knowing that a type has a .copy()
> > method?
>
> It rules out all classes that don't have one. That's nearly all types.
>

Okay, but turn it around for a minute.  Which types should have a .copy()
method and why?

If I'm making a new copyable type, is there a rule of thumb that helps me
decide on .copy(), .__copy__(), or both?

Rules of "dict and set have a .copy() method" or "mapping types have a
.copy() method" seem... arbitrary.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20080212/b0f2e170/attachment.htm 

From guido at python.org  Wed Feb 13 00:15:31 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 12 Feb 2008 15:15:31 -0800
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <eae285400802121511r6872ca26hd4c45843688f0997@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>
	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>
	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>
	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>
	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>
	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>
	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>
	<eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>
	<ca471dc20802121452n76a55ab1g1e07bb22c1383753@mail.gmail.com>
	<eae285400802121511r6872ca26hd4c45843688f0997@mail.gmail.com>
Message-ID: <ca471dc20802121515m3df83a9epcd6e205286d0428b@mail.gmail.com>

On Feb 12, 2008 3:11 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Feb 12, 2008 4:52 PM, Guido van Rossum <guido at python.org> wrote:
>
> >
> > > What useful information do we get by knowing that a type has a .copy()
> > > method?
> >
> > It rules out all classes that don't have one. That's nearly all types.
> >
>
> Okay, but turn it around for a minute.  Which types should have a .copy()
> method and why?
>
> If I'm making a new copyable type, is there a rule of thumb that helps me
> decide on .copy(), .__copy__(), or both?
>
> Rules of "dict and set have a .copy() method" or "mapping types have a
> .copy() method" seem... arbitrary.

I can't answer those questions in the abstract; I'd have to see the
type you are trying to create and the use case for creating copies.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Feb 14 00:34:47 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 14 Feb 2008 12:34:47 +1300
Subject: [Python-3000] ABC for Inequalities
In-Reply-To: <47B13F63.9010302@cs.byu.edu>
References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net>
	<ca471dc20802111908k6aed4fdaj2369ee1403c53dcf@mail.gmail.com>
	<003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1>
	<ca471dc20802112041n76caededp6e3e275c2891378d@mail.gmail.com>
	<47B13F63.9010302@cs.byu.edu>
Message-ID: <47B37E97.5010409@canterbury.ac.nz>

Neil Toronto wrote:

> The problem is that, like all the binary operators, "orderable" is only 
> defined for pairs of types.

I'm loathe to mention it, but it sounds like what's
needed for this is generic functions on types...

-- 
Greg

From van.lindberg at gmail.com  Thu Feb 14 16:13:22 2008
From: van.lindberg at gmail.com (VanL)
Date: Thu, 14 Feb 2008 09:13:22 -0600
Subject: [Python-3000] Nix dict.copy()
In-Reply-To: <eae285400802121511r6872ca26hd4c45843688f0997@mail.gmail.com>
References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net>	<20080210045826.651DD3A40B0@sparrow.telecommunity.com>	<ca471dc20802100822ldc32325u89b9262604602800@mail.gmail.com>	<d11dcfba0802100933g715a6171g5833ccf667d9800c@mail.gmail.com>	<006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1>	<ca471dc20802112047q4e4e8fc5w74f982e1de0aebc1@mail.gmail.com>	<00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1>	<ca471dc20802120956h73a339cbw5c169677904084a1@mail.gmail.com>	<eae285400802121440k2c1c544dpeea4bcc9b7dbbbd4@mail.gmail.com>	<ca471dc20802121452n76a55ab1g1e07bb22c1383753@mail.gmail.com>
	<eae285400802121511r6872ca26hd4c45843688f0997@mail.gmail.com>
Message-ID: <fp1lqj$qgr$1@ger.gmane.org>

Daniel Stutzbach wrote:
> Okay, but turn it around for a minute.  Which types should have a 
> .copy() method and why?

I would argue that it's not about mappings, it's about mutability. I 
always thought the .copy method on dicts was to allow functions to work 
on a passed-in dict without causing spooky action at a distance due to 
mutability issues. (See the silly example at the end of the post.) Sets, 
being by default mutable, got the same treatment.

Other types (other than lists) aren't susceptible to this sort of 
problem. The default idiom for lists,  copy = list[:], is a workaround 
for the lack of a .copy() method.

I would argue that if you want to make some sort of ABC, make it a 
'Mutable' ABC and put the .copy method in *there*. \

- Van

*For example, you can get into this problem when using dicts for string 
substitution:

def fragment1(vars, boilerplate=''):
    if boilerplate: vars['extra'] = boilerplate
    return "%(sentence)s with %(something)s %(extra)s." % vars

def fragment2(vars, boilerplate=''):
    if boilerplate: vars['extra'] = boilerplate
    return "%(another)s %(sentence)s with %(extra)s." % vars

def statement():
    # vars is a dict built from a database query
    vars = get_dict_from_db()
    return '%s %s' % (fragment1(vars, 'extra'), fragment2(vars))

Because of mutability, the 'extra' in fragment1 will also show up in 
fragment2. The solution to this is to make a .copy(). The susceptibility 
of newbies to this 'spooky action at a distance' problem is one of the 
reasons why mutability and immutability have to be addressed almost at 
the very start of many learning Python books.






From skip at pobox.com  Thu Feb 14 22:09:05 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 14 Feb 2008 15:09:05 -0600
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
Message-ID: <18356.44529.51137.327615@montanaro-dyndns-org.local>

A thread on the Python Mac sig got me to wondering if there is any magic in
Python 3's print function for printing Unicode.  Nope, no magic:

    >>> print("\xef")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/skip/local/lib/python3.0/io.py", line 1246, in write
        b = encoder.encode(s)
      File "/Users/skip/local/lib/python3.0/encodings/ascii.py", line 22, in encode
        return codecs.ascii_encode(input, self.errors)[0]
    UnicodeEncodeError: 'ascii' codec can't encode character '\xef' in position 0: ordinal not in range(128)

Which kind of confuses me because on my system my default encoding is utf-8:

    >>> sys.getdefaultencoding()
    'utf-8'

Well, then how about an encoding arg?  Nope again:

    >>> print("\xef", encoding="utf-8")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: 'encoding' is an invalid keyword argument for this function

help(print) doesn't offer any suggestions either.  Any chance that maybe an
encoding keyword arg could make it into an upcoming 3.0aN release?
(Or... Am I missing a simple solution to the problem?)  It seems that
printing Unicode ought to be easier in Python 3 than it is.

Skip

From guido at python.org  Thu Feb 14 23:02:31 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 14 Feb 2008 14:02:31 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <18356.44529.51137.327615@montanaro-dyndns-org.local>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
Message-ID: <ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>

The encoding is a file property, not a print parameter.

The code that guesses the default stdout encoding isn't very good,
especially not on OSX. Suggestions are welcome.

You can hack around this for now by doing (before printing anything)

  sys.stdout._encoding = 'utf-8'

but that ought to be a temporary hack until we've figured out the
right way to set it.

--Guido

On Thu, Feb 14, 2008 at 1:09 PM,  <skip at pobox.com> wrote:
> A thread on the Python Mac sig got me to wondering if there is any magic in
>  Python 3's print function for printing Unicode.  Nope, no magic:
>
>     >>> print("\xef")
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "/Users/skip/local/lib/python3.0/io.py", line 1246, in write
>         b = encoder.encode(s)
>       File "/Users/skip/local/lib/python3.0/encodings/ascii.py", line 22, in encode
>         return codecs.ascii_encode(input, self.errors)[0]
>     UnicodeEncodeError: 'ascii' codec can't encode character '\xef' in position 0: ordinal not in range(128)
>
>  Which kind of confuses me because on my system my default encoding is utf-8:
>
>     >>> sys.getdefaultencoding()
>     'utf-8'
>
>  Well, then how about an encoding arg?  Nope again:
>
>     >>> print("\xef", encoding="utf-8")
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: 'encoding' is an invalid keyword argument for this function
>
>  help(print) doesn't offer any suggestions either.  Any chance that maybe an
>  encoding keyword arg could make it into an upcoming 3.0aN release?
>  (Or... Am I missing a simple solution to the problem?)  It seems that
>  printing Unicode ought to be easier in Python 3 than it is.
>
>  Skip
>  _______________________________________________
>  Python-3000 mailing list
>  Python-3000 at python.org
>  http://mail.python.org/mailman/listinfo/python-3000
>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Feb 14 23:21:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Feb 2008 23:21:40 +0100
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
Message-ID: <47B4BEF4.5040206@v.loewis.de>

> The code that guesses the default stdout encoding isn't very good,
> especially not on OSX. Suggestions are welcome.

Unfortunately, Apple isn't very cooperative here. There is no way
of determining the encoding of a Terminal.app window, AFAIK
(it's normally UTF-8, unless the user has changed it).

Regards,
Martin

From guido at python.org  Thu Feb 14 23:38:44 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 14 Feb 2008 14:38:44 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <47B4BEF4.5040206@v.loewis.de>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<47B4BEF4.5040206@v.loewis.de>
Message-ID: <ca471dc20802141438s77555923va80239bab5145d3d@mail.gmail.com>

In Leopard (OSX 10.5), it looks like LANG is set to convey this: it
has the value en_US.UTF-8 for me, and I haven't done anything to
change it. The Preferences dialog is completely changed from 10.4 and
there's a checkbox (on by default AFAIK) to set the LANG variable.
(However I've noticed that it doesn't set an encoding when I specify
another encoding other than UTF-8 -- perhaps because there is
interaction with the Language choice.)

In 10.4 LANG isn't set at all. The effect is that in 10.5,
sys.stdout.encoding defaults to UTF-8 but in 10.4 to US-ASCII.

Maybe a decent rule of thumb would be:

- if LANG is set, use the encoding it specifies; if there's no
encoding in it, assume it's ASCII.

- if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8
is much more likely to be correct and useful for 10.4 and before.

--Guido

On Thu, Feb 14, 2008 at 2:21 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > The code that guesses the default stdout encoding isn't very good,
>  > especially not on OSX. Suggestions are welcome.
>
>  Unfortunately, Apple isn't very cooperative here. There is no way
>  of determining the encoding of a Terminal.app window, AFAIK
>  (it's normally UTF-8, unless the user has changed it).
>
>  Regards,
>  Martin
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Feb 14 23:48:12 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 14 Feb 2008 17:48:12 -0500 (EST)
Subject: [Python-3000] PEP 3102
Message-ID: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>

I've been exercising the new keyword-only arguments syntax.  It is absolutely wonderful.  I'm amazed at how many long standing problems it solves elegantly.

Thanks Talin!


Raymond

From martin at v.loewis.de  Thu Feb 14 23:59:04 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Feb 2008 23:59:04 +0100
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802141438s77555923va80239bab5145d3d@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>	
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>	
	<47B4BEF4.5040206@v.loewis.de>
	<ca471dc20802141438s77555923va80239bab5145d3d@mail.gmail.com>
Message-ID: <47B4C7B8.7020009@v.loewis.de>

> Maybe a decent rule of thumb would be:
> 
> - if LANG is set, use the encoding it specifies; if there's no
> encoding in it, assume it's ASCII.
> 
> - if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8
> is much more likely to be correct and useful for 10.4 and before.

You mean, on OSX? Sounds fine to me.

Some people will see moji-bake with this surely; they'll have to set
LANG themselves.

(IOW, ASCII is more likely to be correct, but UTF-8 is more likely
to be useful).

Regards,
Martin

From guido at python.org  Fri Feb 15 00:13:21 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 14 Feb 2008 15:13:21 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>

On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger <python at rcn.com> wrote:
> I've been exercising the new keyword-only arguments syntax.  It is absolutely wonderful.  I'm amazed at how many long standing problems it solves elegantly.

Agreed. Now can you come up with a syntax for positional-only
arguments? So far everybody has failed at that, and there are some use
cases where it's useful too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Feb 15 00:14:34 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 14 Feb 2008 15:14:34 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <47B4C7B8.7020009@v.loewis.de>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<47B4BEF4.5040206@v.loewis.de>
	<ca471dc20802141438s77555923va80239bab5145d3d@mail.gmail.com>
	<47B4C7B8.7020009@v.loewis.de>
Message-ID: <ca471dc20802141514x30b4381bm9ac6ab25766410d6@mail.gmail.com>

On Thu, Feb 14, 2008 at 2:59 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Maybe a decent rule of thumb would be:
>  >
>  > - if LANG is set, use the encoding it specifies; if there's no
>  > encoding in it, assume it's ASCII.
>  >
>  > - if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8
>  > is much more likely to be correct and useful for 10.4 and before.
>
>  You mean, on OSX? Sounds fine to me.

Right.

>  Some people will see moji-bake with this surely; they'll have to set
>  LANG themselves.
>
>  (IOW, ASCII is more likely to be correct, but UTF-8 is more likely
>  to be useful).

Right. You can't completely rule out moji-bake -- people can set LANG
to an incorrect value too.

Can anyone supply a patch to open() in io.py?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From stephen at xemacs.org  Fri Feb 15 01:06:27 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 15 Feb 2008 09:06:27 +0900
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
Message-ID: <87abm36q2k.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > The code that guesses the default stdout encoding isn't very good,
 > especially not on OSX. Suggestions are welcome.

FWIW XEmacs 21.5 (still unreleased) was changed to set all system
codecs (path, stdin, and stdout) to UTF-8 at startup if it's running
on Mac OS (a build-time constant) some months ago, and I haven't seen
a complaint yet from the testers.


From python at rcn.com  Fri Feb 15 01:52:43 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 14 Feb 2008 19:52:43 -0500 (EST)
Subject: [Python-3000] Drop list.reverse()?
Message-ID: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>

Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)?


Raymond

From steven.bethard at gmail.com  Fri Feb 15 01:59:59 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 14 Feb 2008 17:59:59 -0700
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
Message-ID: <d11dcfba0802141659j54cfe7f3x54320b4504d19e5@mail.gmail.com>

On Thu, Feb 14, 2008 at 5:52 PM, Raymond Hettinger <python at rcn.com> wrote:
> Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)?

But .reverse() is in-place, while the other two create new sequences, no?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From python at rcn.com  Fri Feb 15 02:06:32 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 14 Feb 2008 20:06:32 -0500 (EST)
Subject: [Python-3000] Drop list.reverse()?
Message-ID: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net>

[Steven]
> But .reverse() is in-place, while the other two create new sequences, no? 

reversed(s) creates an iterator and s[::-1] creates a new list.  They are not the same as s.reverse() but the use cases overlap quite a bit.


Raymond

From skip at pobox.com  Fri Feb 15 02:33:22 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 14 Feb 2008 19:33:22 -0600
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net>
References: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net>
Message-ID: <18356.60386.774541.55072@montanaro-dyndns-org.local>


    >> But .reverse() is in-place, while the other two create new sequences, no? 

    Raymond> reversed(s) creates an iterator and s[::-1] creates a new list.
    Raymond> They are not the same as s.reverse() but the use cases overlap
    Raymond> quite a bit.

If s.reverse() is dumped shouldn't s.sort() be as well?

Skip

From python at rcn.com  Fri Feb 15 02:44:07 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 14 Feb 2008 20:44:07 -0500 (EST)
Subject: [Python-3000] Drop list.reverse()?
Message-ID: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net>

[Skip]
> If s.reverse() is dumped shouldn't s.sort() be as well?

I don't think so.  When you run reversed(s), the s argument
can be any Sequence (either mutable or immutable) and the
return value is an iterator that doesn't copy the whole
dataset.

In contrast, sorted() takes any iterable and makes a full
copy of it before sorting it.

So reversed(s) can be done without doubling memory
consumption (and is just as fast as iterating forward),
but sorted() is a much heavier operation.

Since reversed() and sorted() were introduced, I've almost
never had occasion to want list.reverse() but I still use
list.sort().

One other reason for keeping list.sort() is that sort
stability lets you make multiple passes:

  mylist.sort(key=attrgetter('name')
  mylist.sort(key=attrgetter('score', reverse=True)
  # Now, it's sorted by reverse score, then alphabetically.

There's not a parallel use case for successive reverses.

Raymond

From skip at pobox.com  Fri Feb 15 02:02:11 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 14 Feb 2008 19:02:11 -0600
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
Message-ID: <18356.58515.512592.321048@montanaro-dyndns-org.local>

    Guido> You can hack around this for now by doing (before printing
    Guido> anything)

    Guido>   sys.stdout._encoding = 'utf-8'

    Guido> but that ought to be a temporary hack until we've figured out the
    Guido> right way to set it.

How about being able to reopen() an open file object, e.g.:

    print(..., file=sys.stdout.reopen(encoding="utf-8")

?

Skip

From greg.ewing at canterbury.ac.nz  Fri Feb 15 03:04:55 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 15 Feb 2008 15:04:55 +1300
Subject: [Python-3000] PEP 3102
In-Reply-To: <ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
Message-ID: <47B4F347.7050102@canterbury.ac.nz>

Guido van Rossum wrote:
> Now can you come up with a syntax for positional-only
> arguments? So far everybody has failed at that, and there are some use
> cases where it's useful too.

def foo(*(a, b,c)):
   ...

i.e. catch them in a * arg which is unpacked in-place.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Fri Feb 15 03:09:21 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 15 Feb 2008 15:09:21 +1300
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
Message-ID: <47B4F451.3070707@canterbury.ac.nz>

Raymond Hettinger wrote:
> Would it be worthwhile to leave reversed(s) and s[::-1] 
 > as the two obvious ways to do it?

But that would leave us without an efficient way of
reversing in-place.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Fri Feb 15 03:11:21 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 15 Feb 2008 15:11:21 +1300
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <18356.58515.512592.321048@montanaro-dyndns-org.local>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
Message-ID: <47B4F4C9.3020402@canterbury.ac.nz>

skip at pobox.com wrote:

> How about being able to reopen() an open file object, e.g.:
> 
>     print(..., file=sys.stdout.reopen(encoding="utf-8")

But reopening implies a lot more than just changing the
encoding.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Fri Feb 15 03:16:14 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 15 Feb 2008 15:16:14 +1300
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net>
References: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net>
Message-ID: <47B4F5EE.8000800@canterbury.ac.nz>

Raymond Hettinger wrote:
> When you run reversed(s), the s argument
> can be any Sequence (either mutable or immutable) and the
> return value is an iterator that doesn't copy the whole
> dataset.

If reversed() became a *view* (a mutable one) rather than
an iterator, there might be more of a case for dropping
list.reverse(), although I'm still not sure.

It might still be a good idea to make reversed() a view
(maybe mutable, maybe not).

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From digitalxero at gmail.com  Fri Feb 15 03:28:28 2008
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Thu, 14 Feb 2008 18:28:28 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <47B4F347.7050102@canterbury.ac.nz>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<47B4F347.7050102@canterbury.ac.nz>
Message-ID: <e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>

On Thu, Feb 14, 2008 at 6:04 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>  > Now can you come up with a syntax for positional-only
>  > arguments? So far everybody has failed at that, and there are some use
>  > cases where it's useful too.
>
>  def foo(*(a, b,c)):
>    ...
>
>  i.e. catch them in a * arg which is unpacked in-place.

I thought the ability to do that was being removed in py3?


throwing some ideas around, for all examples arg1 and arg2 would be
positional only

def test(<arg1, arg2>, arg3, *arg4):
   ...

It's ok, since it clearly groups the positional only args, but the
parser might not be able to handle this.

def test(|arg1, |arg2, arg3, *, arg4):
    ...

Simple, but inelegant. In just looking at the code it seems wrong.

def test(arg1, arg2, |, arg3, *, arg4):
    ...

I like this idea better, use the pipe as a separator for positional
only, and normal arguments.

From nick at craig-wood.com  Fri Feb 15 11:12:19 2008
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Fri, 15 Feb 2008 10:12:19 +0000
Subject: [Python-3000] weakrefs of bound methods
Message-ID: <20080215101219.GA29219@craig-wood.com>

I've just been bitten yet again by the fact you can't have a weakref
to a bound method!  I find myself wanting to do this whenever I have a
registry of callback functions.

Could we in py3k either

1) make weakrefs to bound methods work?  A weakref to a bound method
should mean hold the weakref on the instance and bind the method at
the last moment.

or

2) throw an exception when you attempt to take a weak reference of a
bound method?  Maybe only when it has a single refcount?

Eg

  Python 3.0a2+ (py3k:60831, Feb 15 2008, 09:52:40)
  [GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> from weakref import WeakKeyDictionary
  >>> d=WeakKeyDictionary()
  >>> class A(object):
  ...     def f(self): print("hello")
  ...
  >>> a=A()
  >>> a.f()
  hello
  >>> d[a.f] = True
  >>> d.keys()
  []
  >>> 

I know how to work around the problem.  The fact that it passes
completely silently uses up lots of developer time debugging it
though!

Thanks!
-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From niki.spahiev at gmail.com  Fri Feb 15 13:18:21 2008
From: niki.spahiev at gmail.com (Niki Spahiev)
Date: Fri, 15 Feb 2008 14:18:21 +0200
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <20080208041917.4390C3A4076@sparrow.telecommunity.com>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>	<20080204185642.73659254@bhuda.mired.org>	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>	<47A9FE7B.7090405@cheimes.de>	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>	<47AB6F82.4070308@canterbury.ac.nz>	<20080207221027.261D33A40D7@sparrow.telecommunity.com>	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>	<47ABC58B.7010608@canterbury.ac.nz>
	<20080208041917.4390C3A4076@sparrow.telecommunity.com>
Message-ID: <fp3vu9$uag$1@ger.gmane.org>

Phillip J. Eby wrote:
> That's actually a separate problem from the one Paul and I are 
> talking about, one that I'm not even going to *try* to solve.  (Being 
> as I already have, and failed miserably.)  Hell, I'm not sure Windows 
> even has a way to *tell* whether your app was launched from the 
> command line or otherwise.  I couldn't find one for Win2K, anyway; 
> there might be a way to do it in XP and Vista.

Checking for attached console is one possibility.
AttachConsole(ATTACH_PARENT_CONSOLE)

Remarks
The calling application can call any of the console functions. For 
example, a graphical application can use the console to display 
debugging output.

A process can be attached to one console. If the calling process is 
already attached to a console, AttachConsole fails with ERROR_ACCESS_DENIED.

Niki Spahiev


From pje at telecommunity.com  Fri Feb 15 15:34:41 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 15 Feb 2008 09:34:41 -0500
Subject: [Python-3000] Windows gui vs. console
In-Reply-To: <fp3vu9$uag$1@ger.gmane.org>
References: <448576.20652.qm@web31111.mail.mud.yahoo.com>
	<20080204185642.73659254@bhuda.mired.org>
	<aac2c7cb0802050943r54e7d926o6c93019453b50756@mail.gmail.com>
	<20080205175802.3FBF93A409E@sparrow.telecommunity.com>
	<47A9FE7B.7090405@cheimes.de>
	<79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com>
	<47AB6F82.4070308@canterbury.ac.nz>
	<20080207221027.261D33A40D7@sparrow.telecommunity.com>
	<79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com>
	<47ABC58B.7010608@canterbury.ac.nz>
	<20080208041917.4390C3A4076@sparrow.telecommunity.com>
	<fp3vu9$uag$1@ger.gmane.org>
Message-ID: <20080215143429.92C563A4075@sparrow.telecommunity.com>

At 02:18 PM 2/15/2008 +0200, Niki Spahiev wrote:
>Phillip J. Eby wrote:
> > That's actually a separate problem from the one Paul and I are
> > talking about, one that I'm not even going to *try* to solve.  (Being
> > as I already have, and failed miserably.)  Hell, I'm not sure Windows
> > even has a way to *tell* whether your app was launched from the
> > command line or otherwise.  I couldn't find one for Win2K, anyway;
> > there might be a way to do it in XP and Vista.
>
>Checking for attached console is one possibility.
>AttachConsole(ATTACH_PARENT_CONSOLE)

That requires XP, but while I was looking it up I ran across an 
article that suggested using both .com and .exe versions of the same 
program, where the .com is a console version, and the .exe is the GUI 
version linked to by any installed shortcuts.  Apparently, you can 
have a .com file whose contents are actually an .exe.


From ronaldoussoren at mac.com  Fri Feb 15 15:26:02 2008
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 15 Feb 2008 15:26:02 +0100
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <47B4BEF4.5040206@v.loewis.de>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<47B4BEF4.5040206@v.loewis.de>
Message-ID: <DEE01B33-9245-4410-97CA-1617369E4254@mac.com>


On 14 Feb, 2008, at 23:21, Martin v. L?wis wrote:

>> The code that guesses the default stdout encoding isn't very good,
>> especially not on OSX. Suggestions are welcome.
>
> Unfortunately, Apple isn't very cooperative here. There is no way
> of determining the encoding of a Terminal.app window, AFAIK
> (it's normally UTF-8, unless the user has changed it).

On OSX 10.5 Terminal sets the LANG environment variable, which  
pythonrun.c picks up to set the encoding of sys.stdout when that's  
connected to a tty.  Printing arbitary unicode strings therefore works  
fine on my system (at least using python 2.5, I haven't used 3.0 yet).

I haven't checked if other OS releases behave simularly.

Ronald

>
>
> Regards,
> Martin
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ronaldoussoren%40mac.com


From guido at python.org  Fri Feb 15 18:24:40 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 09:24:40 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <18356.58515.512592.321048@montanaro-dyndns-org.local>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
Message-ID: <ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>

On Thu, Feb 14, 2008 at 5:02 PM,  <skip at pobox.com> wrote:
>     Guido> You can hack around this for now by doing (before printing
>     Guido> anything)
>
>     Guido>   sys.stdout._encoding = 'utf-8'
>
>     Guido> but that ought to be a temporary hack until we've figured out the
>     Guido> right way to set it.
>
>  How about being able to reopen() an open file object, e.g.:
>
>     print(..., file=sys.stdout.reopen(encoding="utf-8")
>
>  ?

I think you're on to something. There's an API to access the
underlying (buffered) binary stream: sys.stdout.buffer. It's possible
to create a new TextIOWrapper on top of that:

sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8")

The only problem is that if the original TextIOWrapper is deleted or
closed, its close() method will close the underlying buffer object,
rendering sys.stdout useless. (Well, actually, the lowest-level raw
I/O object refuses to be closed if the file descriptor is 0, 1 or 2,
but is issues a warning and this is a close call. It would be better
if TextIOWrapper didn't explicitly close() when deleted. That requires
some redesign of the interaction between __del__() and close() in the
IOBase class. Maybe __del__() should just flush() instead of close()?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Feb 15 18:35:42 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 09:35:42 -0800
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <20080215101219.GA29219@craig-wood.com>
References: <20080215101219.GA29219@craig-wood.com>
Message-ID: <ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>

On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood <nick at craig-wood.com> wrote:
> I've just been bitten yet again by the fact you can't have a weakref
>  to a bound method!  I find myself wanting to do this whenever I have a
>  registry of callback functions.
>
>  Could we in py3k either
>
>  1) make weakrefs to bound methods work?  A weakref to a bound method
>  should mean hold the weakref on the instance and bind the method at
>  the last moment.

I think you're somehow confused about how weakrefs are supposed to
work. They work fine!

Python 3.0a2+ (py3k:60836, Feb 15 2008, 09:21:45)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import weakref
>>> class C:
...  def __init__(self, x=0): self.x = x
...  def f(self): return self.x
...
>>> f = C(42).f
>>> r = weakref.ref(f)
>>> r
<weakref at 0xf7be8fcc; to 'method' at 0xf7f0c48c (f)>
>>> r()
<bound method C.f of <__main__.C object at 0xf7bfc4ac>>
>>> r()()
42
>>> del f
>>> r()
>>> r
<weakref at 0xf7be8fcc; dead>
>>>

>  or
>
>  2) throw an exception when you attempt to take a weak reference of a
>  bound method?  Maybe only when it has a single refcount?
>
>  Eg
>
>   Python 3.0a2+ (py3k:60831, Feb 15 2008, 09:52:40)
>   [GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2
>   Type "help", "copyright", "credits" or "license" for more information.
>   >>> from weakref import WeakKeyDictionary
>   >>> d=WeakKeyDictionary()
>   >>> class A(object):
>   ...     def f(self): print("hello")
>   ...
>   >>> a=A()
>   >>> a.f()
>   hello
>   >>> d[a.f] = True
>   >>> d.keys()
>   []
>   >>>
>
>  I know how to work around the problem.  The fact that it passes
>  completely silently uses up lots of developer time debugging it
>  though!

Sorry, you don't seem to understand how weak references work.The bound
method object a.f goes away because you're not keeping any other
reference to it. Try this instead:

>>> d = weakref.WeakKeyDictionary()
>>> f = C(100).f
>>> d[f] = 200
>>> d
<WeakKeyDictionary at 4156538988>
>>> d.keys()
[<bound method C.f of <__main__.C object at 0xf7bfc54c>>]
>>> del f
>>> d.keys()
[<bound method C.f of <__main__.C object at 0xf7bfc54c>>]
>>>
>>> 1
1
>>> d.keys()
[]
>>>

(The second time I printed d.keys() it wasn't gone yet because the
hidden variable '_' was holding on to the last object printed, which
was the list of d's keys.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Feb 15 18:37:43 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 09:37:43 -0800
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20802150937h222d67e7xf742bdaca16f521c@mail.gmail.com>

On Thu, Feb 14, 2008 at 4:52 PM, Raymond Hettinger <python at rcn.com> wrote:
> Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)?

I think we should keep it. There's no faster way to reverse a list in place.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Fri Feb 15 18:42:04 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 15 Feb 2008 10:42:04 -0700
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
Message-ID: <aac2c7cb0802150942k6811c617y4cc88f9e4693cd4a@mail.gmail.com>

On Fri, Feb 15, 2008 at 10:24 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Feb 14, 2008 at 5:02 PM,  <skip at pobox.com> wrote:
>  >     Guido> You can hack around this for now by doing (before printing
>  >     Guido> anything)
>  >
>  >     Guido>   sys.stdout._encoding = 'utf-8'
>  >
>  >     Guido> but that ought to be a temporary hack until we've figured out the
>  >     Guido> right way to set it.
>  >
>  >  How about being able to reopen() an open file object, e.g.:
>  >
>  >     print(..., file=sys.stdout.reopen(encoding="utf-8")
>  >
>  >  ?
>
>  I think you're on to something. There's an API to access the
>  underlying (buffered) binary stream: sys.stdout.buffer. It's possible
>  to create a new TextIOWrapper on top of that:
>
>  sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8")
>
>  The only problem is that if the original TextIOWrapper is deleted or
>  closed, its close() method will close the underlying buffer object,
>  rendering sys.stdout useless. (Well, actually, the lowest-level raw
>  I/O object refuses to be closed if the file descriptor is 0, 1 or 2,
>  but is issues a warning and this is a close call. It would be better
>  if TextIOWrapper didn't explicitly close() when deleted. That requires
>  some redesign of the interaction between __del__() and close() in the
>  IOBase class. Maybe __del__() should just flush() instead of close()?

That'd rely on prompt deletion for correct behaviour.  Why not an
explicit detach method?

buf = sys.stdout.buffer
sys.stdout.detach()
sys.stdout = io.TextIOWrapper(buf, "utf-8")

-- 
Adam Olsen, aka Rhamphoryncus

From digitalxero at gmail.com  Fri Feb 15 18:42:39 2008
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Fri, 15 Feb 2008 09:42:39 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<47B4F347.7050102@canterbury.ac.nz>
	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>
	<47B5C1E4.3000808@cornell.edu>
	<e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
Message-ID: <e9764b730802150942q548c4e71pbaa4bfc1438aba46@mail.gmail.com>

On Fri, Feb 15, 2008 at 9:37 AM, Dj Gilcrease <digitalxero at gmail.com> wrote:
>  The intent is to make it so (Assuming arg1 and arg2 are positional
>  only) you cannot do test(arg3=9, arg1=7, arg2=8), thus you would be
>  required to do test(7, 8, 9) {since doing test(7, 8, arg3=9) would
>  raise an error of positional arg being set by position and keyword at
>  the same time}
>

Little correction, test(7, 8, arg3=9) should work and does not raise
the error I though it did

From digitalxero at gmail.com  Fri Feb 15 18:37:50 2008
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Fri, 15 Feb 2008 09:37:50 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <47B5C1E4.3000808@cornell.edu>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<47B4F347.7050102@canterbury.ac.nz>
	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>
	<47B5C1E4.3000808@cornell.edu>
Message-ID: <e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>

On Fri, Feb 15, 2008 at 8:46 AM, Joel Bender <jjb5 at cornell.edu> wrote:
>  How about a BNFish notation?  Use brackets around optional parameters.
>
>      def test([arg1, [arg2,]] arg3):
>          ...

If I understand right, positional only arguments would not be optional
in any way shape or form, thats what default arguments are for.
Positional only args would be arguments that you cannot specify by
keyword, so having a positional only arg that has a default value,
thus making it optional just seems broken to me.

The notation you recommended just confuses the hell outa me when
reading it, it is neither clean nor elegant for the stated intent,
which is a method of defining positional only arguments.

eg.
def test([arg1=1, arg2=2,] arg3): return (arg1, arg2, arg3)
    assert test(9) == (1, 2, 9)
    assert test(7, 8, 9) == (7, 8, 9)
    test(8, 9) == Exception
    test(arg3=9) == ???

This example clearly shows what confuses me about this notation. arg1
and arg2 are supposed to be positional only, but your first example
the first argument you set end up in the 3rd position for no reason
other then you defined default arguments before the positional
arguments, which should and does raise "SyntaxError: non-default
argument follows default argument"


The intent is to make it so (Assuming arg1 and arg2 are positional
only) you cannot do test(arg3=9, arg1=7, arg2=8), thus you would be
required to do test(7, 8, 9) {since doing test(7, 8, arg3=9) would
raise an error of positional arg being set by position and keyword at
the same time}

From jjb5 at cornell.edu  Fri Feb 15 17:46:28 2008
From: jjb5 at cornell.edu (Joel Bender)
Date: Fri, 15 Feb 2008 11:46:28 -0500
Subject: [Python-3000] PEP 3102
In-Reply-To: <e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>	<47B4F347.7050102@canterbury.ac.nz>
	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>
Message-ID: <47B5C1E4.3000808@cornell.edu>

Dj Gilcrease wrote:
> On Thu, Feb 14, 2008 at 6:04 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Guido van Rossum wrote:
>>  > Now can you come up with a syntax for positional-only
>>  > arguments? So far everybody has failed at that, and there are some use
>>  > cases where it's useful too.
> 
> 
> throwing some ideas around...

How about a BNFish notation?  Use brackets around optional parameters.

     def test([arg1, [arg2,]] arg3):
         ...

I haven't found an example of what the value of an unspecified 
positional only argument would be, I think it should be required:

     def test([arg1=1,] arg2): return (arg1, arg2)

This could also accommodate trailing positional only args, even though 
the existing syntax already has it (but keep reading):

     def test(arg1 [, arg2=2]): return (arg1, arg2)
     assert test(1) == (1, 2)
     assert test(1,3) == (1, 3)

If both were provided:

     def test([arg1=1,] arg2 [, arg3=3]): ... # error

I would expect it to be an "ambiguous combination of optional 
positional-only arguments" syntax error (as opposed to runtime error).

If you wanted arg2 specified when arg1 is:

     def test([arg1=1, arg2=2,] arg3): return (arg1, arg2, arg3)
     assert test(9) == (1, 2, 9)
     assert test(7, 8, 9) == (7, 8, 9)

     test(8, 9)                   # error

With nesting you can specify which one is more optional:

     def test([[arg1=1,] arg2=2,] arg3): return (arg1, arg2, arg3)
     assert test(9) == (1, 2, 9)
     assert test(8, 9) == (1, 8, 9)

     def test([arg1=1, [arg2=2,]] arg3): return (arg1, arg2, arg3)
     assert test(8, 9) == (8, 2, 9)

Capturing all of the args except the last one:

     def test([*arg1,] arg2): return (arg1, arg2)
     assert test(9) == ((), 9)
     assert test(8, 9) == ((8,), 9)
     assert test(7, 8, 9) == ((7, 8), 9)

First is a little more important:

     def test([arg1=1, *arg2,] arg3): return (arg1, arg2, arg3)
     assert test(9) == (1, (), 9)
     assert test(8, 9) == (8, (), 9)
     assert test(7, 8, 9) == (7, (8,), 9)

Force the end of positional arguments to switch to keywords:

     def test([*arg1,] arg2, *, arg3=3): return (arg1, arg2, arg3)

Specifying more than one dumping spot for additional positional 
parameters is ambiguous:

     def test([arg1=1, *arg2,] arg3, *arg4): ... # error

Specifying default values for a positional argument may make it ambiguous:

     def test([arg1=1,] arg2, arg3=3): return (arg1, arg2, arg3)

This is OK, but not recommended, it's more complicated than it needs to be:

     def test(arg1 [, arg2=1 [, arg3=2]]): return (arg1, arg2, arg3)

And now for something completely different:

     def test(arg1 [[, arg2=2], arg3=3]): return (arg1, arg2, arg3)
     assert test(8) == (8, 2, 3)
     assert test(8, 9) == (8, 2, 9)
     assert test(7, 8, 9) == (7, 8, 9)

Like the other dumping spot:

     def test(arg1 [[, *arg2], arg3=3]): return (arg1, arg2, arg3)
     assert test(8) == (8, (), 3)
     assert test(8, 9) == (8, (), 9)
     assert test(7, 8, 9) == (7, (8,), 9)

Nested galore:

     def test([arg1=1, [[*arg2,] arg3=3,]] arg4):
         return (arg1, arg2, arg3, arg4)
     assert test(7) = (1, (), 3, 7)
     assert test(7, 8) = (7, (), 3, 8)
     assert test(7, 8, 9) = (7, (), 8, 9)
     assert test(7, 8, 9, 10) = (7, (8,), 9, 10)

The notation for an optional tuple that gets ripped apart seems busy, 
but is still grokable:

     def test([(arg1, arg2)=(1, 2),] arg3): return (arg1, arg2, arg3)
     assert test(8) == (1, 2, 8)
     assert test((7, 8), 9) == (7, 8, 9)


Joel

From pje at telecommunity.com  Fri Feb 15 19:26:21 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 15 Feb 2008 13:26:21 -0500
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com
 >
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
Message-ID: <20080215182609.0251A3A4075@sparrow.telecommunity.com>

At 09:35 AM 2/15/2008 -0800, Guido van Rossum wrote:
>On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood <nick at craig-wood.com> wrote:
> > I've just been bitten yet again by the fact you can't have a weakref
> >  to a bound method!  I find myself wanting to do this whenever I have a
> >  registry of callback functions.
> >
> >  Could we in py3k either
> >
> >  1) make weakrefs to bound methods work?  A weakref to a bound method
> >  should mean hold the weakref on the instance and bind the method at
> >  the last moment.
>
>I think you're somehow confused about how weakrefs are supposed to
>work. They work fine!

The use case here is actually to keep the callback around only so 
long as the object exists.

There are various ways to do this, of course, that don't require 
changing the language or stdlib.  One of the simplest (for 2.4 and 
up) is to create a callable weakref subclass that emulates a bound 
method.  Another is to create a bound method whose im_self is a weakref proxy.

I've never actually encountered a usecase for keeping a standard 
weakref to a standard bound method, though.  It's pretty useless, 
because bound methods are nearly always immediately disposed of, and 
there's no logical place (or reason) to store them.  So, Nick's query 
is (in effect), can we either:

1. make ref(method) create a "weak method" instead, or

2. make ref(method) an error instead of useless behavior that gives 
you the illusion of working code


From guido at python.org  Fri Feb 15 20:41:14 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 11:41:14 -0800
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <20080215182609.0251A3A4075@sparrow.telecommunity.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
Message-ID: <ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>

On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:35 AM 2/15/2008 -0800, Guido van Rossum wrote:
>  >On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood <nick at craig-wood.com> wrote:
>  > > I've just been bitten yet again by the fact you can't have a weakref
>  > >  to a bound method!  I find myself wanting to do this whenever I have a
>  > >  registry of callback functions.
>  > >
>  > >  Could we in py3k either
>  > >
>  > >  1) make weakrefs to bound methods work?  A weakref to a bound method
>  > >  should mean hold the weakref on the instance and bind the method at
>  > >  the last moment.
>  >
>  >I think you're somehow confused about how weakrefs are supposed to
>  >work. They work fine!
>
>  The use case here is actually to keep the callback around only so
>  long as the object exists.

Ah, I see. Nick actually wrote this but because I'm so familiar with
the concept of a bound method object I didn't see what his actual
issue was. Sorry Nick!

>  There are various ways to do this, of course, that don't require
>  changing the language or stdlib.  One of the simplest (for 2.4 and
>  up) is to create a callable weakref subclass that emulates a bound
>  method.  Another is to create a bound method whose im_self is a weakref proxy.
>
>  I've never actually encountered a usecase for keeping a standard
>  weakref to a standard bound method, though.  It's pretty useless,
>  because bound methods are nearly always immediately disposed of, and
>  there's no logical place (or reason) to store them.  So, Nick's query
>  is (in effect), can we either:
>
>  1. make ref(method) create a "weak method" instead, or
>
>  2. make ref(method) an error instead of useless behavior that gives
>  you the illusion of working code

#2 is easily implemented (just remove the machinery to support weak
refs from the bound method object) but the question is, will it break
anybody's code -- and if so, do we care enough to break it? (In Py3k,
that is.)

#1 isn't so easy in terms of how you'd implement it -- I don't think
the weakref.ref() call currently has a way to customize what kind of
thing you get back. I'd be open for a patch containing working code
that can be generalized to other cases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jjb5 at cornell.edu  Fri Feb 15 20:55:46 2008
From: jjb5 at cornell.edu (Joel Bender)
Date: Fri, 15 Feb 2008 14:55:46 -0500
Subject: [Python-3000] PEP 3102
In-Reply-To: <e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>	<47B4F347.7050102@canterbury.ac.nz>	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>	<47B5C1E4.3000808@cornell.edu>
	<e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
Message-ID: <47B5EE42.9000001@cornell.edu>

Dj Gilcrease wrote:

> If I understand right, positional only arguments would not be optional
> in any way shape or form, thats what default arguments are for.

I have been expecting that by providing a default value to an argument 
in the definition that you are making it optional, be it keyword or 
positional.  The PEP doesn't have an example of what this means:

     def f(a, b, *, c): ...

Is 'c' a keyword only argument that must be specified?  In that case I 
don't consider the following confusing, there is a clear indication of 
which arguments are positional and which ones are keyword only.

     def f(a, b=2, *, c=3): ...

Given this definition, which isn't in the PEP but maybe it should be:

     def f(x, *):

I would be perfectly happy if there was an exception for f(x=3), just as 
there is for math.sin(x=3).

> Positional only args would be arguments that you cannot specify by
> keyword, so having a positional only arg that has a default value,
> thus making it optional just seems broken to me.

I think this is a case of differing viewpoints.  When I see a definition 
like this:

     def f(a, b=3): ...

I think of 'b' being an optional argument, you seem to see it 
differently.  I don't use keywords when I specify the values of 
positional arguments, so if I see this in documentation:

     min(iterable[, key=func]) -> value

I don't call min(iterable=[3]) because it returns a TypeError.  So when 
I see something like this:

     def f(x, y):

I've never gotten in the habit of even trying f(y=2, x=3), and I would 
consider it confusing even if and when it works.  Then again, perhaps 
it's because I've never felt compelled to reorder the sequence of values 
of positional parameters when I call a function.


Joel

From g.brandl at gmx.net  Fri Feb 15 21:25:33 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 15 Feb 2008 21:25:33 +0100
Subject: [Python-3000] PEP 3102
In-Reply-To: <e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>	<47B4F347.7050102@canterbury.ac.nz>	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>	<47B5C1E4.3000808@cornell.edu>
	<e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
Message-ID: <fp4sfk$dlc$1@ger.gmane.org>

Dj Gilcrease schrieb:
> On Fri, Feb 15, 2008 at 8:46 AM, Joel Bender <jjb5 at cornell.edu> wrote:
>>  How about a BNFish notation?  Use brackets around optional parameters.
>>
>>      def test([arg1, [arg2,]] arg3):
>>          ...
> 
> If I understand right, positional only arguments would not be optional
> in any way shape or form, thats what default arguments are for.
> Positional only args would be arguments that you cannot specify by
> keyword, so having a positional only arg that has a default value,
> thus making it optional just seems broken to me.

Being positional-only and having a default value does not conflict.
There's nothing at all broken with

def foo(!x, !y='bar'):
     pass

where !x and !y would be positional-only arguments. You can't of course
have something like

def foo(!x, !y='bar', z):
     pass

but you can't have that today either.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From pje at telecommunity.com  Fri Feb 15 21:52:43 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 15 Feb 2008 15:52:43 -0500
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.co
 m>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
Message-ID: <20080215205231.93DD43A4075@sparrow.telecommunity.com>

At 11:41 AM 2/15/2008 -0800, Guido van Rossum wrote:
>On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby 
><pje at telecommunity.com> wrote:
> >  I've never actually encountered a usecase for keeping a standard
> >  weakref to a standard bound method, though.  It's pretty useless,
> >  because bound methods are nearly always immediately disposed of, and
> >  there's no logical place (or reason) to store them.  So, Nick's query
> >  is (in effect), can we either:
> >
> >  1. make ref(method) create a "weak method" instead, or
> >
> >  2. make ref(method) an error instead of useless behavior that gives
> >  you the illusion of working code
>
>#2 is easily implemented (just remove the machinery to support weak
>refs from the bound method object) but the question is, will it break
>anybody's code -- and if so, do we care enough to break it? (In Py3k,
>that is.)

It's hard to imagine what could be using it, but I guess it's possible.


>#1 isn't so easy in terms of how you'd implement it -- I don't think
>the weakref.ref() call currently has a way to customize what kind of
>thing you get back. I'd be open for a patch containing working code
>that can be generalized to other cases.

Well, one could allow for a __weaken__ method that would be called if 
present, instead of the default constructor.  The part I'm not 
immediately sure of is where to call it *from*, so that you can still 
subclass weakref.ref and not end up in a strange delegation loop.  I 
guess you could call it if and only if weakref.ref.__new__ was passed 
weakref.ref as the class to construct.


From guido at python.org  Fri Feb 15 22:20:03 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 13:20:03 -0800
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <20080215205231.93DD43A4075@sparrow.telecommunity.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
Message-ID: <ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>

On Fri, Feb 15, 2008 at 12:52 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:41 AM 2/15/2008 -0800, Guido van Rossum wrote:
>  >On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby
>  ><pje at telecommunity.com> wrote:
>
> > >  I've never actually encountered a usecase for keeping a standard
>  > >  weakref to a standard bound method, though.  It's pretty useless,
>  > >  because bound methods are nearly always immediately disposed of, and
>  > >  there's no logical place (or reason) to store them.  So, Nick's query
>  > >  is (in effect), can we either:
>  > >
>  > >  1. make ref(method) create a "weak method" instead, or
>  > >
>  > >  2. make ref(method) an error instead of useless behavior that gives
>  > >  you the illusion of working code
>  >
>  >#2 is easily implemented (just remove the machinery to support weak
>  >refs from the bound method object) but the question is, will it break
>  >anybody's code -- and if so, do we care enough to break it? (In Py3k,
>  >that is.)
>
>  It's hard to imagine what could be using it, but I guess it's possible.
>
>
>
>  >#1 isn't so easy in terms of how you'd implement it -- I don't think
>  >the weakref.ref() call currently has a way to customize what kind of
>  >thing you get back. I'd be open for a patch containing working code
>  >that can be generalized to other cases.
>
>  Well, one could allow for a __weaken__ method that would be called if
>  present, instead of the default constructor.  The part I'm not
>  immediately sure of is where to call it *from*, so that you can still
>  subclass weakref.ref and not end up in a strange delegation loop.  I
>  guess you could call it if and only if weakref.ref.__new__ was passed
>  weakref.ref as the class to construct.

I think str() does this.

But maybe given how rare the use case is, it would be easier to just
create a custom class in weakref.py that does what Nick requested.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nicko at nicko.org  Fri Feb 15 23:53:18 2008
From: nicko at nicko.org (Nicko van Someren)
Date: Fri, 15 Feb 2008 22:53:18 +0000
Subject: [Python-3000] Drop list.reverse()?
In-Reply-To: <ca471dc20802150937h222d67e7xf742bdaca16f521c@mail.gmail.com>
References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net>
	<ca471dc20802150937h222d67e7xf742bdaca16f521c@mail.gmail.com>
Message-ID: <D34AADFF-E2B8-4C72-B71F-57C7C188D581@nicko.org>

On 15 Feb 2008, at 17:37, Guido van Rossum wrote:

> On Thu, Feb 14, 2008 at 4:52 PM, Raymond Hettinger <python at rcn.com>  
> wrote:
>> Would it be worthwhile to leave reversed(s) and s[::-1] as the two  
>> obvious ways to do it (ways that work with any Sequence, not just  
>> lists)?
>
> I think we should keep it. There's no faster way to reverse a list  
> in place.

Out of curiosity, I checked just how much faster it really is.  The  
answer is that for in place reversal it's more than an order of  
magnitude faster than using either of the suggested alternatives:
   >>> timeit.Timer('s.reverse()', setup='s=range(1000)').timeit()
   0.80964398384094238
   >>> timeit.Timer('s[:] = s[::-1]', setup='s=range(1000)').timeit()
   9.2884359359741211
   >>> timeit.Timer('s[:] = reversed(s)',  
setup='s=range(1000)').timeit()
   16.757926940917969

Not surprising, given that no objects were created or harmed in the  
reversal of this list...

Interestingly, it's not the fastest way to make a reverse copy:
   >>> timeit.Timer('s2 = list(s); s2.reverse()',  
setup='s=range(1000)').timeit()
   6.4023230075836182
   >>> timeit.Timer('s2 = s[::-1]', setup='s=range(1000)').timeit()
   5.3177969455718994
   >>> timeit.Timer('s2 = list(reversed(s))',  
setup='s=range(1000)').timeit()
   12.906921148300171

(Times take on an Intel Mac, Python 2.5.1)

	Nicko


From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:13:25 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:13:25 +1300
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
Message-ID: <47B62AA5.20508@canterbury.ac.nz>

Guido van Rossum wrote:
> It would be better
> if TextIOWrapper didn't explicitly close() when deleted. 
> Maybe __del__() should just flush() instead of close()?

Sounds good to me.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:19:01 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:19:01 +1300
Subject: [Python-3000] PEP 3102
In-Reply-To: <e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<47B4F347.7050102@canterbury.ac.nz>
	<e9764b730802141828t45c04fb8k5c834ba24269e196@mail.gmail.com>
	<47B5C1E4.3000808@cornell.edu>
	<e9764b730802150937x2d7fec49o93c6291603e303ac@mail.gmail.com>
Message-ID: <47B62BF5.90501@canterbury.ac.nz>

Dj Gilcrease wrote:
> Positional only args would be arguments that you cannot specify by
> keyword, so having a positional only arg that has a default value,
> thus making it optional just seems broken to me.

There's no logical problem with optional positional-only
args -- it just means you have to supply all the previous
args as well.

E.g. if the third arg of foo is positional-only and
optional, you can call it as either

    foo(a, b)
    foo(a, b, c)

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:21:12 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:21:12 +1300
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <aac2c7cb0802150942k6811c617y4cc88f9e4693cd4a@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
	<aac2c7cb0802150942k6811c617y4cc88f9e4693cd4a@mail.gmail.com>
Message-ID: <47B62C78.3050609@canterbury.ac.nz>

Adam Olsen wrote:
> That'd rely on prompt deletion for correct behaviour.  Why not an
> explicit detach method?
> 
> buf = sys.stdout.buffer
> sys.stdout.detach()
> sys.stdout = io.TextIOWrapper(buf, "utf-8")

Or keep an explicit count of the number of unclosed
wrappers that are referencing a buffer.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:26:41 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:26:41 +1300
Subject: [Python-3000] Weakref protocol? (Re: weakrefs of bound methods)
In-Reply-To: <20080215182609.0251A3A4075@sparrow.telecommunity.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
Message-ID: <47B62DC1.4070606@canterbury.ac.nz>

Phillip J. Eby wrote:
> So, Nick's query is (in effect), can we either:
> 
> 1. make ref(method) create a "weak method" instead, or
> 
> 2. make ref(method) an error instead of useless behavior that gives 
> you the illusion of working code

Maybe what we want here is a weakref protocol. So instead
of weakref() directly creating a weak ref object, it would
call __weakref__ on its argument.

Most objects would respond by creating a standard weak
ref object, but special cases such as bound methods
could do something different.

-- 
Greg

From guido at python.org  Sat Feb 16 01:36:34 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 16:36:34 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <47B62C78.3050609@canterbury.ac.nz>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
	<aac2c7cb0802150942k6811c617y4cc88f9e4693cd4a@mail.gmail.com>
	<47B62C78.3050609@canterbury.ac.nz>
Message-ID: <ca471dc20802151636q221da660n503dabb3e45c7629@mail.gmail.com>

On Fri, Feb 15, 2008 at 4:21 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Adam Olsen wrote:
>  > That'd rely on prompt deletion for correct behaviour.  Why not an
>  > explicit detach method?
>  >
>  > buf = sys.stdout.buffer
>  > sys.stdout.detach()
>  > sys.stdout = io.TextIOWrapper(buf, "utf-8")
>
>  Or keep an explicit count of the number of unclosed
>  wrappers that are referencing a buffer.

Yuck. We have to do this for sockets and it's a mess.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:34:07 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:34:07 +1300
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <20080215205231.93DD43A4075@sparrow.telecommunity.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
Message-ID: <47B62F7F.2070906@canterbury.ac.nz>

Phillip J. Eby wrote:
> The part I'm not 
> immediately sure of is where to call it *from*, so that you can still 
> subclass weakref.ref and not end up in a strange delegation loop.

The function for creating a weak ref would need to be
different from the weakref class itself. One of them
would have to be renamed (unless you want to do something
magical and potentially confusing).

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Feb 16 01:35:35 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 16 Feb 2008 13:35:35 +1300
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
	<ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
Message-ID: <47B62FD7.5060409@canterbury.ac.nz>

Guido van Rossum wrote:
> But maybe given how rare the use case is, it would be easier to just
> create a custom class in weakref.py that does what Nick requested.

Nick would still forget to use it and get strange bugs
in his code, though!

-- 
Greg

From martin at v.loewis.de  Sat Feb 16 02:03:33 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 16 Feb 2008 02:03:33 +0100
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
Message-ID: <47B63665.6020808@v.loewis.de>

> I think you're on to something. There's an API to access the
> underlying (buffered) binary stream: sys.stdout.buffer. It's possible
> to create a new TextIOWrapper on top of that:
> 
> sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8")

While it might be possible to switch an output stream that way, it's
fairly difficult to do that correctly for input, no? The wrapper
may have already read data from the underlying stream which have not
been given to the application. If you don't want to lose these
data, you either need to copy them over to the new stream, or unread
them from the underlying buffer. That's tricky as some of the data might
be in the codec.

Regards,
Martin

From guido at python.org  Sat Feb 16 03:11:39 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 15 Feb 2008 18:11:39 -0800
Subject: [Python-3000] Does Py3k's print offer any unicode encoding help?
In-Reply-To: <47B63665.6020808@v.loewis.de>
References: <18356.44529.51137.327615@montanaro-dyndns-org.local>
	<ca471dc20802141402k12c48c4bia64cc16a9d6d9457@mail.gmail.com>
	<18356.58515.512592.321048@montanaro-dyndns-org.local>
	<ca471dc20802150924u7628f33fw999b60bc1946525b@mail.gmail.com>
	<47B63665.6020808@v.loewis.de>
Message-ID: <ca471dc20802151811g15830745ica8ddfdd3179aafc@mail.gmail.com>

In general these things are only safe when done before any I/O happens.

But even so it's only a stop-gap measure; the app shouldn't have to do
this, we should fix the guessing algorithm.

On Fri, Feb 15, 2008 at 5:03 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I think you're on to something. There's an API to access the
>  > underlying (buffered) binary stream: sys.stdout.buffer. It's possible
>  > to create a new TextIOWrapper on top of that:
>  >
>  > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8")
>
>  While it might be possible to switch an output stream that way, it's
>  fairly difficult to do that correctly for input, no? The wrapper
>  may have already read data from the underlying stream which have not
>  been given to the application. If you don't want to lose these
>  data, you either need to copy them over to the new stream, or unread
>  them from the underlying buffer. That's tricky as some of the data might
>  be in the codec.
>
>  Regards,
>  Martin
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Sat Feb 16 05:32:57 2008
From: talin at acm.org (Talin)
Date: Fri, 15 Feb 2008 20:32:57 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
Message-ID: <47B66779.8000905@acm.org>

Raymond Hettinger wrote:
> I've been exercising the new keyword-only arguments syntax.  It is absolutely wonderful.  I'm amazed at how many long standing problems it solves elegantly.
> 
> Thanks Talin!

Well, that made my day :)

-- Talin

From rasky at develer.com  Sat Feb 16 16:20:28 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 16 Feb 2008 15:20:28 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
Message-ID: <fp6uvs$p9$1@ger.gmane.org>

Hello,

CPython 2.x (and 3.x) under Win32 has an issue with sys.argv. The list is 
computed using the ANSI version of the windows APIs[*]. The problem is 
apparent when you have a file/directory which can't be represented in the 
system encoding (eg: a japanese-named file or directory on a Western 
Windows), because the Windows ANSI API will encode the filename to the 
system encoding using what we call the "replace" policy, and sys.argv[] 
will contain an entry like "c:\\foo\\??????????????.dat".

At the moment, there's simply no way of passing such a file to a Python 
script/application as an argument (eg: if you double-click on that file, 
and the file is associated to a Python application). This is a wide-
spread problem among Python applications; eg. if you click on a 
Japanese .torrent file, ABC (a Bittorent client written in Python) won't 
be able to open it and will complain "cannot access 
file ??????????.torrent".

I understand that fixing this properly in the 2.x serie might have 
backward compatibility issues, but I propose that this be fixed at least 
in the Python 3.x serie, and I volunteer to write a patch. I would be 
glad if someone expert with ANSI/Unicode/Windows (MvL?) would show me 
what he believes being the correct way of approaching this problem. 

My suggestion is that:

* At the Python level, we still expose a single sys.argv[], which will 
contain unicode strings. I think this exactly matches what Py3k does now. 
(Back in the time, there were proposals to add a sys.argvu, but I guess 
it does not make sense right now).
* At the C level, I believe it involves using GetCommandLineW() and 
CommandLineToArgvW() in WinMain.c, but should Py_Main/PySys_SetArgv() be 
changed to also accept wchar_t** arguments? Or is it better to allow for 
NULL to be passed (under Windows at least), so that the Windows code-path 
in there can use GetCommandLineW()/CommandLineToArgvW() to get the 
current process' arguments?

Thanks!

[*] In detail: it actually comes from __argc/__argv (see WinMain.c), 
which in turn are computed by the CRT startup code, which would adapt to 
user's choice but Python is being compiled in ANSI mode.
-- 
Giovanni Bajo


From guido at python.org  Sat Feb 16 16:49:22 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 16 Feb 2008 07:49:22 -0800
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp6uvs$p9$1@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>
Message-ID: <ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>

Thanks for reminding us of this!

Is there any chance that you can submit a patch (or even the
beginnings of one) for this?

At the very least, can you add the issue to the tracker so we'll have
a permanent reminder until it's resolved?

On Feb 16, 2008 7:20 AM, Giovanni Bajo <rasky at develer.com> wrote:
> Hello,
>
> CPython 2.x (and 3.x) under Win32 has an issue with sys.argv. The list is
> computed using the ANSI version of the windows APIs[*]. The problem is
> apparent when you have a file/directory which can't be represented in the
> system encoding (eg: a japanese-named file or directory on a Western
> Windows), because the Windows ANSI API will encode the filename to the
> system encoding using what we call the "replace" policy, and sys.argv[]
> will contain an entry like "c:\\foo\\??????????????.dat".
>
> At the moment, there's simply no way of passing such a file to a Python
> script/application as an argument (eg: if you double-click on that file,
> and the file is associated to a Python application). This is a wide-
> spread problem among Python applications; eg. if you click on a
> Japanese .torrent file, ABC (a Bittorent client written in Python) won't
> be able to open it and will complain "cannot access
> file ??????????.torrent".
>
> I understand that fixing this properly in the 2.x serie might have
> backward compatibility issues, but I propose that this be fixed at least
> in the Python 3.x serie, and I volunteer to write a patch. I would be
> glad if someone expert with ANSI/Unicode/Windows (MvL?) would show me
> what he believes being the correct way of approaching this problem.
>
> My suggestion is that:
>
> * At the Python level, we still expose a single sys.argv[], which will
> contain unicode strings. I think this exactly matches what Py3k does now.
> (Back in the time, there were proposals to add a sys.argvu, but I guess
> it does not make sense right now).
> * At the C level, I believe it involves using GetCommandLineW() and
> CommandLineToArgvW() in WinMain.c, but should Py_Main/PySys_SetArgv() be
> changed to also accept wchar_t** arguments? Or is it better to allow for
> NULL to be passed (under Windows at least), so that the Windows code-path
> in there can use GetCommandLineW()/CommandLineToArgvW() to get the
> current process' arguments?
>
> Thanks!
>
> [*] In detail: it actually comes from __argc/__argv (see WinMain.c),
> which in turn are computed by the CRT startup code, which would adapt to
> user's choice but Python is being compiled in ANSI mode.
> --
> Giovanni Bajo
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rasky at develer.com  Sat Feb 16 17:32:43 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 16 Feb 2008 16:32:43 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
Message-ID: <fp737b$s0c$1@ger.gmane.org>

On Sat, 16 Feb 2008 07:49:22 -0800, Guido van Rossum wrote:

> Thanks for reminding us of this!
> 
> Is there any chance that you can submit a patch (or even the beginnings
> of one) for this?

Sure, during next week. I'd appreciate if, meanwhile, a core developer 
commented on my proposal at the C level, so that I don't write a totally 
wrong patch to begin with. I'm not really familiar with CPython hacking.

> At the very least, can you add the issue to the tracker so we'll have a
> permanent reminder until it's resolved?

Done: issue 2128. I didn't find a way to mark it Windows-only in the 
metadata (I guess there is not one).
-- 
Giovanni Bajo


From lists at cheimes.de  Sat Feb 16 17:51:08 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 16 Feb 2008 17:51:08 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp737b$s0c$1@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org>
Message-ID: <47B7147C.10506@cheimes.de>

Giovanni Bajo wrote:
> Sure, during next week. I'd appreciate if, meanwhile, a core developer 
> commented on my proposal at the C level, so that I don't write a totally 
> wrong patch to begin with. I'm not really familiar with CPython hacking.

We have dropped support for 9x, ME and NT4 for the 2.6 and 3.0 series.
This allows us to use wide API methods without restrictions. IIRC Win2k
and newer versions of CE have full support for wide unicode.

Several places need an update to support unicode on Windows. This
includes argv, the environment variables and file names / path handling.
 See http://bugs.python.org/issue1342 for another example.

In the last months Amaury, Martin von L?wis and I have done most of the
Windows work. Any help from an experienced Windows developer is greatly
appreciated! We *need* an additional Windows developer.

So if you are interested in getting the problems fixed I welcome you on
board. I suggest that you start by writing down a list of required
changes (file names, function names) and a proposal how to pursue the
issues.

Christian

From rasky at develer.com  Sat Feb 16 18:06:45 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 16 Feb 2008 17:06:45 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
Message-ID: <fp7575$s0c$2@ger.gmane.org>

On Sat, 16 Feb 2008 17:51:08 +0100, Christian Heimes wrote:

> In the last months Amaury, Martin von L?wis and I have done most of the
> Windows work. Any help from an experienced Windows developer is greatly
> appreciated! We *need* an additional Windows developer.

I would really like to help you out on these wide-API tasks, but it is 
unlikely I will ever have much time in the next weeks (till May, let's 
say). Is there a todo list, a wiki, a keyword to use when searching for 
bugs, etc.? Anything to see what still needs to be done, who's doing it, 
etc.

BTW: is there a long-time plan to make the Python core *not* link against 
msvcrt dll anymore but only rely on Windows APIs (or maybe also the 
static C runtime, I don't really care)?
 
> So if you are interested in getting the problems fixed I welcome you on
> board. I suggest that you start by writing down a list of required
> changes (file names, function names) and a proposal how to pursue the
> issues.

OK. I'll have a look at issue 2128 during next week and come back with 
something. 
-- 
Giovanni Bajo


From pje at telecommunity.com  Sat Feb 16 18:40:52 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 16 Feb 2008 12:40:52 -0500
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp7575$s0c$2@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
Message-ID: <20080216174039.826583A4075@sparrow.telecommunity.com>

At 05:06 PM 2/16/2008 +0000, Giovanni Bajo wrote:
>BTW: is there a long-time plan to make the Python core *not* link 
>against msvcrt dll anymore but only rely on Windows APIs (or maybe 
>also the static C runtime, I don't really care)?

How would this affect using MinGW to build Python extensions?


From rasky at develer.com  Sat Feb 16 18:55:00 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 16 Feb 2008 17:55:00 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
Message-ID: <fp781k$s0c$3@ger.gmane.org>

On Sat, 16 Feb 2008 12:40:52 -0500, Phillip J. Eby wrote:

>>BTW: is there a long-time plan to make the Python core *not* link
>>against msvcrt dll anymore but only rely on Windows APIs (or maybe also
>>the static C runtime, I don't really care)?
> 
> How would this affect using MinGW to build Python extensions?

AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus 
being compatible with any extension, compiled with any compiler (with any 
version of msvcrt, or without msvcrt at all).

Putting an end to this annoying "what version of Visual Studio / msvcrt 
do I need for this version of Python?" is a sensible goal, IMHO.
-- 
Giovanni Bajo


From p.f.moore at gmail.com  Sat Feb 16 19:29:07 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 16 Feb 2008 18:29:07 +0000
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp781k$s0c$3@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
Message-ID: <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>

On 16/02/2008, Giovanni Bajo <rasky at develer.com> wrote:
> On Sat, 16 Feb 2008 12:40:52 -0500, Phillip J. Eby wrote:
>
> >>BTW: is there a long-time plan to make the Python core *not* link
> >>against msvcrt dll anymore but only rely on Windows APIs (or maybe also
> >>the static C runtime, I don't really care)?
> >
> > How would this affect using MinGW to build Python extensions?
>
> AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus
> being compatible with any extension, compiled with any compiler (with any
> version of msvcrt, or without msvcrt at all).
>
> Putting an end to this annoying "what version of Visual Studio / msvcrt
> do I need for this version of Python?" is a sensible goal, IMHO.

Actually, it would mean that there was no way of building extensions,
as Python would be using a static C runtime, which extensions wouldn't
have access to (so a FILE* from an extension would be totally
incompatible with the core CRT, a memory block malloc'd in an
extension couldn't be freed by the core, etc...)

In practice, it's possible to take a more relaxed view (differing CRTs
can be used, but only as long as nothing CRT-dependent gets passed
across the boundaries) but it's not completely clear what is affected,
and so the official policy is strict.

If you avoided the static CRT, and only used Windows APIs, that would
avoid this issue, but that means reimplementing everything -
malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone
is contemplating that.

Paul.

From p.f.moore at gmail.com  Sat Feb 16 19:37:45 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 16 Feb 2008 18:37:45 +0000
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <47B7147C.10506@cheimes.de>
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
Message-ID: <79990c6b0802161037n3f2f8462vaff0b7484ab4afdb@mail.gmail.com>

On 16/02/2008, Christian Heimes <lists at cheimes.de> wrote:
> In the last months Amaury, Martin von L?wis and I have done most of the
> Windows work. Any help from an experienced Windows developer is greatly
> appreciated! We *need* an additional Windows developer.

Is there a list anywhere of tasks that need a Windows C developer? I
may be able to have a go at picking a few up, but unfortunately, I
don't have the time to find what needs to be done.

If you point me at something, I'll see what I can pick up.

Paul.

From rasky at develer.com  Sat Feb 16 20:08:54 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 16 Feb 2008 19:08:54 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
Message-ID: <fp7cc6$s0c$4@ger.gmane.org>

On Sat, 16 Feb 2008 18:29:07 +0000, Paul Moore wrote:

>> >>BTW: is there a long-time plan to make the Python core *not* link
>> >>against msvcrt dll anymore but only rely on Windows APIs (or maybe
>> >>also the static C runtime, I don't really care)?
>> >
>> > How would this affect using MinGW to build Python extensions?
>>
>> AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus
>> being compatible with any extension, compiled with any compiler (with
>> any version of msvcrt, or without msvcrt at all).
>>
>> Putting an end to this annoying "what version of Visual Studio / msvcrt
>> do I need for this version of Python?" is a sensible goal, IMHO.
> 
> Actually, it would mean that there was no way of building extensions, as
> Python would be using a static C runtime, which extensions wouldn't have
> access to (so a FILE* from an extension would be totally incompatible
> with the core CRT, a memory block malloc'd in an extension couldn't be
> freed by the core, etc...)

Yes. In fact, what I really meant was to get rid of any C-ism from the 
Python API, so that these problems you are listing simply do not arise.

For instance, memory allocation is not a problem as far as I can see, as 
extensions always use core calls to allocate objects (and those objects 
get freed through core calls too).

But yes, one would have to get rid of the APIs with FILE* and a few 
others (I wonder if anybody has already made a list). It does not seem 
like a terrible big deal to me in practice, but I don't know how hard it 
would be implementation-wise.

Consider for instance that it was even proposed to rewrite fileobject.c 
to use OS native file support (eg: CreateFile on Windows), as the 
theoretically-portable C API has too many quirks in small details which 
makes it hard to really provide a Python fileobject consistent across 
operating systems. In such a scenario, an API like PyFile_FromFile() does 
not make much sense (but a PyFile_FromDescriptor() might still be useful 
of course).
-- 
Giovanni Bajo


From martin at v.loewis.de  Sat Feb 16 20:58:51 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 16 Feb 2008 20:58:51 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp7575$s0c$2@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	<fp737b$s0c$1@ger.gmane.org>
	<47B7147C.10506@cheimes.de> <fp7575$s0c$2@ger.gmane.org>
Message-ID: <47B7407B.2080006@v.loewis.de>

> BTW: is there a long-time plan to make the Python core *not* link against 
> msvcrt dll anymore but only rely on Windows APIs (or maybe also the 
> static C runtime, I don't really care)?

I have plans to reduce usage of the CRT as much as possible. I don't
believe that linking statically would actually work very well.

Regards,
Martin

From martin at v.loewis.de  Sat Feb 16 21:01:10 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 16 Feb 2008 21:01:10 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	<fp737b$s0c$1@ger.gmane.org>
	<47B7147C.10506@cheimes.de>	<fp7575$s0c$2@ger.gmane.org>	<20080216174039.826583A4075@sparrow.telecommunity.com>	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
Message-ID: <47B74106.1060102@v.loewis.de>

> If you avoided the static CRT, and only used Windows APIs, that would
> avoid this issue, but that means reimplementing everything -
> malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone
> is contemplating that.

Actually, I am. Py3k will already remove usage of stdio, for file input
and output. I would also like to remove CRT file handles, and replace
them with platform file handles. As for malloc and free, using the Heap
API might be an option.

Regards,
Martin

From p.f.moore at gmail.com  Sat Feb 16 21:20:28 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 16 Feb 2008 20:20:28 +0000
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <47B74106.1060102@v.loewis.de>
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
	<47B74106.1060102@v.loewis.de>
Message-ID: <79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com>

On 16/02/2008, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > If you avoided the static CRT, and only used Windows APIs, that would
> > avoid this issue, but that means reimplementing everything -
> > malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone
> > is contemplating that.
>
> Actually, I am.

I stand corrected :-) If you need any help, I'd be willing to assist.

Paul.

From martin at v.loewis.de  Sat Feb 16 21:45:39 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 16 Feb 2008 21:45:39 +0100
Subject: [Python-3000] malloc on Windows (Was: Windows,
	sys.argv and unicode)
In-Reply-To: <79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com>
References: <fp6uvs$p9$1@ger.gmane.org>	
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>	
	<fp7575$s0c$2@ger.gmane.org>	
	<20080216174039.826583A4075@sparrow.telecommunity.com>	
	<fp781k$s0c$3@ger.gmane.org>	
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>	
	<47B74106.1060102@v.loewis.de>
	<79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com>
Message-ID: <47B74B73.3010300@v.loewis.de>

>>> If you avoided the static CRT, and only used Windows APIs, that would
>>> avoid this issue, but that means reimplementing everything -
>>> malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone
>>> is contemplating that.
>> Actually, I am.
> 
> I stand corrected :-) If you need any help, I'd be willing to assist.

It's a long-term project. If you are curious, I would recommend you
look into memory management. At a minimum, pymalloc should use 
VirtualAlloc instead of malloc - as that can guarantee page alignment
out of the box (at the same time, mmap should be used on Unix).

For variable-sized blocks, you could try HeapAlloc, probably on
the process heap. My only concern would be whether HeapAlloc can
compete with the CRT's malloc performance-wise. However, during the
implementation, additional issues might come up.

Regards,
Martin

From nick at craig-wood.com  Sun Feb 17 10:29:56 2008
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Sun, 17 Feb 2008 09:29:56 +0000
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
	<ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
Message-ID: <20080217092956.GA17269@craig-wood.com>

On Fri, Feb 15, 2008 at 01:20:03PM -0800, Guido van Rossum wrote:
> But maybe given how rare the use case is, it would be easier to just
> create a custom class in weakref.py that does what Nick requested.

I still don't like the silent failure, but a noisy failure and a
custom class would be great.

FYI here is the class I was writing which tripped me up.  I created
this to untangle some spaghetti code in a reasonably complex GUI
application I'm writing.

Instead of there being lots of hardcoded callbacks in the GUI objects
I made a central registry which made the code a lot cleaner.  The
weakrefs avoided registration and deregistration.

Since this is used from GUI code all the callback functions I needed
to add are bound methods.

(The last time I got tripped by this was a similar case involving 3rd
parties registering interests in other classes changing.)

------------------------------------------------------------

class Callback(object):
    """
    A holder for functions which need to be called back at some specific point

    Silently fails if you pass in bound methods!
    """
    def __init__(self):
        self.fns = WeakKeyDictionary()
    def add(self, fn):
        """Add function to be called later."""
        self.fns[fn] = True
    def call(self):
        """Call all the registered functions"""
        for fn in self.fns.keys():
            fn()

------------------------------------------------------------

Here is my explicit (no introspection) work-around - there are
obviously better solutions!

class Callback(object):
    """
    A holder for functions which need to be called back at some specific point
    """
    def __init__(self):
        self.fns = WeakKeyDictionary()
    def add(self, fn, attribute=None):
        """
        Add function to be called later.  Either pass in a function,
        or an object and an attribute to call.  Don't pass in bound
        methods since you can't take a weakref of a bound method. It
        holds weakrefs so doesn't need to be deregistered.
        """
        self.fns[fn] = attribute
    def call(self):
        """Call all the registered functions"""
        for fn, attribute in self.fns.iteritems():
            if attribute is not None:
                fn = getattr(fn, attribute)
            fn()


-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From nick at craig-wood.com  Sun Feb 17 10:33:18 2008
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Sun, 17 Feb 2008 09:33:18 +0000
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <47B62FD7.5060409@canterbury.ac.nz>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
	<ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
	<47B62FD7.5060409@canterbury.ac.nz>
Message-ID: <20080217093318.0A59A14C62F@irishsea.home.craig-wood.com>

Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>  Guido van Rossum wrote:
> > But maybe given how rare the use case is, it would be easier to just
> > create a custom class in weakref.py that does what Nick requested.
> 
>  Nick would still forget to use it and get strange bugs
>  in his code, though!

More than likely ;-)  Though an exception if I tried to add a bound
method to a weak dictionary would certainly remind me!

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From rasky at develer.com  Sun Feb 17 12:24:23 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 11:24:23 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
	<47B74106.1060102@v.loewis.de>
Message-ID: <fp95h7$qm7$1@ger.gmane.org>

On Sat, 16 Feb 2008 21:01:10 +0100, Martin v. L?wis wrote:

>> If you avoided the static CRT, and only used Windows APIs, that would
>> avoid this issue, but that means reimplementing everything -
>> malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone
>> is contemplating that.
> 
> Actually, I am. Py3k will already remove usage of stdio, for file input
> and output. 

Are you doing this work on some public branch?
-- 
Giovanni Bajo


From rasky at develer.com  Sun Feb 17 12:27:55 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 11:27:55 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org> <47B7407B.2080006@v.loewis.de>
Message-ID: <fp95nr$qm7$2@ger.gmane.org>

On Sat, 16 Feb 2008 20:58:51 +0100, Martin v. L?wis wrote:

>> BTW: is there a long-time plan to make the Python core *not* link
>> against msvcrt dll anymore but only rely on Windows APIs (or maybe also
>> the static C runtime, I don't really care)?
> 
> I have plans to reduce usage of the CRT as much as possible. I don't
> believe that linking statically would actually work very well.

Given a reduced usage of CRT, why statically linking should not work?

For instance, if all else fail wrt memory allocation, I can't see why it 
is a problem using the statically-linked CRT's malloc (or another multi-
platform malloc implementation like Google's tcmalloc -- which would of 
course be statically linked as well).
-- 
Giovanni Bajo


From rasky at develer.com  Sun Feb 17 14:09:40 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 13:09:40 +0000 (UTC)
Subject: [Python-3000] Windows: debug mode
Message-ID: <fp9bmk$9lb$1@ger.gmane.org>

Hi,

an issue related to the CRT removal/reduced usage under Windows is that 
of debug mode, which I stand that it is mostly useless as it is currently 
implemented.

Python's debug mode under Windows mandates an ABI change: the debug 
version of dynamic libraries (_d) are linked against the CRT debug 
runtime, which is not ABI-compatible to the release runtime. So you 
either have the _d version of all the extensions you need or you can't 
even start your program in debug mode. I believe this started back in the 
days where Python programs were small, had only a couple of dependencies, 
and most developers were also core developers (core developers are 
probably the only ones that can succesfully use the current debug mode).

Thus, what happens is that if you get a segfault in an extension library 
(eg: one you're developing) within a large application, you are forced to 
*recompile all your dependencies* to debug mode before being able to 
debug your extension. This is long and tiresome: you might be using large 
and complex extensions (like numpy or PyOpenGL) which are really hard to 
compile and requires many C-level dependecies and complicated setup; in 
fact, it is likely that you have always used them through their binary 
release-only distributions, and now, simply to get a small stacktrace of 
a crash in your extension, you are forced to setup the whole development 
environment needed to recompile all those extensions!

So, what does a programmer do instead, today? It simply hacks his own 
distutils script to compile his own extension in debug mode, but 
*without* relying on Python's official debug mode support, that is 
*without* using CRT's debug libraries; for example, adding /Zi and /DEBUG 
switches to Visual Studio compiler and linker is sufficient to get a 
*fully debuggable* extension that fully cooperates with the standard 
Python runtime and the standard binary extensions. So, the programmer 
totally ditches the official debug support (eg: --debug in distutils), 
and has to come up with something himself. 

Thus, I'm hereby proposing that we change this in Py3k so that Python's 
debug mode does not mandate any ABI change anymore. This means that a 
programmer will be able to mixmatch release-mode extensions and compile-
mode extensions, thus getting debuggability only of those extensions he 
really cares. He will be able to compile in debug mode only his extension 
by simply doing "setup.py build --debug" and get useful stacktraces and 
debuggability (step-by-step, whatever) through it, without even needed a 
debug build of Python.

If people agree on this goal, I'm willing to actually implement it in the 
Python core (there must be a change in the logic through which "_d" 
extensions are picked up since they would become optional rather than 
required) and everywhere else it might be needed (eg: distutils).
-- 
Giovanni Bajo


From lists at cheimes.de  Sun Feb 17 15:23:17 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 17 Feb 2008 15:23:17 +0100
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <fp9bmk$9lb$1@ger.gmane.org>
References: <fp9bmk$9lb$1@ger.gmane.org>
Message-ID: <fp9g0m$oc3$1@ger.gmane.org>

Giovanni Bajo wrote:
> Python's debug mode under Windows mandates an ABI change: the debug 
> version of dynamic libraries (_d) are linked against the CRT debug 
> runtime, which is not ABI-compatible to the release runtime. So you 
> either have the _d version of all the extensions you need or you can't 
> even start your program in debug mode. I believe this started back in the 
> days where Python programs were small, had only a couple of dependencies, 
> and most developers were also core developers (core developers are 
> probably the only ones that can succesfully use the current debug mode).

[snip]

Python's Py_DEBUG mode adds lots of extra checks which require an ABI
change. For example in debug mode memory allocation and and reference
counting do extra checks etc. In release mode those extra checks are
either disabled or they are replaced by much simpler and faster macros.

It *may* be possible to add the extra functions to the Python library so
that a debug build can also load release libraries. It's a complex and
time consuming task.If you want to invest your time I'm fine with
enhancing the situation for Windows developers. But keep in mind that
any change must neither slow down Python in release mode nor make the
debug mode harder to use for a core developer! Windows issue must not
place an extra burden on Unix users.

Christian


From pje at telecommunity.com  Sun Feb 17 16:10:59 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 17 Feb 2008 10:10:59 -0500
Subject: [Python-3000] weakrefs of bound methods
In-Reply-To: <20080217092956.GA17269@craig-wood.com>
References: <20080215101219.GA29219@craig-wood.com>
	<ca471dc20802150935g3173f54qe86fe8feda2197ba@mail.gmail.com>
	<20080215182609.0251A3A4075@sparrow.telecommunity.com>
	<ca471dc20802151141o1f6e090cl888ae560976643cb@mail.gmail.com>
	<20080215205231.93DD43A4075@sparrow.telecommunity.com>
	<ca471dc20802151320l7c3074i2ae72178e5a3fe66@mail.gmail.com>
	<20080217092956.GA17269@craig-wood.com>
Message-ID: <20080217151045.C3CC93A40A5@sparrow.telecommunity.com>

At 09:29 AM 2/17/2008 +0000, Nick Craig-Wood wrote:
>On Fri, Feb 15, 2008 at 01:20:03PM -0800, Guido van Rossum wrote:
> > But maybe given how rare the use case is, it would be easier to just
> > create a custom class in weakref.py that does what Nick requested.
>
>I still don't like the silent failure, but a noisy failure and a
>custom class would be great.
>
>FYI here is the class I was writing which tripped me up.  I created
>this to untangle some spaghetti code in a reasonably complex GUI
>application I'm writing.
>
>Instead of there being lots of hardcoded callbacks in the GUI objects
>I made a central registry which made the code a lot cleaner.  The
>weakrefs avoided registration and deregistration.

You might want to check this out:

http://pypi.python.org/pypi/PyDispatcher

"""Included in the package are the robustapply and saferef modules, 
which provide the ability to selectively apply arguments to callable 
objects and to reference instance methods using weak-references."""


From pje at telecommunity.com  Sun Feb 17 16:15:22 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 17 Feb 2008 10:15:22 -0500
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <fp9g0m$oc3$1@ger.gmane.org>
References: <fp9bmk$9lb$1@ger.gmane.org>
 <fp9g0m$oc3$1@ger.gmane.org>
Message-ID: <20080217151509.1C7A33A40A5@sparrow.telecommunity.com>

At 03:23 PM 2/17/2008 +0100, Christian Heimes wrote:
>Giovanni Bajo wrote:
> > Python's debug mode under Windows mandates an ABI change: the debug
> > version of dynamic libraries (_d) are linked against the CRT debug
> > runtime, which is not ABI-compatible to the release runtime. So you
> > either have the _d version of all the extensions you need or you can't
> > even start your program in debug mode. I believe this started back in the
> > days where Python programs were small, had only a couple of dependencies,
> > and most developers were also core developers (core developers are
> > probably the only ones that can succesfully use the current debug mode).
>
>[snip]
>
>Python's Py_DEBUG mode adds lots of extra checks which require an ABI
>change. For example in debug mode memory allocation and and reference
>counting do extra checks etc. In release mode those extra checks are
>either disabled or they are replaced by much simpler and faster macros.
>
>It *may* be possible to add the extra functions to the Python library so
>that a debug build can also load release libraries.

Actually, I think Giovanni simply wants to be able to build extension 
modules with debugging information, *without* needing to use a Py_DEBUG build.

That is, this is about using debug extensions with a release Python, 
not the other way 'round.  And if I understand correctly, this could 
be done by additions to the distutils.


From martin at v.loewis.de  Sun Feb 17 18:09:18 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 17 Feb 2008 18:09:18 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp95h7$qm7$1@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	<fp737b$s0c$1@ger.gmane.org>
	<47B7147C.10506@cheimes.de>	<fp7575$s0c$2@ger.gmane.org>	<20080216174039.826583A4075@sparrow.telecommunity.com>	<fp781k$s0c$3@ger.gmane.org>	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>	<47B74106.1060102@v.loewis.de>
	<fp95h7$qm7$1@ger.gmane.org>
Message-ID: <47B86A3E.2090503@v.loewis.de>

> Are you doing this work on some public branch?

Yes, on the Py3k branch, and the trunk.

Regards,
Martin

From lists at cheimes.de  Sun Feb 17 18:09:51 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 17 Feb 2008 18:09:51 +0100
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <20080217151509.1C7A33A40A5@sparrow.telecommunity.com>
References: <fp9bmk$9lb$1@ger.gmane.org> <fp9g0m$oc3$1@ger.gmane.org>
	<20080217151509.1C7A33A40A5@sparrow.telecommunity.com>
Message-ID: <fp9ppd$lc7$1@ger.gmane.org>

Phillip J. Eby wrote:
> Actually, I think Giovanni simply wants to be able to build extension 
> modules with debugging information, *without* needing to use a Py_DEBUG build.
> 
> That is, this is about using debug extensions with a release Python, 
> not the other way 'round.  And if I understand correctly, this could 
> be done by additions to the distutils.

Ah, I see. It's about building an extension with debug symbols but
without a Py_DEBUG build. That's possible with some minor adjustments to
PC/pyconfig.h and PCbuild/debug.vsprops. One has to decouple _DEBUG from
Py_DEBUG. Currently _DEBUG implies Py_DEBUG.

A debug build without Py_DEBUG is semi useful. Bugs and memory leaks are
usually caused by reference counting issues.

Christian


From martin at v.loewis.de  Sun Feb 17 18:13:32 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 17 Feb 2008 18:13:32 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fp95nr$qm7$2@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	<fp737b$s0c$1@ger.gmane.org>
	<47B7147C.10506@cheimes.de>	<fp7575$s0c$2@ger.gmane.org>
	<47B7407B.2080006@v.loewis.de> <fp95nr$qm7$2@ger.gmane.org>
Message-ID: <47B86B3C.3080400@v.loewis.de>

>> I have plans to reduce usage of the CRT as much as possible. I don't
>> believe that linking statically would actually work very well.
> 
> Given a reduced usage of CRT, why statically linking should not work?

When this project is complete (say, in 2015), there won't be a need
to link statically. Until then, various things might fail. For example,
setting the locale through setlocale might not have the desired effect
on a different module.

In addition, there is the size increase - you would easily get multiple
copies of stdio in the distribution. It's fairly tricky not to link
stdio in static linking, even if you don't call any stdio functions.

Regards,
Martin

From martin at v.loewis.de  Sun Feb 17 18:19:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Feb 2008 18:19:30 +0100
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <fp9bmk$9lb$1@ger.gmane.org>
References: <fp9bmk$9lb$1@ger.gmane.org>
Message-ID: <47B86CA2.7090908@v.loewis.de>

> Thus, I'm hereby proposing that we change this in Py3k so that Python's 
> debug mode does not mandate any ABI change anymore.

It seems people still debate what it is that you *actually* want. This
sentence suggests Christian's interpretation, i.e. you might be unaware
that the debug mode not only adds debug symbols, but also changes the
layout of objects. That is *very* useful, and should not be dropped.

> This means that a 
> programmer will be able to mixmatch release-mode extensions and compile-
> mode extensions, thus getting debuggability only of those extensions he 
> really cares.

Whereas this suggests Phillip's interpretation, i.e. you merely want
debug symbols on a single DLL.

You can get that today - just enable debug information for your 
extension module, and the compiler will emit the PDB files you need
for debugging.

> If people agree on this goal, I'm willing to actually implement it in the 
> Python core (there must be a change in the logic through which "_d" 
> extensions are picked up since they would become optional rather than 
> required) and everywhere else it might be needed (eg: distutils).

See above. Don't drop what Python calls the debug mode. Instead, just
learn to build your extension with debugging, for Python's release mode.

Regards,
Martin

From arnodel at googlemail.com  Sun Feb 17 18:45:48 2008
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Sun, 17 Feb 2008 09:45:48 -0800 (PST)
Subject: [Python-3000] PEP 3102
In-Reply-To: <ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> 
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
Message-ID: <f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>



On Feb 14, 11:13?pm, "Guido van Rossum" <gu... at python.org> wrote:
> On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger <pyt... at rcn.com> wrote:
> > I've been exercising the new keyword-only arguments syntax. ?It is absolutely wonderful. ?I'm amazed at how many long standing problems it solves elegantly.
>
> Agreed. Now can you come up with a syntax for positional-only
> arguments? So far everybody has failed at that, and there are some use
> cases where it's useful too.

The idea in PEP 3102 is that *args gobbles up all remaining positional
arguments, hence what comes after can only be interpreted as
keyword-only arguments. This is extended to allowing * on its own.

e.g. f(*, z, t=2)
     => z, t are keyword-only

Following the same train of thought, **kwargs gobbles up all remaining
keyword arguments, hence what comes after can only interpreted as
positional-only arguments.  Then one could extend it to allowing ** on
its own.

e.g. f(**, x, y=1)
     => x, y are positional-only

Now, how do keyword-only arguments fit in?  Say x, y=1 are
positional-only and z, t=2 are keyword-only.  One could have:

f(**, x, y=1, *, z, t=2)

The general cases could look like

f(x, y=1, *args, z, t=2, **kwargs):
     => x, y are hybrid
     => args: all remaining positional arguments
     => z, t are keyword-only
     => kwargs: all remaining keyword arguments

f(**kwargs, x, y=1, *args, z, t=2):
     => x, y are positional-only
     => args: all remaining positional arguments
     => z, t are keyword-only
     => kwargs: all remaining keyword arguments

'args' and 'kwargs' are optional in both cases. If 'kwargs' is present
in the second version, it is a bit strange as it usually only appears
at the end.  Alternatively, only '**' could be allowed at the start
and if needed '**kwargs' would come at the end:

f(**, x, y=1, *args, z, t=2, **kwargs)

Note that this syntax doesn't allow for positional-only and hybrid
arguments to coexist.  But this is nice IMHO, as it seems to me that
it would only create confusion.

Unconvincingly yours,

--
Arnaud


From rasky at develer.com  Sun Feb 17 22:25:42 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 21:25:42 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
	<47B74106.1060102@v.loewis.de> <fp95h7$qm7$1@ger.gmane.org>
	<47B86A3E.2090503@v.loewis.de>
Message-ID: <fpa8om$46i$1@ger.gmane.org>

On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote:

>> Actually, I am. Py3k will already remove usage of stdio, for file input
>> and output. 

>> Are you doing this work on some public branch?
> 
> Yes, on the Py3k branch, and the trunk.

I looked at fileobject.c in py3k branch today and it looked like it was 
still using FILE* everywhere. I'm not sure I know where to look, though.

BTW, is there a public description about how you are planning to remove 
usage of stdio from fileobject? For instance, will PyFile_FromFile() be 
deprecated/removed?

Generically speaking, all the UNIX world seems to speak char* and UTF-8 
lately, while the Windows world speaks either char* and MBCS or wchar_t 
and UTF-16[*]. How do you plan to make these two world live together?

In case the question is not clear, I elaborate with an example. Today, I 
found out that putting Unicode strings in sys.path breaks zipimporter, 
because it expects a 8-byte string; I guess the whole import machinery 
passes char* everywhere for filenames and directories. How do you plan to 
make all that code Unicode-aware on Windows?

[*] I'm not a Unicode guru, so I'm not 100% sure that the wide APIs in 
Windows formally accepts UTF-16, but at least it looks like UTF-16 to 
me :) It's not really relevant to the question, I reckon.
-- 
Giovanni Bajo


From lists at cheimes.de  Sun Feb 17 22:31:56 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 17 Feb 2008 22:31:56 +0100
Subject: [Python-3000] Windows, sys.argv and unicode
In-Reply-To: <fpa8om$46i$1@ger.gmane.org>
References: <fp6uvs$p9$1@ger.gmane.org>	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>	<fp737b$s0c$1@ger.gmane.org>
	<47B7147C.10506@cheimes.de>	<fp7575$s0c$2@ger.gmane.org>	<20080216174039.826583A4075@sparrow.telecommunity.com>	<fp781k$s0c$3@ger.gmane.org>	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>	<47B74106.1060102@v.loewis.de>
	<fp95h7$qm7$1@ger.gmane.org>	<47B86A3E.2090503@v.loewis.de>
	<fpa8om$46i$1@ger.gmane.org>
Message-ID: <fpa94c$5sg$1@ger.gmane.org>

Giovanni Bajo wrote:
> On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote:
> 
>>> Actually, I am. Py3k will already remove usage of stdio, for file input
>>> and output. 
> 
>>> Are you doing this work on some public branch?
>> Yes, on the Py3k branch, and the trunk.
> 
> I looked at fileobject.c in py3k branch today and it looked like it was 
> still using FILE* everywhere. I'm not sure I know where to look, though.

Are you looking at the correct sources? Neither fileobject nor _fileio.c
are using FILE except for Py_UniversalNewlineFgets.

Christian


From rasky at develer.com  Sun Feb 17 22:33:46 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 21:33:46 +0000 (UTC)
Subject: [Python-3000] Windows: debug mode
References: <fp9bmk$9lb$1@ger.gmane.org> <47B86CA2.7090908@v.loewis.de>
Message-ID: <fpa97q$46i$2@ger.gmane.org>

On Sun, 17 Feb 2008 18:19:30 +0100, Martin v. L?wis wrote:

>> Thus, I'm hereby proposing that we change this in Py3k so that Python's
>> debug mode does not mandate any ABI change anymore.
> 
> It seems people still debate what it is that you *actually* want. 

Yes, sorry for the confusion. 

> This
> sentence suggests Christian's interpretation, i.e. you might be unaware
> that the debug mode not only adds debug symbols, but also changes the
> layout of objects. That is *very* useful, and should not be dropped.

Yes, I know it changes the ABI, and I know it must say as it is. I 
believe that Py_DEBUG is mostly useful to core developers and not much to 
extension developer or regular Python developers, but it's immaterial in 
this discussion.

>> This means that a
>> programmer will be able to mixmatch release-mode extensions and
>> compile- mode extensions, thus getting debuggability only of those
>> extensions he really cares.
> 
> Whereas this suggests Phillip's interpretation, i.e. you merely want
> debug symbols on a single DLL.

Yes I want this.

> You can get that today - just enable debug information for your
> extension module, and the compiler will emit the PDB files you need for
> debugging.

Yes, this is what I do everyday by manually hand-edit my setup.py script. 
Is there a way to make distutils do this for me with a simple command-
line switch?

I think my request can be resumed like this:

1) Add to distutils a way support to compile extensions with debug 
informations, without optimizations, and linked with the non-debug CRT.

2) Add the same compilation mode to Python itself, so that users can 
easily compile a Python DLL which has debug symbols but with the release 
ABI (since stepping into the Python's core can be useful while debugging 
an extension).

I hope it's clear now :)
-- 
Giovanni Bajo


From rasky at develer.com  Sun Feb 17 22:45:38 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 17 Feb 2008 21:45:38 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org> <47B7147C.10506@cheimes.de>
	<fp7575$s0c$2@ger.gmane.org>
	<20080216174039.826583A4075@sparrow.telecommunity.com>
	<fp781k$s0c$3@ger.gmane.org>
	<79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com>
	<47B74106.1060102@v.loewis.de> <fp95h7$qm7$1@ger.gmane.org>
	<47B86A3E.2090503@v.loewis.de> <fpa8om$46i$1@ger.gmane.org>
	<fpa94c$5sg$1@ger.gmane.org>
Message-ID: <fpa9u2$46i$3@ger.gmane.org>

On Sun, 17 Feb 2008 22:31:56 +0100, Christian Heimes wrote:

> Giovanni Bajo wrote:
>> On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote:
>> 
>>>> Actually, I am. Py3k will already remove usage of stdio, for file
>>>> input and output.
>> 
>>>> Are you doing this work on some public branch?
>>> Yes, on the Py3k branch, and the trunk.
>> 
>> I looked at fileobject.c in py3k branch today and it looked like it was
>> still using FILE* everywhere. I'm not sure I know where to look,
>> though.
> 
> Are you looking at the correct sources? Neither fileobject nor _fileio.c
> are using FILE except for Py_UniversalNewlineFgets.

Doh! I don't know what I was looking at... I probably got my directory 
crossed. Sorry for the spam.
-- 
Giovanni Bajo


From lists at cheimes.de  Sun Feb 17 23:11:24 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 17 Feb 2008 23:11:24 +0100
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <fpa97q$46i$2@ger.gmane.org>
References: <fp9bmk$9lb$1@ger.gmane.org> <47B86CA2.7090908@v.loewis.de>
	<fpa97q$46i$2@ger.gmane.org>
Message-ID: <fpabed$dfv$1@ger.gmane.org>

Giovanni Bajo wrote:
> Yes, I know it changes the ABI, and I know it must say as it is. I 
> believe that Py_DEBUG is mostly useful to core developers and not much to 
> extension developer or regular Python developers, but it's immaterial in 
> this discussion.

Py_DEBUG builds are useful to extension developers, too. It's the best
way to debug reference leaks. You'll need at least a Py_REF_DEBUG build
for the task.

> Yes, this is what I do everyday by manually hand-edit my setup.py script. 
> Is there a way to make distutils do this for me with a simple command-
> line switch?

Yes - provide a patch :)

> 2) Add the same compilation mode to Python itself, so that users can 
> easily compile a Python DLL which has debug symbols but with the release 
> ABI (since stepping into the Python's core can be useful while debugging 
> an extension).

The Windows build files were carefully design for the job. I've invested
quite some time to build a project inheritance tree. All projects
inherit settings from pyproject.vsprops. In order to build a release
version of Python with less optimization and debug symbols you've to
alter the global properties for Release|Win32 and recompile the project.

Christian


From martin at v.loewis.de  Mon Feb 18 19:17:22 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 18 Feb 2008 19:17:22 +0100
Subject: [Python-3000] Windows: debug mode
In-Reply-To: <fpabed$dfv$1@ger.gmane.org>
References: <fp9bmk$9lb$1@ger.gmane.org>
	<47B86CA2.7090908@v.loewis.de>	<fpa97q$46i$2@ger.gmane.org>
	<fpabed$dfv$1@ger.gmane.org>
Message-ID: <47B9CBB2.3070105@v.loewis.de>

>> Is there a way to make distutils do this for me with a simple command-
>> line switch?
> 
> Yes - provide a patch :)
> 
>> 2) Add the same compilation mode to Python itself, so that users can 
>> easily compile a Python DLL which has debug symbols but with the release 
>> ABI (since stepping into the Python's core can be useful while debugging 
>> an extension).
> 
> The Windows build files were carefully design for the job. [...]

As he says.

Regards,
Martin

From talin at acm.org  Mon Feb 18 20:39:37 2008
From: talin at acm.org (Talin)
Date: Mon, 18 Feb 2008 11:39:37 -0800
Subject: [Python-3000] PEP 3102
In-Reply-To: <f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>
Message-ID: <47B9DEF9.2070606@acm.org>

Arnaud Delobelle wrote:
> 
> On Feb 14, 11:13 pm, "Guido van Rossum" <gu... at python.org> wrote:
>> On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger <pyt... at rcn.com> wrote:
>>> I've been exercising the new keyword-only arguments syntax.  It is absolutely wonderful.  I'm amazed at how many long standing problems it solves elegantly.
>> Agreed. Now can you come up with a syntax for positional-only
>> arguments? So far everybody has failed at that, and there are some use
>> cases where it's useful too.
> 
> The idea in PEP 3102 is that *args gobbles up all remaining positional
> arguments, hence what comes after can only be interpreted as
> keyword-only arguments. This is extended to allowing * on its own.
> 
> e.g. f(*, z, t=2)
>      => z, t are keyword-only
> 
> Following the same train of thought, **kwargs gobbles up all remaining
> keyword arguments, hence what comes after can only interpreted as
> positional-only arguments.  Then one could extend it to allowing ** on
> its own.
> 
> e.g. f(**, x, y=1)
>      => x, y are positional-only
> 
> Now, how do keyword-only arguments fit in?  Say x, y=1 are
> positional-only and z, t=2 are keyword-only.  One could have:
> 
> f(**, x, y=1, *, z, t=2)
> 
> The general cases could look like
> 
> f(x, y=1, *args, z, t=2, **kwargs):
>      => x, y are hybrid
>      => args: all remaining positional arguments
>      => z, t are keyword-only
>      => kwargs: all remaining keyword arguments
> 
> f(**kwargs, x, y=1, *args, z, t=2):
>      => x, y are positional-only
>      => args: all remaining positional arguments
>      => z, t are keyword-only
>      => kwargs: all remaining keyword arguments
> 
> 'args' and 'kwargs' are optional in both cases. If 'kwargs' is present
> in the second version, it is a bit strange as it usually only appears
> at the end.  Alternatively, only '**' could be allowed at the start
> and if needed '**kwargs' would come at the end:
> 
> f(**, x, y=1, *args, z, t=2, **kwargs)
> 
> Note that this syntax doesn't allow for positional-only and hybrid
> arguments to coexist.  But this is nice IMHO, as it seems to me that
> it would only create confusion.
> 
> Unconvincingly yours,

Well, that is certainly a logical continuation of the train of thought 
behind the 'single *' syntax.

I'd be curious to know which parts of 3102 people are finding most 
useful. You see, the itch that I was originally attempting to scratch 
concerned just the narrow use case of combining a 'varargs' function 
with optional keywords. I've seen/written a lot of code where you have a 
function that takes a list of arguments, with a couple of optional 
keywords that control how that list is handled, and 3102 was intended to 
make that case much easier to handle.

I was never all that interested in 'forcing' some of the arguments to a 
non-varargs function to be specified by keyword. I am perfectly 
comfortable with the idea that the caller always gets to choose whether 
to pass something positionally or via keyword, and limiting the caller's 
choice seems vaguely unpythonic to me. However, I included that part in 
the PEP based on the feedback from here.

In any case, limiting arguments to being specified by keyword-only or 
positional-only was not part of my original 'itch', and I am curious as 
to what are the use cases for it that people are envisioning.

> --
> Arnaud
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org
> 

From arnodel at googlemail.com  Mon Feb 18 21:17:41 2008
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Mon, 18 Feb 2008 20:17:41 +0000
Subject: [Python-3000] PEP 3102
In-Reply-To: <47B9DEF9.2070606@acm.org>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>
	<47B9DEF9.2070606@acm.org>
Message-ID: <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>


On 18 Feb 2008, at 19:39, Talin wrote:

> Arnaud Delobelle wrote:
>>
[...]
>> Unconvincingly yours,
>
> Well, that is certainly a logical continuation of the train of  
> thought behind the 'single *' syntax.

> I'd be curious to know which parts of 3102 people are finding most  
> useful. You see, the itch that I was originally attempting to  
> scratch concerned just the narrow use case of combining a 'varargs'  
> function with optional keywords. I've seen/written a lot of code  
> where you have a function that takes a list of arguments, with a  
> couple of optional keywords that control how that list is handled,  
> and 3102 was intended to make that case much easier to handle.

Another use would be allowing the '_cache trick' with a varargs  
function, i.e.

def f(*args, _cache={}):
	...

Personally I don't like this trick though...

> I was never all that interested in 'forcing' some of the arguments  
> to a non-varargs function to be specified by keyword. I am perfectly  
> comfortable with the idea that the caller always gets to choose  
> whether to pass something positionally or via keyword, and limiting  
> the caller's choice seems vaguely unpythonic to me. However, I  
> included that part in the PEP based on the feedback from here.
>
> In any case, limiting arguments to being specified by keyword-only  
> or positional-only was not part of my original 'itch', and I am  
> curious as to what are the use cases for it that people are  
> envisioning.
>

A typical use of positional-only arguments is with a function

def f(x, y=1, **kwargs):
	...

where keyword arguments are potentially anything at all, including x  
and y.  For example: dict.update().  In fact it is a fairly  
symmetrical itch to yours: without positional only arguments, the  
above must be written something like:

def f(*args, **kwargs):
	nargs = len(args)
	if 1 <= nargs <= 2:
		x = args[0]
		y = args[1] if nargs==2 else 1
	else:
		raise blah
	...

whereas with positional-only arguments (using the syntax I put  
forward) one could write simply:

def f(**kwargs, x, y=1)

Just like

def f(*args, a, b=1):
	...

had to be written pre-PEP 3102 as something like:

def f(*args, **kwargs):
	a = kwargs.pop('a')
	b = kwargs.pop('b', 1)
	if kwargs:
		raise blah	

Both itches are about not having to unpack something: **kwargs in the  
case of PEP 3102, *args in the case of positional-only arguments.


-- 
Arnaud


From kumar.mcmillan at gmail.com  Tue Feb 19 01:32:35 2008
From: kumar.mcmillan at gmail.com (Kumar McMillan)
Date: Mon, 18 Feb 2008 18:32:35 -0600
Subject: [Python-3000] detect file encoding or always use the default, UTF-8?
Message-ID: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>

Hello.

In Python 3, when opening a file without declaring the encoding
keyword is Python going to guess the encoding of the file or just use
the default?  That is, assume the file is UTF-8?

What led me to wonder this is that on Mac OS X 10.4.11, opening a file
containing UTF-8 encoded text creates a unicode object under the
assumption the text was "mac roman" encoding.  This seems like a bug.
It works if I set the encoding to UTF-8 explicitly.  I can post the
code but it sounds like there are some Mac encoding issues in flux so
I thought to ask first.

thanks, Kumar

From rhamph at gmail.com  Tue Feb 19 02:29:39 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 18 Feb 2008 18:29:39 -0700
Subject: [Python-3000] PEP 3102
In-Reply-To: <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>
	<47B9DEF9.2070606@acm.org>
	<6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>
Message-ID: <aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>

On Feb 18, 2008 1:17 PM, Arnaud Delobelle <arnodel at googlemail.com> wrote:
> A typical use of positional-only arguments is with a function
>
> def f(x, y=1, **kwargs):
>         ...
>
> where keyword arguments are potentially anything at all, including x
> and y.  For example: dict.update().  In fact it is a fairly
> symmetrical itch to yours: without positional only arguments, the
> above must be written something like:

I worry that dict is not just an example, it may be the *only*
example.  Does somebody have a use case that doesn't involve dict?

Whereas keyword-only arguments force explicitness (at the cost of
brevity), positional-only arguments do the opposite.  The default
already allows us to choose brevity when appropriate, and the only
reason I see to force it is when there's a conflict with **kwargs.
Such obscure features warrant an obscure yet well-named decorator, not
an obscure syntax that's difficult to get help on.


-- 
Adam Olsen, aka Rhamphoryncus

From ncoghlan at gmail.com  Tue Feb 19 11:49:28 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Feb 2008 20:49:28 +1000
Subject: [Python-3000] PEP 3102
In-Reply-To: <aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>	<47B9DEF9.2070606@acm.org>	<6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>
	<aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>
Message-ID: <47BAB438.9050504@gmail.com>

Adam Olsen wrote:
> Whereas keyword-only arguments force explicitness (at the cost of
> brevity), positional-only arguments do the opposite.  The default
> already allows us to choose brevity when appropriate, and the only
> reason I see to force it is when there's a conflict with **kwargs.

Which for me happens when I have a method which I want to accept 
arbitrary keyword arguments (I'll grant that isn't very often, but it 
has happened). I have to choose between disallowing 'self' or '_self' as 
one of the no-longer-quite-so-arbitrary keyword arguments, or else I 
have to do the song and dance to implement manual unpacking.

> Such obscure features warrant an obscure yet well-named decorator, not
> an obscure syntax that's difficult to get help on.

It's difficult to handle this in a decorator, as the problem that occurs 
is the underlying function reporting duplicate definitions for a parameter.

 >>> def f(self, **kwds):
...     print self, kwds
...
 >>> f(1, self=2)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'self'

There really isn't a lot a decorator can do to prevent this, unless we 
modify function calls to permit direct assignment of the keywords 
dictionary without unpacking it. Then it would become possible to 
provide a decorator in functools along the lines of:

def positional_only(required=1):
     def wrap(f):
         @wraps(f)
         def wrapper(*args, **kwds):
             if len(args) < required:
                 raise TypeError("Need %d positional arguments "
                                 "(%d given)" % (required, len(args)))
             f(*args, **=kwds)

The idea of the **= syntax would be that the supplied dictionary is used 
directly as the arbitrary keywords parameter for the underlying 
function. Other named arguments would *not* be extracted from that 
dictionary (obviously, the above decorator would need further work in 
order to correctly handle parameters which can passed as either 
positional or keywords parameters).

Personally, I don't think the problem comes up often enough to be overly 
worried about it, as much as my sense of aesthetics would like to see an 
elegant solution (as judged by own personal definition of elegance of 
course ;)

We've also veered fairly far off topic for the Py3k list - further ideas 
for positional-only argument syntax or decorators should probably be 
kicked around on python-ideas rather than here or python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Tue Feb 19 19:33:03 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 19 Feb 2008 10:33:03 -0800
Subject: [Python-3000] detect file encoding or always use the default,
	UTF-8?
In-Reply-To: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>
References: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>
Message-ID: <ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>

Well, we're basically hoping that the folks who actually uses Python
to read and write text files containing non-ASCII characters on OSX
tell us what they want. At least that's where I am. Since I personally
still live in a nearly-ASCII world (and probably always will), my own
experience just doesn't give me any guidance as to what would be the
most useful.

How does Apple's TextEdit do the guessing (assuming it guesses at all)?

I just typed a few non-ASCII characters into a text file with TextEdit
on OSX 10.5, and it seems to have written the file in MacRoman. This
with the preferences for open and save set to automatic.

--Guido

On Feb 18, 2008 4:32 PM, Kumar McMillan <kumar.mcmillan at gmail.com> wrote:
> Hello.
>
> In Python 3, when opening a file without declaring the encoding
> keyword is Python going to guess the encoding of the file or just use
> the default?  That is, assume the file is UTF-8?
>
> What led me to wonder this is that on Mac OS X 10.4.11, opening a file
> containing UTF-8 encoded text creates a unicode object under the
> assumption the text was "mac roman" encoding.  This seems like a bug.
> It works if I set the encoding to UTF-8 explicitly.  I can post the
> code but it sounds like there are some Mac encoding issues in flux so
> I thought to ask first.
>
> thanks, Kumar
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From arnodel at googlemail.com  Tue Feb 19 19:45:06 2008
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Tue, 19 Feb 2008 18:45:06 +0000
Subject: [Python-3000] PEP 3102
In-Reply-To: <47BAB438.9050504@gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>	<47B9DEF9.2070606@acm.org>	<6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>
	<aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>
	<47BAB438.9050504@gmail.com>
Message-ID: <2C0C17DD-7DD1-4563-BB6E-9F3DC9491EF6@gmail.com>


On 19 Feb 2008, at 10:49, Nick Coghlan wrote:
[...]
>> Such obscure features warrant an obscure yet well-named decorator,  
>> not
>> an obscure syntax that's difficult to get help on.
>
> It's difficult to handle this in a decorator, as the problem that  
> occurs is the underlying function reporting duplicate definitions  
> for a parameter.
>
> >>> def f(self, **kwds):
> ...     print self, kwds
> ...
> >>> f(1, self=2)
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: f() got multiple values for keyword argument 'self'
>
> There really isn't a lot a decorator can do to prevent this, unless  
> we modify function calls to permit direct assignment of the keywords  
> dictionary without unpacking it. Then it would become possible to  
> provide a decorator in functools along the lines of:
>
> def positional_only(required=1):
>    def wrap(f):
>        @wraps(f)
>        def wrapper(*args, **kwds):
>            if len(args) < required:
>                raise TypeError("Need %d positional arguments "
>                                "(%d given)" % (required, len(args)))
>            f(*args, **=kwds)
>

[...]

> We've also veered fairly far off topic for the Py3k list - further  
> ideas for positional-only argument syntax or decorators should  
> probably be kicked around on python-ideas rather than here or python- 
> dev.

For reference, here was a long thread on this on python-ideas [1].   
Several syntax options were discussed, I proposed using a decorator  
and several ways of achieving this were posted:

* by Steven Bethard [2]
* by me [3], [4]

I posted a cookbook recipe based on [4].

[1] http://mail.python.org/pipermail/python-ideas/2007-May/000704.html
[2] http://mail.python.org/pipermail/python-ideas/2007-May/000763.html
[3] http://mail.python.org/pipermail/python-ideas/2007-May/000808.html
[4] http://mail.python.org/pipermail/python-ideas/2007-May/000860.html

-- 
Arnaud


From kumar.mcmillan at gmail.com  Tue Feb 19 20:35:38 2008
From: kumar.mcmillan at gmail.com (Kumar McMillan)
Date: Tue, 19 Feb 2008 13:35:38 -0600
Subject: [Python-3000] detect file encoding or always use the default,
	UTF-8?
In-Reply-To: <ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>
References: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>
	<ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>
Message-ID: <b571660a0802191135v5656b747ve058879072da2a9e@mail.gmail.com>

Hi.
I was mainly confused about the auto-detection of the encoding.  I
thought I read something earlier that Python 3 will not try to guess
encodings and that opening files will always use the sys encoding
(defaulting to UTF-8) instead of guessing.  Can't find where I read
that so I might have made it up :D  Now at least I know it will be
guessing encodings.

On Feb 19, 2008 12:33 PM, Guido van Rossum <guido at python.org> wrote:
> Well, we're basically hoping that the folks who actually uses Python
> to read and write text files containing non-ASCII characters on OSX
> tell us what they want. At least that's where I am. Since I personally
> still live in a nearly-ASCII world (and probably always will), my own
> experience just doesn't give me any guidance as to what would be the
> most useful.

I too pretty much work entirely in ascii for day to day stuff so it's
hard to say.

>
> How does Apple's TextEdit do the guessing (assuming it guesses at all)?
>
> I just typed a few non-ASCII characters into a text file with TextEdit
> on OSX 10.5, and it seems to have written the file in MacRoman. This
> with the preferences for open and save set to automatic.

Were you saving in RTF format or plain text?  TextEdit is strange in
that it makes it hard to save a plain text file so if you were working
with RTF files I'd think you'd have to open it from Python in binary
mode anyway, no?

But for plain text TextEdit doesn't guess.  I typed some non-ascii and
went to format -> Make plain text.  Then in the save dialog it
defaulted to UTF-16 and the only options (without making custom
modifications) were UTF-8 and simplified Chinese.  However, this was
on 10.4; someone checked 10.5 for me and said the default is UTF-8
(sounds like they fixed something!).


thanks, Kumar

From hashcollision at gmail.com  Tue Feb 19 20:54:40 2008
From: hashcollision at gmail.com (hashcollision)
Date: Tue, 19 Feb 2008 14:54:40 -0500
Subject: [Python-3000] Library Change Request: change
	functools.partial.keywords to kwargs
Message-ID: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>

I have a library change request: currently partial Objects created by
functools.partial has the "keywords" attribute. However, the standard
name for something like it is kwargs. I request that it be renamed to
"kwargs".

http://docs.python.org/lib/partial-objects.html

From brett at python.org  Tue Feb 19 21:03:31 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 19 Feb 2008 12:03:31 -0800
Subject: [Python-3000] Library Change Request: change
	functools.partial.keywords to kwargs
In-Reply-To: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>
References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>
Message-ID: <bbaeab100802191203rf305d8v1213ee193ffbe56f@mail.gmail.com>

On Feb 19, 2008 11:54 AM, hashcollision <hashcollision at gmail.com> wrote:
> I have a library change request: currently partial Objects created by
> functools.partial has the "keywords" attribute. However, the standard
> name for something like it is kwargs. I request that it be renamed to
> "kwargs".

-1

The change has minor benefit while breaking code in the process. If
you want this to happen it is going to take at least a full release
cycle of having a deprecation warning on the old attribute name and I
still don't see enough benefit to bother.

-Brett

From collinw at gmail.com  Tue Feb 19 21:03:40 2008
From: collinw at gmail.com (Collin Winter)
Date: Tue, 19 Feb 2008 12:03:40 -0800
Subject: [Python-3000] Library Change Request: change
	functools.partial.keywords to kwargs
In-Reply-To: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>
References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>
Message-ID: <43aa6ff70802191203m71594d56gb15f2806d5a0ad22@mail.gmail.com>

On Feb 19, 2008 11:54 AM, hashcollision <hashcollision at gmail.com> wrote:
> I have a library change request: currently partial Objects created by
> functools.partial has the "keywords" attribute. However, the standard
> name for something like it is kwargs. I request that it be renamed to
> "kwargs".

Is it conflicting with something you're doing or this just a personal
aesthetic preference?

Collin Winter

From ncoghlan at gmail.com  Wed Feb 20 00:14:56 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Feb 2008 09:14:56 +1000
Subject: [Python-3000] Library Change Request:
 change	functools.partial.keywords to kwargs
In-Reply-To: <bbaeab100802191203rf305d8v1213ee193ffbe56f@mail.gmail.com>
References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com>
	<bbaeab100802191203rf305d8v1213ee193ffbe56f@mail.gmail.com>
Message-ID: <47BB62F0.5050202@gmail.com>

Brett Cannon wrote:
> On Feb 19, 2008 11:54 AM, hashcollision <hashcollision at gmail.com> wrote:
>> I have a library change request: currently partial Objects created by
>> functools.partial has the "keywords" attribute. However, the standard
>> name for something like it is kwargs. I request that it be renamed to
>> "kwargs".
> 
> -1
> 
> The change has minor benefit while breaking code in the process. If
> you want this to happen it is going to take at least a full release
> cycle of having a deprecation warning on the old attribute name and I
> still don't see enough benefit to bother.

Calling 'kwargs' a standard name is overstating the case quite a bit as 
well (I always use '**kwds' myself).

Cheers,
Nick.

P.S. -1 if that wasn't obvious ;)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jimjjewett at gmail.com  Wed Feb 20 02:33:48 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 19 Feb 2008 20:33:48 -0500
Subject: [Python-3000] PEP 3102
In-Reply-To: <aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>
References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net>
	<ca471dc20802141513i1f62c9d9mea4deb8d16f10b65@mail.gmail.com>
	<f9c7d49c-2807-4f5d-9a97-017de341dbda@60g2000hsy.googlegroups.com>
	<47B9DEF9.2070606@acm.org>
	<6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com>
	<aac2c7cb0802181729x4c5ee5f4tc85daa67c6ac379b@mail.gmail.com>
Message-ID: <fb6fbf560802191733l254a3ea2id914812970b50259@mail.gmail.com>

On 2/18/08, Adam Olsen <rhamph at gmail.com> wrote:
> On Feb 18, 2008 1:17 PM, Arnaud Delobelle <arnodel at googlemail.com> wrote:
> > A typical use of positional-only arguments is with a function

> > def f(x, y=1, **kwargs):
> >         ...
> >
> > where keyword arguments are potentially anything at all, including x
> > and y.  For example: dict.update().

> I worry that dict is not just an example, it may be the *only*
> example.  Does somebody have a use case that doesn't involve dict?

Some existing functions are positional only, often because they expose
underlying (positional-only) C library functions.  Attemping to make a
duck-type replacement just doesn't feel as clean if the arguments are
named.

That said, I don't think it does any actual damage except for the "and
it takes arbitrary keywords" case.  Even then, a reserved set (such as
__* names) is usually acceptable.

-jJ

From ronaldoussoren at mac.com  Wed Feb 20 15:54:30 2008
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 20 Feb 2008 15:54:30 +0100
Subject: [Python-3000] detect file encoding or always use the default,
	UTF-8?
In-Reply-To: <ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>
References: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>
	<ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>
Message-ID: <DE65E9A2-70D6-44FF-A56B-C9679B762F62@mac.com>


On 19 Feb, 2008, at 19:33, Guido van Rossum wrote:

> Well, we're basically hoping that the folks who actually uses Python
> to read and write text files containing non-ASCII characters on OSX
> tell us what they want. At least that's where I am. Since I personally
> still live in a nearly-ASCII world (and probably always will), my own
> experience just doesn't give me any guidance as to what would be the
> most useful.

I'd prefer UTF-8 because that's a much more standard format in the  
wider world.
>
>
> How does Apple's TextEdit do the guessing (assuming it guesses at  
> all)?

TextEdit by default saves as UTF-8 on 10.5 (at encoding set to  
"automatic" in the preferences).

Ronald

>
>
> I just typed a few non-ASCII characters into a text file with TextEdit
> on OSX 10.5, and it seems to have written the file in MacRoman. This
> with the preferences for open and save set to automatic.
>
> --Guido
>
> On Feb 18, 2008 4:32 PM, Kumar McMillan <kumar.mcmillan at gmail.com>  
> wrote:
>> Hello.
>>
>> In Python 3, when opening a file without declaring the encoding
>> keyword is Python going to guess the encoding of the file or just use
>> the default?  That is, assume the file is UTF-8?
>>
>> What led me to wonder this is that on Mac OS X 10.4.11, opening a  
>> file
>> containing UTF-8 encoded text creates a unicode object under the
>> assumption the text was "mac roman" encoding.  This seems like a bug.
>> It works if I set the encoding to UTF-8 explicitly.  I can post the
>> code but it sounds like there are some Mac encoding issues in flux so
>> I thought to ask first.
>>
>> thanks, Kumar
>> _______________________________________________
>> Python-3000 mailing list
>> Python-3000 at python.org
>> http://mail.python.org/mailman/listinfo/python-3000
>> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>>
>
>
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20080220/f294a21e/attachment.bin 

From kumar.mcmillan at gmail.com  Wed Feb 20 16:30:57 2008
From: kumar.mcmillan at gmail.com (Kumar McMillan)
Date: Wed, 20 Feb 2008 09:30:57 -0600
Subject: [Python-3000] detect file encoding or always use the default,
	UTF-8?
In-Reply-To: <DE65E9A2-70D6-44FF-A56B-C9679B762F62@mac.com>
References: <b571660a0802181632k5acf9781v410f7229b316c0c7@mail.gmail.com>
	<ca471dc20802191033y1f52fd6cm80e0f7176d3ff696@mail.gmail.com>
	<DE65E9A2-70D6-44FF-A56B-C9679B762F62@mac.com>
Message-ID: <b571660a0802200730h29b6751ey6c3079f7cee307eb@mail.gmail.com>

On Feb 20, 2008 8:54 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> I'd prefer UTF-8 because that's a much more standard format in the
> wider world.

I agree.  Mac OS Roman is only marginally better than ASCII.  UTF-8
would be more versatile.

From python3now at gmail.com  Wed Feb 20 20:34:22 2008
From: python3now at gmail.com (James Thiele)
Date: Wed, 20 Feb 2008 11:34:22 -0800
Subject: [Python-3000] PEP 3119 "Open issues"
Message-ID: <8f01efd00802201134v96c9bccu9095ea178631e8df@mail.gmail.com>

PEP 3119 has some "Open issues", three of which I list below. These
have (I assume) probably been resolved but it is hard to know what
search terms to use to find them in this mailing list. It would be
useful to record the decisions in the PEP, or at least resolve them if
they are still open issues.

(in "One Trick Ponies") Open issues: Conceivably, instead of using the
ABCMeta metaclass, these classes could override __instancecheck__ and
__subclasscheck__ to check for the presence of the applicable special
method

(in Sequence) Open issues: Other candidate methods, which can all have
default concrete implementations that only depend on __len__ and
__getitem__ with an integer argument: __reversed__, index, count,
__add__, __mul__.

(in Strings) Open issues: define the base interfaces for these so
alternative implementations and subclasses know what they are in for.
This may be the subject of a new PEP or PEPs (PEP 358 should be
co-opted for the bytes type).

From barry at python.org  Thu Feb 21 16:01:56 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 21 Feb 2008 10:01:56 -0500
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
Message-ID: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I know this will get shot down like a Navy missile aimed at an errant  
spy satellite, but I'm going to suggest it anyway.

Several of us have been talking about merging the two Emacs Python  
modes and this prompted a discussion of Ken Manheimer's excellent  
pdbtrack hack, which got me thinking about the way I use pdb.  I don't  
know if this is typical or unique, or whether anyone else is annoyed  
the same way I am.  Admittedly it's a minor annoyance, but I still  
think we can do better.

So I have some particularly troublesome code that I want to set a  
breakpoint at and start stepping through.  I usually will stick this  
tidbit at the place I want to break:

     good_code()
     import pdb; pdb.set_trace()
     call_buggy_code()

In doctests it's a little ickier because I have to put that all on the  
line where I want to break, not before it or doctest won't do the  
right thing, e.g.:

     >>> import pdb; pdb.set_trace(); call_buggy_code()

I suggest that setting a breakpoint be done with a new built-in, e.g  
breakpoint().  So now I would do this:

     good_code()
     breakpoint()
     call_buggy_code()

or

     >>> breakpoint(); call_buggy_code()

Yes, all it saves me is an import, but it also elevates debugging to a  
more important, and more visible location.

Okay, so now that I've convinced you this is the most brilliant idea  
this century <wink>, I'm really going to smoke some crack.

The other thing this should do is allow us to use different debuggers  
if we wanted.  Maybe today breakpoint() would call into pdb, but maybe  
tomorrow we'll have a better debugger, or someone will have installed  
a better debugger in site-packages (e.g. pydb).  I would suggest that  
breakpoint() consult an environment variable such as PYTHONDEBUGGER to  
see which debugger to invoke (the first time in a session only of  
course).  Perhaps you could also set PYTHONDEBUGGER=ignore to noop any  
and all calls to breakpoint.  Maybe there's a python command line  
switch to mirror $PYTHONDEBUGGER.

trying-not-to-spray-hydrazine-all-over-you-ly y'rs,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR72SZXEjvBPtnXfVAQK7gwP7BKyIMAarMZEPzTQVfR/W8J4KzBEQFhtj
3pLd83I8oQMgP4SD0BapiJZ2BOws3i2TAZ501V8VF8iR50nyNGTYCP4pJ/sy6Cr0
Q79ahrvKq5+eS/17+Zi/c/yC028h585m1pvg9/VALu1decLvz3m6UPxJWsZJ+ldw
Qj5bUtuCJ7c=
=YBMp
-----END PGP SIGNATURE-----

From exarkun at divmod.com  Thu Feb 21 16:23:48 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 21 Feb 2008 10:23:48 -0500
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
In-Reply-To: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
Message-ID: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm>

On Thu, 21 Feb 2008 10:01:56 -0500, Barry Warsaw <barry at python.org> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I know this will get shot down like a Navy missile aimed at an errant
>spy satellite, but I'm going to suggest it anyway.
>
>Several of us have been talking about merging the two Emacs Python
>modes and this prompted a discussion of Ken Manheimer's excellent
>pdbtrack hack, which got me thinking about the way I use pdb.  I don't
>know if this is typical or unique, or whether anyone else is annoyed
>the same way I am.  Admittedly it's a minor annoyance, but I still
>think we can do better.
>
>So I have some particularly troublesome code that I want to set a
>breakpoint at and start stepping through.  I usually will stick this
>tidbit at the place I want to break:
>
>     good_code()
>     import pdb; pdb.set_trace()
>     call_buggy_code()

I do this all the time, too.  Basically any time I need to use a debugger
to fix something.

>
>In doctests it's a little ickier because I have to put that all on the
>line where I want to break, not before it or doctest won't do the
>right thing, e.g.:
>
>     >>> import pdb; pdb.set_trace(); call_buggy_code()
>
>I suggest that setting a breakpoint be done with a new built-in, e.g
>breakpoint().  So now I would do this:
>
>     good_code()
>     breakpoint()
>     call_buggy_code()
>
>or
>
>     >>> breakpoint(); call_buggy_code()
>
>Yes, all it saves me is an import, but it also elevates debugging to a
>more important, and more visible location.
>
>Okay, so now that I've convinced you this is the most brilliant idea
>this century <wink>, I'm really going to smoke some crack.
>
>The other thing this should do is allow us to use different debuggers
>if we wanted.  Maybe today breakpoint() would call into pdb, but maybe
>tomorrow we'll have a better debugger, or someone will have installed
>a better debugger in site-packages (e.g. pydb).  I would suggest that
>breakpoint() consult an environment variable such as PYTHONDEBUGGER to
>see which debugger to invoke (the first time in a session only of
>course).  Perhaps you could also set PYTHONDEBUGGER=ignore to noop any
>and all calls to breakpoint.  Maybe there's a python command line
>switch to mirror $PYTHONDEBUGGER.
>
>trying-not-to-spray-hydrazine-all-over-you-ly y'rs,
>- -Barry

Wouldn't a less invasive solution be to make pdb better?  In particular,
to make `breakpoint? actually reliably set a breakpoint?

Jean-Paul

From facundobatista at gmail.com  Thu Feb 21 16:28:40 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 21 Feb 2008 12:28:40 -0300
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
In-Reply-To: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
References: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
Message-ID: <e04bdf310802210728h6b8c62b8g371c66bf752801d9@mail.gmail.com>

2008/2/21, Barry Warsaw <barry at python.org>:

>      good_code()
>      import pdb; pdb.set_trace()
>      call_buggy_code()
>
>  ...
>
>      good_code()
>      breakpoint()
>      call_buggy_code()

"import gdb;gdb.set_trace()" is my second-most-used debugging tool
(the first one is "print"), so I'm with you here!

+1

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From aahz at pythoncraft.com  Thu Feb 21 16:41:32 2008
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 21 Feb 2008 07:41:32 -0800
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
In-Reply-To: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
References: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
Message-ID: <20080221154132.GA9990@panix.com>

On Thu, Feb 21, 2008, Barry Warsaw wrote:
> 
> So I have some particularly troublesome code that I want to set a  
> breakpoint at and start stepping through.  I usually will stick this  
> tidbit at the place I want to break:
> 
>      good_code()
>      import pdb; pdb.set_trace()
>      call_buggy_code()
> 
> In doctests it's a little ickier because I have to put that all on the  
> line where I want to break, not before it or doctest won't do the  
> right thing, e.g.:
> 
>      >>> import pdb; pdb.set_trace(); call_buggy_code()
> 
> I suggest that setting a breakpoint be done with a new built-in, e.g  
> breakpoint().  So now I would do this:
> 
>      good_code()
>      breakpoint()
>      call_buggy_code()

-0

>From my POV, this is cruft in the builtins (I mostly do web apps) -- and
I don't particularly like your suggestions for modifying the result of
breakpoint() (note that Facundo wants to use gdb).  Why not simply
inject set_trace() into the builtins in your startup code?  You can
control whether it gets injected with a DEBUG env var or something.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From facundobatista at gmail.com  Thu Feb 21 16:51:09 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 21 Feb 2008 12:51:09 -0300
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
In-Reply-To: <20080221154132.GA9990@panix.com>
References: <C15961F9-23FC-441C-AFF9-3EB34A4C2580@python.org>
	<20080221154132.GA9990@panix.com>
Message-ID: <e04bdf310802210751w76905fb0ta4110a136c9fe924@mail.gmail.com>

2008/2/21, Aahz <aahz at pythoncraft.com>:

 >  >From my POV, this is cruft in the builtins (I mostly do web apps) -- and
 >  I don't particularly like your suggestions for modifying the result of
 >  breakpoint() (note that Facundo wants to use gdb).  Why not simply

No, no, my mistake, I also use pdb.

 BTW, I use this even when doing PyGTK... didn't try it on web
 development, though.

 --
 .    Facundo

 Blog: http://www.taniquetil.com.ar/plog/
 PyAr: http://www.python.org/ar/

From rasky at develer.com  Thu Feb 21 21:18:19 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 21 Feb 2008 20:18:19 +0000 (UTC)
Subject: [Python-3000] Windows, sys.argv and unicode
References: <fp6uvs$p9$1@ger.gmane.org>
	<ca471dc20802160749q2f958322ve21e60a97e838def@mail.gmail.com>
	<fp737b$s0c$1@ger.gmane.org>
Message-ID: <fpkmab$qg$1@ger.gmane.org>

On Sat, 16 Feb 2008 16:32:43 +0000, Giovanni Bajo wrote:

>> At the very least, can you add the issue to the tracker so we'll have a
>> permanent reminder until it's resolved?
> 
> Done: issue 2128. 

The patch was uploaded a few days ago. Can any Windows maintainer find 
some time to review it?

Thanks!
-- 
Giovanni Bajo


From dalke at dalkescientific.com  Wed Feb 20 21:45:40 2008
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Wed, 20 Feb 2008 21:45:40 +0100
Subject: [Python-3000] removing destructuring del
Message-ID: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>

For lack of a better term, I'll call this a destructuring del

    del (a, (b, (c)))

I propose that it should not be valid in Python 3.

It's supported in 2.x and 3.0a2.  It's never caused anyone problems.   
No one (that I can find through 10 minutes of grepping) uses it.   
When I figured out that I could do it, by looking at the grammar, I  
didn't believe it was valid.  I still can't figure out why anyone  
would use it.

Python 3 removes support for destructuring in function calls

def f((a,b)):  # SyntaxError in Python3
    pass

and I think this falls into the same category.

I would even argue that

    del(x)

should not be allowed because it's suggests and is likely based on  
the false belief that 'del' is a function call.  But I'm not  
proposing getting rid of return(x).


				Andrew
				dalke at dalkescientific.com



From guido at python.org  Fri Feb 22 02:25:32 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 21 Feb 2008 17:25:32 -0800
Subject: [Python-3000] removing destructuring del
In-Reply-To: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
Message-ID: <ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>

What syntax would you allow instead? Just del variable, variable, ...?

On Wed, Feb 20, 2008 at 12:45 PM, Andrew Dalke
<dalke at dalkescientific.com> wrote:
> For lack of a better term, I'll call this a destructuring del
>
>     del (a, (b, (c)))
>
>  I propose that it should not be valid in Python 3.
>
>  It's supported in 2.x and 3.0a2.  It's never caused anyone problems.
>  No one (that I can find through 10 minutes of grepping) uses it.
>  When I figured out that I could do it, by looking at the grammar, I
>  didn't believe it was valid.  I still can't figure out why anyone
>  would use it.
>
>  Python 3 removes support for destructuring in function calls
>
>  def f((a,b)):  # SyntaxError in Python3
>     pass
>
>  and I think this falls into the same category.
>
>  I would even argue that
>
>     del(x)
>
>  should not be allowed because it's suggests and is likely based on
>  the false belief that 'del' is a function call.  But I'm not
>  proposing getting rid of return(x).
>
>
>                                 Andrew
>                                 dalke at dalkescientific.com
>
>
>  _______________________________________________
>  Python-3000 mailing list
>  Python-3000 at python.org
>  http://mail.python.org/mailman/listinfo/python-3000
>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dalke at dalkescientific.com  Fri Feb 22 08:27:37 2008
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Fri, 22 Feb 2008 08:27:37 +0100
Subject: [Python-3000] removing destructuring del
In-Reply-To: <ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
Message-ID: <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>

On Feb 22, 2008, at 2:25 AM, Guido van Rossum wrote:
> What syntax would you allow instead? Just del variable, variable, ...?

Yes.  These are and should be legal

del a, b[1], c.d
del (e+f).g[9]().h

but I think this should not be legal

del a, (b, c)

That should raise

   SyntaxError: can't delete tuple

and I would also like it if the following raised the same error

   del (a, b)

but people use it now and I can see that perhaps, like with the from  
statement, using '()' for long expressions is useful

   del (some_very_large_variable_name[1],
        another_large_variable_name[2].with_attribute)


I also think

   del a,

should not be legal ("SyntaxError: trailing comma not allowed without  
surrounding parentheses"?), but that's getting into my own personal  
preferences.

				Andrew
				dalke at dalkescientific.com



From guido at python.org  Fri Feb 22 16:44:25 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 22 Feb 2008 07:44:25 -0800
Subject: [Python-3000] removing destructuring del
In-Reply-To: <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
Message-ID: <ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>

On Thu, Feb 21, 2008 at 11:27 PM, Andrew Dalke
<dalke at dalkescientific.com> wrote:
> On Feb 22, 2008, at 2:25 AM, Guido van Rossum wrote:
>  > What syntax would you allow instead? Just del variable, variable, ...?
>
>  Yes.  These are and should be legal
>
>  del a, b[1], c.d
>  del (e+f).g[9]().h
>
>  but I think this should not be legal
>
>  del a, (b, c)
>
>  That should raise
>
>    SyntaxError: can't delete tuple
>
>  and I would also like it if the following raised the same error
>
>    del (a, b)
>
>  but people use it now and I can see that perhaps, like with the from
>  statement, using '()' for long expressions is useful
>
>    del (some_very_large_variable_name[1],
>         another_large_variable_name[2].with_attribute)

Well, you could rewrite that as two del statements of course... The
only use I can think if would be

del (very_long_name.
        very_long_attribute)

>  I also think
>
>    del a,
>
>  should not be legal ("SyntaxError: trailing comma not allowed without
>  surrounding parentheses"?), but that's getting into my own personal
>  preferences.

These all make sense to me, but at the same time it seems such a minor
issue that I'm not sure we should bother. It would probably end up
being more custom syntax -- right now it just uses exprlist in the
grammar and is limited to assignment targets by the compiler.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexander.belopolsky at gmail.com  Fri Feb 22 18:07:12 2008
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 22 Feb 2008 17:07:12 +0000 (UTC)
Subject: [Python-3000] removing destructuring del
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
Message-ID: <loom.20080222T165155-100@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> 
> On Thu, Feb 21, 2008 at 11:27 PM, Andrew Dalke
> <dalke <at> dalkescientific.com> wrote:
> .. I think this should not be legal
> >
> >  del a, (b, c)
> >
> >  That should raise
> >
> >    SyntaxError: can't delete tuple
..
> These all make sense to me, but at the same time it seems such a minor
> issue that I'm not sure we should bother. It would probably end up
> being more custom syntax -- right now it just uses exprlist in the
> grammar and is limited to assignment targets by the compiler.
> 

I was always wondering why both lists and tuples are allowed as
assignment targets.  As far as I can tell, the only difference is in
the corner cases such as

>>> del []
>>> del ()
  File "<stdin>", line 1
SyntaxError: can't assign to ()
>>> [] = ()
>>> () = ()
  File "<stdin>", line 1
SyntaxError: can't assign to ()

Maybe Py3k should eliminate the list form, but allow empty tuples
as useful in auto-generated code (deleting variable number of
names.)


From greg.ewing at canterbury.ac.nz  Fri Feb 22 21:08:14 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 23 Feb 2008 09:08:14 +1300
Subject: [Python-3000] removing destructuring del
In-Reply-To: <loom.20080222T165155-100@post.gmane.org>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
	<loom.20080222T165155-100@post.gmane.org>
Message-ID: <47BF2BAE.9090709@canterbury.ac.nz>

Alexander Belopolsky wrote:
> I was always wondering why both lists and tuples are allowed as
> assignment targets.

I seem to remember a very early version of Python when you had
to use the list form for unpacking lists and the tuple form for
unpacking tuples, and you'd be told off for using the wrong one.

That was quickly seen to be a silly idea, but both syntaxes
were kept for compatibility.

--
Greg

From dalke at dalkescientific.com  Fri Feb 22 21:42:18 2008
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Fri, 22 Feb 2008 21:42:18 +0100
Subject: [Python-3000] removing destructuring del
In-Reply-To: <loom.20080222T165155-100@post.gmane.org>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
	<loom.20080222T165155-100@post.gmane.org>
Message-ID: <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com>

On Feb 22, 2008, at 6:07 PM, Alexander Belopolsky wrote:
> Maybe Py3k should eliminate the list form, but allow empty tuples
> as useful in auto-generated code (deleting variable number of
> names.)

That seems pretty theoretical.  It's easy to add a "if has variables"  
check first.  I looked at quite a few libraries grepping for unusual  
del statements and didn't find any, including nothing like 'del []'

BTW, there's a bug in the error message for 'del ()'

Python 2.5 (r25:51908, Oct  9 2006, 13:24:22)
[GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
Type "help", "copyright", "credits" or "license" for more information.
 >>> del []
 >>> del ()
   File "<stdin>", line 1
SyntaxError: can't assign to ()
 >>>


Python 3.0a2 (r30a2:59382, Feb 20 2008, 12:09:57)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> del []
 >>> del ()
SyntaxError: can't assign to () (<stdin>, line 1)
 >>>

This is the exception I want to see when someone does a destructuring  
del, though with corrected wording.

				Andrew
				dalke at dalkescientific.com



From guido at python.org  Fri Feb 22 22:49:16 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 22 Feb 2008 13:49:16 -0800
Subject: [Python-3000] removing destructuring del
In-Reply-To: <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
	<loom.20080222T165155-100@post.gmane.org>
	<1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com>
Message-ID: <ca471dc20802221349w213e691fr214a2bf4e93da8a8@mail.gmail.com>

On Fri, Feb 22, 2008 at 12:42 PM, Andrew Dalke
<dalke at dalkescientific.com> wrote:
> On Feb 22, 2008, at 6:07 PM, Alexander Belopolsky wrote:
>  > Maybe Py3k should eliminate the list form, but allow empty tuples
>  > as useful in auto-generated code (deleting variable number of
>  > names.)
>
>  That seems pretty theoretical.  It's easy to add a "if has variables"
>  check first.  I looked at quite a few libraries grepping for unusual
>  del statements and didn't find any, including nothing like 'del []'
>
>  BTW, there's a bug in the error message for 'del ()'
>
>  Python 2.5 (r25:51908, Oct  9 2006, 13:24:22)
>  [GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
>  Type "help", "copyright", "credits" or "license" for more information.
>
>  >>> del []
>   >>> del ()
>    File "<stdin>", line 1
>  SyntaxError: can't assign to ()
>   >>>

The only bug I see here is that it doesn't complain about "del []".

>  Python 3.0a2 (r30a2:59382, Feb 20 2008, 12:09:57)
>  [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
>  Type "help", "copyright", "credits" or "license" for more information.
>   >>> del []
>   >>> del ()
>  SyntaxError: can't assign to () (<stdin>, line 1)
>   >>>
>
>  This is the exception I want to see when someone does a destructuring
>  del, though with corrected wording.

I guess if you're really motivated to get this through, you'll have to
submit a patch, including unittests and doc updates.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dalke at dalkescientific.com  Fri Feb 22 23:00:35 2008
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Fri, 22 Feb 2008 23:00:35 +0100
Subject: [Python-3000] removing destructuring del
In-Reply-To: <ca471dc20802221349w213e691fr214a2bf4e93da8a8@mail.gmail.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
	<loom.20080222T165155-100@post.gmane.org>
	<1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com>
	<ca471dc20802221349w213e691fr214a2bf4e93da8a8@mail.gmail.com>
Message-ID: <12FF10F2-E91D-47A3-B9CF-416BFFADF55F@dalkescientific.com>

On Feb 22, 2008, at 10:49 PM, Guido van Rossum wrote:
>>>>> del []
>>>>> del ()
>>    File "<stdin>", line 1
>>  SyntaxError: can't assign to ()
>>>>>
>
> The only bug I see here is that it doesn't complain about "del []".

I expected it to say "can't delete" instead of "can't assign"

 >>> del 1
SyntaxError: can't delete literal (<stdin>, line 1)


> I guess if you're really motivated to get this through, you'll have to
> submit a patch, including unittests and doc updates.

I figured that was a likely response - and it's better than "nope" or  
"too late".  Mmm, _ast.c, I presume?  Let me find my pith helmet.  :)



				Andrew
				dalke at dalkescientific.com



From barry at python.org  Sat Feb 23 00:45:23 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 22 Feb 2008 18:45:23 -0500
Subject: [Python-3000] Python 2.6 and 3.0
Message-ID: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi everyone,

I've volunteered to be the release manager for Python 2.6 and 3.0.   
It's been several years since I've RM'd a Python release, and I'm  
happy to do it again (he says while the medication is still  
working :).  I would like to get the next alpha releases of both  
versions out before Pycon, so I propose next Friday, February 29 for  
both.

Guido reminded me that we released Python 1.6 and 2.0 together and it  
makes sense to both of us to do the same for Python 2.6 and 3.0.  I  
don't think it will be that much more work (for me at least :) to  
release them in lockstep, so I think we should try it.  I won't try to  
sync their pre-release version numbers except at the milestones (e.g.  
first beta, release candidates, final releases).

I propose to change PEP 361 to outline the release schedule for both  
Python 2.6 and 3.0.  I'm hoping we can work out a more definite  
schedule at Pycon, but for now I want to at least describe the  
lockstep release schedule and the Feb 29 date.

I'd also like for us to consider doing regular monthly releases.   
Several other FLOSS projects I'm involved with are doing this to very  
good success.  The nice thing is that everyone knows well in advance  
when the next release is going to happen, and so all developers and  
users know what to expect and what is needed from them.

I'd like to propose that we do a joint release the last Friday of  
every month.  For the alphas, it's basically what's in svn.  This  
gives us some time to experiment with the process out and see if we  
like it enough to keep it going through the betas and final releases.

Comments?
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR79ek3EjvBPtnXfVAQJrmQP+KAzy0lSzake2BZsVxErD0kpFQJM+Iij0
qN86wjH0NoqARMfYKVA6eUzEY8vZPu7sJl+SjCOmhnE/KP+l/ArOdis5smiSKwQI
klL4XKd/qdorTMqQ9mWgA0DeBb0Asvln9y64nxzNqgve+36q9j6ytPuRey1GjSI5
nVWoJDb3WsM=
=4SRa
-----END PGP SIGNATURE-----

From brett at python.org  Sat Feb 23 00:54:52 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 22 Feb 2008 15:54:52 -0800
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
Message-ID: <bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>

On Fri, Feb 22, 2008 at 3:45 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>  Hi everyone,
>
>  I've volunteered to be the release manager for Python 2.6 and 3.0.
>  It's been several years since I've RM'd a Python release, and I'm
>  happy to do it again (he says while the medication is still
>  working :).

Can the PSF buy you more of the meds? =)

>  I would like to get the next alpha releases of both
>  versions out before Pycon, so I propose next Friday, February 29 for
>  both.
>

Since they are just alphas, sure. Not like I am going to make any
earth-shattering changes that soon.

>  Guido reminded me that we released Python 1.6 and 2.0 together and it
>  makes sense to both of us to do the same for Python 2.6 and 3.0.  I
>  don't think it will be that much more work (for me at least :) to
>  release them in lockstep, so I think we should try it.  I won't try to
>  sync their pre-release version numbers except at the milestones (e.g.
>  first beta, release candidates, final releases).
>
>  I propose to change PEP 361 to outline the release schedule for both
>  Python 2.6 and 3.0.  I'm hoping we can work out a more definite
>  schedule at Pycon, but for now I want to at least describe the
>  lockstep release schedule and the Feb 29 date.
>
>  I'd also like for us to consider doing regular monthly releases.
>  Several other FLOSS projects I'm involved with are doing this to very
>  good success.  The nice thing is that everyone knows well in advance
>  when the next release is going to happen, and so all developers and
>  users know what to expect and what is needed from them.
>
>  I'd like to propose that we do a joint release the last Friday of
>  every month.  For the alphas, it's basically what's in svn.  This
>  gives us some time to experiment with the process out and see if we
>  like it enough to keep it going through the betas and final releases.
>
>  Comments?

If you want to do monthly alphas, go for it! But if you are going to
do that frequently is a source release going to make more sense than
doing binary builds?

-Brett

From python at rcn.com  Sat Feb 23 01:29:32 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 22 Feb 2008 19:29:32 -0500 (EST)
Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0
Message-ID: <20080222192932.AHK93803@ms19.lnh.mail.rcn.net>

[Barry]
> I'd also like for us to consider doing regular monthly releases. 

+1

From brett at python.org  Sat Feb 23 04:46:20 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 22 Feb 2008 19:46:20 -0800
Subject: [Python-3000] Please sign up for the PyCon sprint if you are
	attending!
Message-ID: <bbaeab100802221946x3510d6b3lf714e342afd3959e@mail.gmail.com>

With the PyCon sprint approaching I would like all attendees to be
signed up on the wiki page for the sprint:
http://wiki.python.org/moin/PyCon2008/SprintSignups/Python . I am
going to be using that list to send out an email about what is exactly
expected of people in terms of setup ahead of time, etc.

And even if you do know what you are doing, I still would appreciate
people adding themselves so I can have a head count. That will let the
sprint coordinators make sure we have a space big enough for us.

-Brett

From nnorwitz at gmail.com  Sun Feb 24 00:26:04 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 23 Feb 2008 15:26:04 -0800
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
Message-ID: <ee2a432c0802231526k4e3241f3x1900b4e65c4058e7@mail.gmail.com>

+1 to everything -- n

On Fri, Feb 22, 2008 at 3:45 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>  Hi everyone,
>
>  I've volunteered to be the release manager for Python 2.6 and 3.0.
>  It's been several years since I've RM'd a Python release, and I'm
>  happy to do it again (he says while the medication is still
>  working :).  I would like to get the next alpha releases of both
>  versions out before Pycon, so I propose next Friday, February 29 for
>  both.
>
>  Guido reminded me that we released Python 1.6 and 2.0 together and it
>  makes sense to both of us to do the same for Python 2.6 and 3.0.  I
>  don't think it will be that much more work (for me at least :) to
>  release them in lockstep, so I think we should try it.  I won't try to
>  sync their pre-release version numbers except at the milestones (e.g.
>  first beta, release candidates, final releases).
>
>  I propose to change PEP 361 to outline the release schedule for both
>  Python 2.6 and 3.0.  I'm hoping we can work out a more definite
>  schedule at Pycon, but for now I want to at least describe the
>  lockstep release schedule and the Feb 29 date.
>
>  I'd also like for us to consider doing regular monthly releases.
>  Several other FLOSS projects I'm involved with are doing this to very
>  good success.  The nice thing is that everyone knows well in advance
>  when the next release is going to happen, and so all developers and
>  users know what to expect and what is needed from them.
>
>  I'd like to propose that we do a joint release the last Friday of
>  every month.  For the alphas, it's basically what's in svn.  This
>  gives us some time to experiment with the process out and see if we
>  like it enough to keep it going through the betas and final releases.
>
>  Comments?
>  - -Barry
>
>  -----BEGIN PGP SIGNATURE-----
>  Version: GnuPG v1.4.8 (Darwin)
>
>  iQCVAwUBR79ek3EjvBPtnXfVAQJrmQP+KAzy0lSzake2BZsVxErD0kpFQJM+Iij0
>  qN86wjH0NoqARMfYKVA6eUzEY8vZPu7sJl+SjCOmhnE/KP+l/ArOdis5smiSKwQI
>  klL4XKd/qdorTMqQ9mWgA0DeBb0Asvln9y64nxzNqgve+36q9j6ytPuRey1GjSI5
>  nVWoJDb3WsM=
>  =4SRa
>  -----END PGP SIGNATURE-----
>  _______________________________________________
>  Python-3000 mailing list
>  Python-3000 at python.org
>  http://mail.python.org/mailman/listinfo/python-3000
>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/nnorwitz%40gmail.com
>

From barry at python.org  Sun Feb 24 13:32:02 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 24 Feb 2008 07:32:02 -0500
Subject: [Python-3000] Putting pdb.set_trace() in builtins?
In-Reply-To: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm>
References: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm>
Message-ID: <BF149209-AA8D-4E4E-9A32-A7949444C005@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 21, 2008, at 10:23 AM, Jean-Paul Calderone wrote:

> Wouldn't a less invasive solution be to make pdb better?  In  
> particular,
> to make `breakpoint? actually reliably set a breakpoint?

Maybe, although I like the directness and simplicity of setting a  
break point explicitly in the code (maybe inside a conditional,  
etc.).  I just want to type less. :)

Well, it was probably a dumb idea anyway born of minor frustration, so  
I'll drop it.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8FjwnEjvBPtnXfVAQIvqQQApHsTLcflVp4TkVWNhgdRYTcIxaWCpOjS
9XCz40WKgh9tg0m4iiZng+g7/h2rtlL4ythRGJfNFXr0xEGEt62appu87XHOt00g
sOwLfglpRG/pA+RCnmrb/ezNCZMMJYjCfGbbN5huqb4EYUoAH2Pml5GjrG8bJN+G
gozq5Pb1Jxk=
=aELO
-----END PGP SIGNATURE-----

From lists at cheimes.de  Sun Feb 24 15:19:02 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 24 Feb 2008 15:19:02 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
Message-ID: <47C17CD6.1040707@cheimes.de>

Barry Warsaw wrote:
> I'd also like for us to consider doing regular monthly releases.
> Several other FLOSS projects I'm involved with are doing this to very
> good success.  The nice thing is that everyone knows well in advance
> when the next release is going to happen, and so all developers and
> users know what to expect and what is needed from them.
> 
> I'd like to propose that we do a joint release the last Friday of
> every month.  For the alphas, it's basically what's in svn.  This
> gives us some time to experiment with the process out and see if we
> like it enough to keep it going through the betas and final releases.

Thanks for volunteering for the job, Barry!

+1 for release early, release often but I want to point out a resource
issue that may speak against a monthly release cycle. The Windows
binaries still require a large amount of attention and human
interaction. The last Windows binaries were build by me and I spent half
the night ironing out issues and testing the installers. As far as I
know the team only Martin and I have the infrastructure and tools to
build the Windows binaries.

We could cut down the time requirements by shipping only normal binaries
instead of PGO (profile guided optimization) binaries. IMHO we could
also skip the AMD64 builds until we have reached beta stage. But it's
still going to cost either Martin or me the better half of a Friday night.

I won't have time to assist with the Windows builds next Friday. I'm
more than busy with personal life and job interviews. Hopefully I'm
going to find a job that allows me to work on the Python core as much as
during the past few months.


That's for the Windows builds. Now I have a list of bugs and features I
like to see fixed / applied before the next release:

http://bugs.python.org/issue1692335 Fix exception pickling: Move initial
args assignment to BaseException.__new__

http://bugs.python.org/issue1792 (trivial performance patch for marshal)

http://bugs.python.org/issue1569 MS VCRT redist issue. IIRC Martin had a
working solution for it in his sandbox.

http://bugs.python.org/issue1657 my kqueue and epoll patch, just needs
another review

Christian

From barry at python.org  Sun Feb 24 18:51:24 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 24 Feb 2008 12:51:24 -0500
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
Message-ID: <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 22, 2008, at 6:54 PM, Brett Cannon wrote:

>> Hi everyone,
>>
>> I've volunteered to be the release manager for Python 2.6 and 3.0.
>> It's been several years since I've RM'd a Python release, and I'm
>> happy to do it again (he says while the medication is still
>> working :).
>
> Can the PSF buy you more of the meds? =)

Depends on the jurisdiction. :)

>> I would like to get the next alpha releases of both
>> versions out before Pycon, so I propose next Friday, February 29 for
>> both.
>>
>
> Since they are just alphas, sure. Not like I am going to make any
> earth-shattering changes that soon.

Cool.

>> Guido reminded me that we released Python 1.6 and 2.0 together and it
>> makes sense to both of us to do the same for Python 2.6 and 3.0.  I
>> don't think it will be that much more work (for me at least :) to
>> release them in lockstep, so I think we should try it.  I won't try  
>> to
>> sync their pre-release version numbers except at the milestones (e.g.
>> first beta, release candidates, final releases).
>>
>> I propose to change PEP 361 to outline the release schedule for both
>> Python 2.6 and 3.0.  I'm hoping we can work out a more definite
>> schedule at Pycon, but for now I want to at least describe the
>> lockstep release schedule and the Feb 29 date.
>>
>> I'd also like for us to consider doing regular monthly releases.
>> Several other FLOSS projects I'm involved with are doing this to very
>> good success.  The nice thing is that everyone knows well in advance
>> when the next release is going to happen, and so all developers and
>> users know what to expect and what is needed from them.
>>
>> I'd like to propose that we do a joint release the last Friday of
>> every month.  For the alphas, it's basically what's in svn.  This
>> gives us some time to experiment with the process out and see if we
>> like it enough to keep it going through the betas and final releases.
>>
>> Comments?
>
> If you want to do monthly alphas, go for it! But if you are going to
> do that frequently is a source release going to make more sense than
> doing binary builds?

It very well might.  See Christian Heimes's follow up re: Windows  
builds.  OTOH, I'm okay if at least for the alphas, the binary builds  
lag behind the source releases, though I'd like to get the process as  
streamlined as possible.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8GunXEjvBPtnXfVAQIP0AQAo5F2tH1vXWbMAFGARZN576xopbQXSokX
uVNXbeg5yjopCx38sHb5OCbublyIDOO8/2ubuuQ6uvAOJc3Br4BuMGHoC5ymQGqf
6pZYZLf4YUGLqFlYOB/huXpJPfH98RJJnK99zA8IQh4B7pN4xg14MF22gGij3Ybt
z2hoy1EbYEk=
=hW7b
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Sun Feb 24 19:57:41 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 24 Feb 2008 19:57:41 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
Message-ID: <47C1BE25.9050300@v.loewis.de>

> It very well might.  See Christian Heimes's follow up re: Windows  
> builds.  OTOH, I'm okay if at least for the alphas, the binary builds  
> lag behind the source releases, though I'd like to get the process as  
> streamlined as possible.

I can continue to provide Windows binaries if desired.

Regards,
Martin

From brett at python.org  Sun Feb 24 22:17:55 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 24 Feb 2008 13:17:55 -0800
Subject: [Python-3000] Buildbots are all red
Message-ID: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>

test_types is triggering an assertion error::

  Objects/stringlib/string_format.h:497: failed assertion
`PyString_Check(result)'

A *really* quick session with gdb would seem to suggest that 'result'
is actually a str, so I don't know what is going on. Don't have time
to dig any deeper right now.

-Brett

From eric+python-dev at trueblade.com  Sun Feb 24 22:19:20 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Sun, 24 Feb 2008 16:19:20 -0500
Subject: [Python-3000] Buildbots are all red
In-Reply-To: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>
References: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>
Message-ID: <47C1DF58.6060903@trueblade.com>

Brett Cannon wrote:
> test_types is triggering an assertion error::
> 
>   Objects/stringlib/string_format.h:497: failed assertion
> `PyString_Check(result)'
> 
> A *really* quick session with gdb would seem to suggest that 'result'
> is actually a str, so I don't know what is going on. Don't have time
> to dig any deeper right now.

I'll check.

Eric.

From guido at python.org  Sun Feb 24 22:26:00 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 24 Feb 2008 13:26:00 -0800
Subject: [Python-3000] Buildbots are all red
In-Reply-To: <47C1DF58.6060903@trueblade.com>
References: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>
	<47C1DF58.6060903@trueblade.com>
Message-ID: <ca471dc20802241326u42e46b99q683a0942dce28e41@mail.gmail.com>

If this is Py3k, perhaps the result should be a PyUnicode instead?

On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Brett Cannon wrote:
>  > test_types is triggering an assertion error::
>  >
>  >   Objects/stringlib/string_format.h:497: failed assertion
>  > `PyString_Check(result)'
>  >
>  > A *really* quick session with gdb would seem to suggest that 'result'
>  > is actually a str, so I don't know what is going on. Don't have time
>  > to dig any deeper right now.
>
>  I'll check.
>
>  Eric.
>
>
> _______________________________________________
>  Python-3000 mailing list
>  Python-3000 at python.org
>  http://mail.python.org/mailman/listinfo/python-3000
>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric+python-dev at trueblade.com  Sun Feb 24 22:31:13 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Sun, 24 Feb 2008 16:31:13 -0500
Subject: [Python-3000] Buildbots are all red
In-Reply-To: <ca471dc20802241326u42e46b99q683a0942dce28e41@mail.gmail.com>
References: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>	
	<47C1DF58.6060903@trueblade.com>
	<ca471dc20802241326u42e46b99q683a0942dce28e41@mail.gmail.com>
Message-ID: <47C1E221.4020108@trueblade.com>

Guido van Rossum wrote:
> If this is Py3k, perhaps the result should be a PyUnicode instead?

Probably so, yes.

But I can't reproduce this on a Fedora Core 6 box.  I'm compiling under 
Windows now to see if I can reproduce it there.  I'll change it to 
PyUnicode_Check shortly, even if I can't reproduce it.

Eric.

> 
> On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Brett Cannon wrote:
>>  > test_types is triggering an assertion error::
>>  >
>>  >   Objects/stringlib/string_format.h:497: failed assertion
>>  > `PyString_Check(result)'
>>  >
>>  > A *really* quick session with gdb would seem to suggest that 'result'
>>  > is actually a str, so I don't know what is going on. Don't have time
>>  > to dig any deeper right now.
>>
>>  I'll check.
>>
>>  Eric.
>>
>>
>> _______________________________________________
>>  Python-3000 mailing list
>>  Python-3000 at python.org
>>  http://mail.python.org/mailman/listinfo/python-3000
>>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>>
> 
> 
> 


From eric+python-dev at trueblade.com  Sun Feb 24 22:46:30 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Sun, 24 Feb 2008 16:46:30 -0500
Subject: [Python-3000] Buildbots are all red
In-Reply-To: <47C1E221.4020108@trueblade.com>
References: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>		<47C1DF58.6060903@trueblade.com>	<ca471dc20802241326u42e46b99q683a0942dce28e41@mail.gmail.com>
	<47C1E221.4020108@trueblade.com>
Message-ID: <47C1E5B6.7070407@trueblade.com>

Eric Smith wrote:
> Guido van Rossum wrote:
>> If this is Py3k, perhaps the result should be a PyUnicode instead?
> 
> Probably so, yes.
> 
> But I can't reproduce this on a Fedora Core 6 box.  I'm compiling under 
> Windows now to see if I can reproduce it there.  I'll change it to 
> PyUnicode_Check shortly, even if I can't reproduce it.

I reproduced it on Windows, and checked in the fix, which Guido 
diagnosed correctly.  I'm not sure what the issue with my FC6 box is, 
I'll investigate.

>> On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith
>> <eric+python-dev at trueblade.com> wrote:
>>> Brett Cannon wrote:
>>>  > test_types is triggering an assertion error::
>>>  >
>>>  >   Objects/stringlib/string_format.h:497: failed assertion
>>>  > `PyString_Check(result)'
>>>  >
>>>  > A *really* quick session with gdb would seem to suggest that 'result'
>>>  > is actually a str, so I don't know what is going on. Don't have time
>>>  > to dig any deeper right now.
>>>
>>>  I'll check.
>>>
>>>  Eric.
>>>
>>>
>>> _______________________________________________
>>>  Python-3000 mailing list
>>>  Python-3000 at python.org
>>>  http://mail.python.org/mailman/listinfo/python-3000
>>>  Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>>>
>>
>>
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/eric%2Bpython-dev%40trueblade.com
> 


From lists at cheimes.de  Sun Feb 24 23:35:18 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 24 Feb 2008 23:35:18 +0100
Subject: [Python-3000] Buildbots are all red
In-Reply-To: <47C1E5B6.7070407@trueblade.com>
References: <bbaeab100802241317g61f47fafmb036761fb1cefd90@mail.gmail.com>		<47C1DF58.6060903@trueblade.com>	<ca471dc20802241326u42e46b99q683a0942dce28e41@mail.gmail.com>	<47C1E221.4020108@trueblade.com>
	<47C1E5B6.7070407@trueblade.com>
Message-ID: <fpsrf7$cf$1@ger.gmane.org>

Eric Smith wrote:
> Eric Smith wrote:
>> Guido van Rossum wrote:
>>> If this is Py3k, perhaps the result should be a PyUnicode instead?
>> Probably so, yes.
>>
>> But I can't reproduce this on a Fedora Core 6 box.  I'm compiling under 
>> Windows now to see if I can reproduce it there.  I'll change it to 
>> PyUnicode_Check shortly, even if I can't reproduce it.
> 
> I reproduced it on Windows, and checked in the fix, which Guido 
> diagnosed correctly.  I'm not sure what the issue with my FC6 box is, 
> I'll investigate.

I assume it's yet another problem with the makefile. Most likely the
header files aren't listed correctly. I'm working on it.

Christian


From mhammond at skippinet.com.au  Sun Feb 24 23:58:56 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Mon, 25 Feb 2008 09:58:56 +1100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <20080224201613.GL66273@nexus.in-nomine.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
Message-ID: <01a101c87738$da592b90$8f0b82b0$@com.au>

Jeroen Ruigrok van der Werven:
> -On [20080224 19:57], "Martin v. Lwis" (martin at v.loewis.de) wrote:
> >I can continue to provide Windows binaries if desired.
> 
> If need be, I can help testing the build infrastructure since I have
> access
> to various releases of Visual Studio as well.

Me too - I still regularly build the svn version of Python, and although I don't regularly turn them into .MSI files, I'm confident I could learn how to do that ;)  I understand that over time the binary process will need some tweaking, but in the general case, I expect that turning the crank for a test build needn't be that much or a burden. I expect the biggest problem is that I could no longer ignore certain extensions I don't care about, such as Tk.

Cheers,

Mark


From nnorwitz at gmail.com  Mon Feb 25 06:55:11 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 24 Feb 2008 21:55:11 -0800
Subject: [Python-3000] removing destructuring del
In-Reply-To: <ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
Message-ID: <ee2a432c0802242155wa5af222t9fbc7b343b8241ab@mail.gmail.com>

On Fri, Feb 22, 2008 at 7:44 AM, Guido van Rossum <guido at python.org> wrote:

[... del (tuple, of, variables), etc deleted]

>
>  >  I also think
>  >
>  >    del a,
>  >
>  >  should not be legal ("SyntaxError: trailing comma not allowed without
>  >  surrounding parentheses"?), but that's getting into my own personal
>  >  preferences.
>
>  These all make sense to me, but at the same time it seems such a minor
>  issue that I'm not sure we should bother. It would probably end up
>  being more custom syntax -- right now it just uses exprlist in the
>  grammar and is limited to assignment targets by the compiler.

What bothers me more is assert of a tuple.  It does the wrong thing
from what newbies expect.  pychecker warns about this, but the version
of pylint in use at Google doesn't warn about this.  I wonder if we
should add a warning?

The code I recently saw was:
  assert (something_short,
               'Some long assertion string that needs its own line')

The original code that pylint warned about was:
  assert something_short, \
               'Some long assertion string that needs its own line'

Index: Python/compile.c
===================================================================
--- Python/compile.c    (revision 61058)
+++ Python/compile.c    (working copy)
@@ -2056,6 +2056,9 @@
                if (assertion_error == NULL)
                        return 0;
        }
+       if (s->v.Assert.test->kind == Tuple_kind) {
+               printf("got a tuple\n");
+       }
        VISIT(c, expr, s->v.Assert.test);
        end = compiler_new_block(c);
        if (end == NULL)

Seems to do the trick.  Maybe this should be turned into a real patch
with using PyErr_Warn()?

n

From lists at cheimes.de  Mon Feb 25 16:01:21 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 25 Feb 2008 16:01:21 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <01a101c87738$da592b90$8f0b82b0$@com.au>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
Message-ID: <47C2D841.2030501@cheimes.de>

Mark Hammond wrote:
> Me too - I still regularly build the svn version of Python, and although I don't regularly turn them into .MSI files, I'm confident I could learn how to do that ;)  I understand that over time the binary process will need some tweaking, but in the general case, I expect that turning the crank for a test build needn't be that much or a burden. I expect the biggest problem is that I could no longer ignore certain extensions I don't care about, such as Tk.

It's not too hard. Tkinter, bsddb and friends are explained in
PCbuild/README.txt. You should be able to compile them in less than half
an hour.

For the MSI installers you also need Python 2.5, your pywin32 package
and some additional tools like the help compiler and cabarc.exe.

Martin:
Have you solved the problem with the VS CRT redist
(http://bugs.python.org/issue1569)? Maybe Mark is able to assist you.

Christian

From asmodai at in-nomine.org  Sun Feb 24 21:16:13 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sun, 24 Feb 2008 21:16:13 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C1BE25.9050300@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
Message-ID: <20080224201613.GL66273@nexus.in-nomine.org>

-On [20080224 19:57], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>I can continue to provide Windows binaries if desired.

If need be, I can help testing the build infrastructure since I have access
to various releases of Visual Studio as well.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
What's in a name? That which we call a rose by any other name would
smell as sweet...

From asmodai at in-nomine.org  Mon Feb 25 16:44:43 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 25 Feb 2008 16:44:43 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C2D841.2030501@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
Message-ID: <20080225154443.GP66273@nexus.in-nomine.org>

-On [20080225 16:02], Christian Heimes (lists at cheimes.de) wrote:
>For the MSI installers you also need Python 2.5, your pywin32 package
>and some additional tools like the help compiler and cabarc.exe.

Have you looked at http://wix.sourceforge.net/ ?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
What good will it be for a man if he gains the whole world, yet forfeits
his soul?

From lists at cheimes.de  Mon Feb 25 17:06:12 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 25 Feb 2008 17:06:12 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <20080225154443.GP66273@nexus.in-nomine.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
Message-ID: <47C2E774.2040205@cheimes.de>

Jeroen Ruigrok van der Werven wrote:
> Have you looked at http://wix.sourceforge.net/ ?

WiX looks interesting but I'm neither in the position to change the
installer nor do I have a strong opinion. It's Martin's area of expertise.

On the one hand a XML based MSI generator could be easier to maintain.
On the other hand it would introduce another dependency to a 3rd party
tool and things may get complicated if we leave the road. Automation
tools tend to make common things really easy but special and uncommon
stuff really hard.

Christian

From guido at python.org  Mon Feb 25 18:07:23 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 25 Feb 2008 09:07:23 -0800
Subject: [Python-3000] removing destructuring del
In-Reply-To: <ee2a432c0802242155wa5af222t9fbc7b343b8241ab@mail.gmail.com>
References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com>
	<ca471dc20802211725i39c0ef49nb07943ab5531a99e@mail.gmail.com>
	<741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com>
	<ca471dc20802220744w6c0ae599vec3ed979b2b32577@mail.gmail.com>
	<ee2a432c0802242155wa5af222t9fbc7b343b8241ab@mail.gmail.com>
Message-ID: <ca471dc20802250907w791f7a7ax9c28d514543cc016@mail.gmail.com>

On Sun, Feb 24, 2008 at 9:55 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:
>  What bothers me more is assert of a tuple.  It does the wrong thing
>  from what newbies expect.  pychecker warns about this, but the version
>  of pylint in use at Google doesn't warn about this.  I wonder if we
>  should add a warning?
>
>  The code I recently saw was:
>   assert (something_short,
>                'Some long assertion string that needs its own line')
>
>  The original code that pylint warned about was:
>   assert something_short, \
>                'Some long assertion string that needs its own line'
>
>  Index: Python/compile.c
>  ===================================================================
>  --- Python/compile.c    (revision 61058)
>  +++ Python/compile.c    (working copy)
>  @@ -2056,6 +2056,9 @@
>                 if (assertion_error == NULL)
>                         return 0;
>         }
>  +       if (s->v.Assert.test->kind == Tuple_kind) {
>  +               printf("got a tuple\n");
>  +       }
>         VISIT(c, expr, s->v.Assert.test);
>         end = compiler_new_block(c);
>         if (end == NULL)
>
>  Seems to do the trick.  Maybe this should be turned into a real patch
>  with using PyErr_Warn()?

+1

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From hasan.diwan at gmail.com  Mon Feb 25 18:14:59 2008
From: hasan.diwan at gmail.com (Hasan Diwan)
Date: Mon, 25 Feb 2008 09:14:59 -0800
Subject: [Python-3000] Print Statement => Function
Message-ID: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com>

I attended Guido's talk in which he said that the print statement was
going to become a function, but gave no sense of direction. There is a
print-like statement in the file statement called "write". So, perhaps
print could be reworked as follows, for starters:

def print(self, fileobj, stringobj, *kwarg):
  fileobj.write(stringobj)

-- 
Cheers,
Hasan Diwan <hasan.diwan at gmail.com>

From guido at python.org  Mon Feb 25 18:32:36 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 25 Feb 2008 09:32:36 -0800
Subject: [Python-3000] Print Statement => Function
In-Reply-To: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com>
References: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com>
Message-ID: <ca471dc20802250932x4efac826x8ab911d11d9636f2@mail.gmail.com>

PEP 3105

On Mon, Feb 25, 2008 at 9:14 AM, Hasan Diwan <hasan.diwan at gmail.com> wrote:
> I attended Guido's talk in which he said that the print statement was
>  going to become a function, but gave no sense of direction. There is a
>  print-like statement in the file statement called "write". So, perhaps
>  print could be reworked as follows, for starters:
>
>  def print(self, fileobj, stringobj, *kwarg):
>   fileobj.write(stringobj)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Feb 25 22:07:11 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 25 Feb 2008 16:07:11 -0500
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C17CD6.1040707@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<47C17CD6.1040707@cheimes.de>
Message-ID: <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 24, 2008, at 9:19 AM, Christian Heimes wrote:

> Barry Warsaw wrote:
>> I'd also like for us to consider doing regular monthly releases.
>> Several other FLOSS projects I'm involved with are doing this to very
>> good success.  The nice thing is that everyone knows well in advance
>> when the next release is going to happen, and so all developers and
>> users know what to expect and what is needed from them.
>>
>> I'd like to propose that we do a joint release the last Friday of
>> every month.  For the alphas, it's basically what's in svn.  This
>> gives us some time to experiment with the process out and see if we
>> like it enough to keep it going through the betas and final releases.
>
> Thanks for volunteering for the job, Barry!
>
> +1 for release early, release often but I want to point out a resource
> issue that may speak against a monthly release cycle. The Windows
> binaries still require a large amount of attention and human
> interaction. The last Windows binaries were build by me and I spent  
> half
> the night ironing out issues and testing the installers. As far as I
> know the team only Martin and I have the infrastructure and tools to
> build the Windows binaries.

 From the follow ups, it sounds like others can pitch in here.  A  
question though: is it reasonable to hold up the monthly release  
because a binary build we're going to make available can't be done at  
the same time?

My preference (at least for the alphas) is "no".  If we can make a  
source release, and if we can build a binary release from exactly the  
same revision, then we should go ahead and release.  I'd rather get  
the alpha out there and in people's hands.

We'll almost certainly be stricter for the candidates, finals, and  
maybe betas.

> We could cut down the time requirements by shipping only normal  
> binaries
> instead of PGO (profile guided optimization) binaries. IMHO we could
> also skip the AMD64 builds until we have reached beta stage. But it's
> still going to cost either Martin or me the better half of a Friday  
> night.

Dang.  I actually don't know how long it's going to take me to do the  
source release, but I hope it's no more than 3 or 4 hours.

> I won't have time to assist with the Windows builds next Friday. I'm
> more than busy with personal life and job interviews. Hopefully I'm
> going to find a job that allows me to work on the Python core as  
> much as
> during the past few months.

When's the PSF gonna start hiring? :)

> That's for the Windows builds. Now I have a list of bugs and  
> features I
> like to see fixed / applied before the next release:
>
> http://bugs.python.org/issue1692335 Fix exception pickling: Move  
> initial
> args assignment to BaseException.__new__
>
> http://bugs.python.org/issue1792 (trivial performance patch for  
> marshal)
>
> http://bugs.python.org/issue1569 MS VCRT redist issue. IIRC Martin  
> had a
> working solution for it in his sandbox.
>
> http://bugs.python.org/issue1657 my kqueue and epoll patch, just needs
> another review

So, as I mentioned in my last reply, I'm planning to only allow  
critical bugs (as described in roundup) hold up the release.  Right  
now there are no critical bugs open.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8MuAHEjvBPtnXfVAQJ4JQP9F8AijArF8KhyxC7lp0ePDBGphgZjq7h8
9vZZ13oGOCtED/6M4bGDaZWrI1LEcj3iuf61Kdk6KwaeAi3dnGHkrP1XOTxZbLcz
8euKbC8JhBHan/A4SO4+xzxx4ZI9vCMRQqe+sLOQJsE9vH+4UMU1FDrhROxYwLbb
aG0+fzGPdzA=
=zpPQ
-----END PGP SIGNATURE-----

From barry at python.org  Mon Feb 25 22:07:25 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 25 Feb 2008 16:07:25 -0500
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C1BE25.9050300@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
Message-ID: <4CE13670-0F86-444E-9F3F-B94AEA9B8295@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 24, 2008, at 1:57 PM, Martin v. L?wis wrote:

>> It very well might.  See Christian Heimes's follow up re: Windows   
>> builds.  OTOH, I'm okay if at least for the alphas, the binary  
>> builds  lag behind the source releases, though I'd like to get the  
>> process as  streamlined as possible.
>
> I can continue to provide Windows binaries if desired.

Great, thanks!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8MuDXEjvBPtnXfVAQJeJwP/R8MMAhOsyRtpxIISEUoqMGDJksI5EtVM
PcsxNO1p0MGcHfm5lNg0YNYwxsfc/0ghkRbsidegvcyN6BZWgHSaA7I0O1cBTG1x
R4eNmLJBWCOcJNmTGgxCA7G8eEHTNTxneaQ0APO+yQbbHS/eyGGMcmFldNMkDqNO
ycqikt0XiWI=
=M3eC
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Mon Feb 25 23:02:18 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 25 Feb 2008 23:02:18 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C2D841.2030501@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
Message-ID: <47C33AEA.2010600@v.loewis.de>

> Have you solved the problem with the VS CRT redist
> (http://bugs.python.org/issue1569)? Maybe Mark is able to assist you.

No, I still haven't found a solution. I do want to use the merge
module; anything else probably isn't going to work.

Regards,
Martin

From martin at v.loewis.de  Mon Feb 25 23:02:36 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 25 Feb 2008 23:02:36 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <20080225154443.GP66273@nexus.in-nomine.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
Message-ID: <47C33AFC.8010307@v.loewis.de>

>> For the MSI installers you also need Python 2.5, your pywin32 package
>> and some additional tools like the help compiler and cabarc.exe.
> 
> Have you looked at http://wix.sourceforge.net/ ?

Yes.

Martin


From martin at v.loewis.de  Mon Feb 25 23:06:11 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 25 Feb 2008 23:06:11 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C2E774.2040205@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
	<47C2E774.2040205@cheimes.de>
Message-ID: <47C33BD3.5040003@v.loewis.de>

> On the one hand a XML based MSI generator could be easier to maintain.

I've looked at it, and I seriously doubt that. In WiX, you need to
specify a fixed file list (perhaps with wildcards; I'm unsure). This
will be tricky for Python, where the list of files to be installed
changes all the time.

You *need* to have a turing-complete packing language (such as Python).

> On the other hand it would introduce another dependency to a 3rd party
> tool and things may get complicated if we leave the road. Automation
> tools tend to make common things really easy but special and uncommon
> stuff really hard.

Exactly so. When I started with MSI generation, and tried what comes
with Visual Studio, and found it unusable - it did not support Itanium,
and could not be changed to support it.

Regards,
Martin

From db3l.net at gmail.com  Mon Feb 25 23:50:36 2008
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 25 Feb 2008 17:50:36 -0500
Subject: [Python-3000] Python 2.6 and 3.0
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<4CE13670-0F86-444E-9F3F-B94AEA9B8295@python.org>
Message-ID: <m2skzgr6pv.fsf@valheru.db3l.homeip.net>

Barry Warsaw <barry at python.org> writes:

> On Feb 24, 2008, at 1:57 PM, Martin v. L?wis wrote:
>
>>> It very well might.  See Christian Heimes's follow up re: Windows   
>>> builds.  OTOH, I'm okay if at least for the alphas, the binary  
>>> builds  lag behind the source releases, though I'd like to get the  
>>> process as  streamlined as possible.
>>
>> I can continue to provide Windows binaries if desired.
>
> Great, thanks!

Note that my buildbot is still also building MSIs each night based on
the svn head for 2.5, 2.6 and 3.0, and uploading them back to
python.org (viewable at http://www.python.org/dev/daily-msi).

So at least for an alpha based on the current SVN trunk, that might be
an easy place to grab a binary snapshot from, unless I'm missing
something.

Conversely, the machine is there to make builds upon request, I
presume, depending on the master configuration.  I know Martin set the
current scheme up.

-- David



From lists at cheimes.de  Tue Feb 26 00:50:19 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 26 Feb 2008 00:50:19 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C33AEA.2010600@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de>
Message-ID: <47C3543B.8060303@cheimes.de>

Martin v. L?wis wrote:
> No, I still haven't found a solution. I do want to use the merge
> module; anything else probably isn't going to work.

Da...ng
Didn't you prepare a new MSI installer for 3.0a2 that includes the VS
Redist MSM for X86? I vaguely remember that you've replaced my installer
with a new one.

The issue should be resolved before Python 2.6 and 3.0 are reaching beta
stage. Maybe we can get some help from outside?

Christian


From lists at cheimes.de  Tue Feb 26 00:58:32 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 26 Feb 2008 00:58:32 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C33BD3.5040003@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
	<47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de>
Message-ID: <47C35628.6060200@cheimes.de>

Martin v. L?wis wrote:
> I've looked at it, and I seriously doubt that. In WiX, you need to
> specify a fixed file list (perhaps with wildcards; I'm unsure). This
> will be tricky for Python, where the list of files to be installed
> changes all the time.
>
> You *need* to have a turing-complete packing language (such as Python).

You are most likely right. A pure XML based solution ain't going to work
for Python. But how about a mixed solution?

    XML templates -> Python fu -> WiX XML -> MSI

We take some XML templates, modify them from Python and add the files.
Finalliy we let WiX create the MSI installer from the resulting XML file.

What do you think?

> Exactly so. When I started with MSI generation, and tried what comes
> with Visual Studio, and found it unusable - it did not support
> Itanium, and could not be changed to support it.

Yeah, been there, got frustrated, went back :/

Christian


From lists at cheimes.de  Tue Feb 26 01:11:26 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 26 Feb 2008 01:11:26 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<47C17CD6.1040707@cheimes.de>
	<8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>
Message-ID: <47C3592E.2080704@cheimes.de>

Barry Warsaw wrote:
> From the follow ups, it sounds like others can pitch in here.  A
> question though: is it reasonable to hold up the monthly release because
> a binary build we're going to make available can't be done at the same
> time?
> 
> My preference (at least for the alphas) is "no".  If we can make a
> source release, and if we can build a binary release from exactly the
> same revision, then we should go ahead and release.  I'd rather get the
> alpha out there and in people's hands.
> 
> We'll almost certainly be stricter for the candidates, finals, and maybe
> betas.

I agree. It's not reasonable to hold of alphas because the Windows
binaries may be released a few days later. Do we need official Windows
binaries for alphas at all? Martin's buildbot creates nightly Windows
builds. We could point users to the nightly builds and ask them to test
the version.

> Dang.  I actually don't know how long it's going to take me to do the
> source release, but I hope it's no more than 3 or 4 hours.

I guess it's less than 2 hours. You can prepare most of the work like
the announcements a couple of days earlier. I (perhaps naively) assume
you have to smack the whip to get everything in place, do the svn cp to
tag, svn export, tar cz, tar xzf && ./configure && make && make test
dance and upload the tar.gz. Am I missing something important? :]

> When's the PSF gonna start hiring? :)

Dunno :) But I'm probably the last in a long line of Python core
developers to get hired. Don't forget I'm still fresh and a junior core
developer. *jk*

> So, as I mentioned in my last reply, I'm planning to only allow critical
> bugs (as described in roundup) hold up the release.  Right now there are
> no critical bugs open.

#1569 is critical but not important enough to stop an alpha. As I said
in the other mail it should be fixed for the first beta and must be
fixed for the first rc.

Christian


From mhammond at skippinet.com.au  Tue Feb 26 10:09:06 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue, 26 Feb 2008 20:09:06 +1100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C35628.6060200@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
	<47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de>
	<47C35628.6060200@cheimes.de>
Message-ID: <02cf01c87857$4d19bf40$e74d3dc0$@com.au>

> Martin v. L?wis wrote:
> > I've looked at it, and I seriously doubt that. In WiX, you need to
> > specify a fixed file list (perhaps with wildcards; I'm unsure). This
> > will be tricky for Python, where the list of files to be installed
> > changes all the time.
> >
> > You *need* to have a turing-complete packing language (such as
> Python).
> 
> You are most likely right. A pure XML based solution ain't going to
> work
> for Python. But how about a mixed solution?
> 
>     XML templates -> Python fu -> WiX XML -> MSI
> 
> We take some XML templates, modify them from Python and add the files.
> Finalliy we let WiX create the MSI installer from the resulting XML
> file.
> 
> What do you think?

I'm inclined to agree with Martin that WiX doesn't offer us much value (it
offers value in many places though - just not for our requirements given
Martin's msilib).  I believe that once we know how to solve a particular
problem, it would not be significantly easier to implement using WiX than it
would using the current infrastructure.  My problem is still getting my head
around various MSI issues at any level (eg, bdist_msi needs some tweaking to
allow for different releases of the same "package" to be recognized as such,
but I'm not sure what MSI concept I'm dealing with yet...)

WiX is an excellent inspiration though - if a WiX example can be found for
something, it should be a significant help in implementing it via msilib.

Cheers,

Mark



From barry at python.org  Tue Feb 26 13:28:20 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 26 Feb 2008 07:28:20 -0500
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C3592E.2080704@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<47C17CD6.1040707@cheimes.de>
	<8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>
	<47C3592E.2080704@cheimes.de>
Message-ID: <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 25, 2008, at 7:11 PM, Christian Heimes wrote:

> Barry Warsaw wrote:
>> From the follow ups, it sounds like others can pitch in here.  A
>> question though: is it reasonable to hold up the monthly release  
>> because
>> a binary build we're going to make available can't be done at the  
>> same
>> time?
>>
>> My preference (at least for the alphas) is "no".  If we can make a
>> source release, and if we can build a binary release from exactly the
>> same revision, then we should go ahead and release.  I'd rather get  
>> the
>> alpha out there and in people's hands.
>>
>> We'll almost certainly be stricter for the candidates, finals, and  
>> maybe
>> betas.
>
> I agree. It's not reasonable to hold of alphas because the Windows
> binaries may be released a few days later. Do we need official Windows
> binaries for alphas at all? Martin's buildbot creates nightly Windows
> builds. We could point users to the nightly builds and ask them to  
> test
> the version.

That would be find with me.  Where are those Windows binaries  
available for download from?

>> Dang.  I actually don't know how long it's going to take me to do the
>> source release, but I hope it's no more than 3 or 4 hours.
>
> I guess it's less than 2 hours. You can prepare most of the work like
> the announcements a couple of days earlier. I (perhaps naively) assume
> you have to smack the whip to get everything in place, do the svn cp  
> to
> tag, svn export, tar cz, tar xzf && ./configure && make && make test
> dance and upload the tar.gz. Am I missing something important? :]

Dunno yet!  It's been years since I did a release and I really want to  
check out Anthony's welease tool this time.  I may not have time  
before Friday to look at this though.

>> When's the PSF gonna start hiring? :)
>
> Dunno :) But I'm probably the last in a long line of Python core
> developers to get hired. Don't forget I'm still fresh and a junior  
> core
> developer. *jk*

:)

>> So, as I mentioned in my last reply, I'm planning to only allow  
>> critical
>> bugs (as described in roundup) hold up the release.  Right now  
>> there are
>> no critical bugs open.
>
> #1569 is critical but not important enough to stop an alpha. As I said
> in the other mail it should be fixed for the first beta and must be
> fixed for the first rc.

It's not marked critical in roundup, and that's the only thing I'm  
going by!  But it doesn't seem critical in the sense that it should  
hold up the alpha release, IMO.

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8QF5nEjvBPtnXfVAQJhrgP/Xz3IQrlCB9QPGsMGIL+xG3I5t+aThNg6
4n/bMjt4DRzTCRiNBjUllyCb5+VtPfTZu2wFVdi5I7NLMDG4WI4jfDGZlhvodbHW
TPG/7bN/ykx9yE1hUPI5X+Kqrg0lG7Tbp9Zev5eHJCMwParSVu+hfWqD48+1bQqw
JGfzz8AlqE0=
=PQgE
-----END PGP SIGNATURE-----

From asmodai at in-nomine.org  Tue Feb 26 07:31:04 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Tue, 26 Feb 2008 07:31:04 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C33AEA.2010600@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de>
Message-ID: <20080226063104.GU66273@nexus.in-nomine.org>

-On [20080225 23:03], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>No, I still haven't found a solution. I do want to use the merge
>module; anything else probably isn't going to work.

I updated the ticket with some links to how to approach this issue.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
To conquer fear is the beginning of wisdom...

From christian at cheimes.de  Tue Feb 26 15:13:54 2008
From: christian at cheimes.de (Christian Heimes)
Date: Tue, 26 Feb 2008 15:13:54 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<47C17CD6.1040707@cheimes.de>
	<8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>
	<47C3592E.2080704@cheimes.de>
	<7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org>
Message-ID: <47C41EA2.4070508@cheimes.de>

Barry Warsaw wrote:
> That would be find with me.  Where are those Windows binaries available
> for download from?

The Windows builds are hidden in the development section. It took me 10
minutes to find them because I was searching in the download section and
for nightly builds. The *daily* builds are available at
http://www.python.org/dev/daily-msi/

Christian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-3000/attachments/20080226/e9756b6b/attachment.pgp 

From martin at v.loewis.de  Tue Feb 26 21:13:14 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 26 Feb 2008 21:13:14 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <02cf01c87857$4d19bf40$e74d3dc0$@com.au>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
	<47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de>
	<47C35628.6060200@cheimes.de>
	<02cf01c87857$4d19bf40$e74d3dc0$@com.au>
Message-ID: <47C472DA.1070708@v.loewis.de>

> My problem is still getting my head
> around various MSI issues at any level (eg, bdist_msi needs some tweaking to
> allow for different releases of the same "package" to be recognized as such,
> but I'm not sure what MSI concept I'm dealing with yet...)

Don't hesitate to ask here. Not sure what problem you are talking to 
specifically, but I guess you are asking for "upgrade codes"; there is
also upgrade codes and package codes.

Each package file should have its own unique *package? code, mere 
rebuilding should generate a new one. msilib does that correctly.
A package code code can be installed at most once on a system.

Related packages for the same software should share a *product* code.
Rebuilding should not change the package code (but might in msilib;
I'd have to check). Again, installer enforces that a specific package
code can only be installed once on a system. Python assigns a
separate product code for each bug fix release. Product codes
are most useful for uninstallation, e.g. to uninstall Python 2.5.1,
do

   msiexec /x {31800004-6386-4999-a519-518f2d78d8f0}

Use separate product codes if you want to allow for simultaneous
versions.

Subsequent versions of the same product should share an *upgrade*
code. MSI will check the Upgrade table, to see whether a package
with the same upgrade code (but a different product code) is
already installed, and if so, whether the version range matches.
If the installed product is newer, it will refuse to install the
older one. If the installed product is older, it will perform
an "upgrade installation", which involves uninstalling the older
version (possibly on a file-by-file basis), and possibly
migrating the feature selections.

Python uses a single upgrade code (until 2.5.2, which introduces
a separate upgrade code for Win64). It then uses version ranges
to make 2.5.2 an upgrade of 2.5.1 and 2.5.0, but not of 2.4.2
(say), essentially causing only one bug fix release per 2.5.x
to be installed on the system, but allowing simultaneous
installation of 2.5 and 2.4 (say). With 2.5.2, simultaneous
installation of Win64 and Win32 releases on a single system becomes
possible - which also requires to assign separate product codes
to Win64, namely

    2.5.2, Win32: 6b976adf-8ae8-434e-b282-a06c7f624d2f
    2.5.2, Win64: 6b976adf-8ae8-434e-b282-a06c7f624d20

> WiX is an excellent inspiration though - if a WiX example can be found for
> something, it should be a significant help in implementing it via msilib.

The current challenge is merge modules: How can I merge the VC msm into
the Python MSI (including support for SxS).

Regards,
Martin

From martin at v.loewis.de  Tue Feb 26 21:26:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 26 Feb 2008 21:26:20 +0100
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C3543B.8060303@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de>
	<47C3543B.8060303@cheimes.de>
Message-ID: <47C475EC.1020004@v.loewis.de>

>> No, I still haven't found a solution. I do want to use the merge
>> module; anything else probably isn't going to work.
> 
> Da...ng
> Didn't you prepare a new MSI installer for 3.0a2 that includes the VS
> Redist MSM for X86? I vaguely remember that you've replaced my installer
> with a new one.

Right. I produced a package that ships the CRT, but not by using the
merge module. Instead, I arranged to include sufficient copies of the
manifest file to make it work in the non-admin installation (and yes,
you do need to install multiple copies of it - just see the 3.0a2 MSI
file).

> The issue should be resolved before Python 2.6 and 3.0 are reaching beta
> stage. Maybe we can get some help from outside?

Perhaps. I'm confident that I can find a solution when I get the time;
I'm not so confident that I can find the time, though. Of course, I 
would prefer if the outside help would propose something better than
"switch to this completely different technology; it may work" (unless
a complete working solution is presented in that other technology, and
as long as that other technology still creates MSI files with
free-as-in-beer tools).

In any case, contributions are welcome.

Regards,
Martin

From martin at v.loewis.de  Tue Feb 26 21:35:10 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 26 Feb 2008 21:35:10 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C35628.6060200@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>
	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>
	<47C2D841.2030501@cheimes.de>
	<20080225154443.GP66273@nexus.in-nomine.org>
	<47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de>
	<47C35628.6060200@cheimes.de>
Message-ID: <47C477FE.3000301@v.loewis.de>

> What do you think?

Feel free to try it out. I'm skeptical that it will be a better overall
solution than the current one - the main difference would be that, 
rather than me being the only one who can realistically change the
packaging chain, it would be you who is the only one - which, in 
principle, would be fine with me.

I believe you need deep inside knowledge of the MSI database format
for WiX, just as you do for using the automation API. I think I could
learn WiX fairly quickly after all these years of learning MSI in the
first place. I think the WiX designers did right in tying WiX so close
to the MSI data model, but it means that WiX makes package creation not
simpler - merely more productive for the experienced user (who I
hesitate to call WiXers :-)

In any case, when you work with WiX, I'm sure you'll gain a lot of
expert knowledge on Windows packaging. Depending on your job situation,
that might pay some day :-)

Regards,
Martin

From oliphant.travis at ieee.org  Tue Feb 26 21:51:59 2008
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 26 Feb 2008 14:51:59 -0600
Subject: [Python-3000] Python 2.6 and 3.0
In-Reply-To: <47C475EC.1020004@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>	<01a101c87738$da592b90$8f0b82b0$@com.au>	<47C2D841.2030501@cheimes.de>
	<47C33AEA.2010600@v.loewis.de>	<47C3543B.8060303@cheimes.de>
	<47C475EC.1020004@v.loewis.de>
Message-ID: <fq1u5g$d4p$1@ger.gmane.org>

Martin v. L?wis wrote:
>>> No, I still haven't found a solution. I do want to use the merge
>>> module; anything else probably isn't going to work.
>> Da...ng
>> Didn't you prepare a new MSI installer for 3.0a2 that includes the VS
>> Redist MSM for X86? I vaguely remember that you've replaced my installer
>> with a new one.
> 
> Right. I produced a package that ships the CRT, but not by using the
> merge module. Instead, I arranged to include sufficient copies of the
> manifest file to make it work in the non-admin installation (and yes,
> you do need to install multiple copies of it - just see the 3.0a2 MSI
> file).
> 
>> The issue should be resolved before Python 2.6 and 3.0 are reaching beta
>> stage. Maybe we can get some help from outside?
> 
> Perhaps. I'm confident that I can find a solution when I get the time;
> I'm not so confident that I can find the time, though. Of course, I 
> would prefer if the outside help would propose something better than
> "switch to this completely different technology; it may work" (unless
> a complete working solution is presented in that other technology, and
> as long as that other technology still creates MSI files with
> free-as-in-beer tools).
> 
> In any case, contributions are welcome.
>

I've been involved in packaging MSI's at Enthought for a few months now 
and I have been using Martin's msi Tool extensively.  We explored 
several other options (including WiX) before settling back to Martin's 
tool.  It is really a fantastic thing.

I've changed a few things in msilib that I will offer back (as soon as 
I'm happy with the results) --- mainly in the area of File-Table naming 
strategies to avoid conflicts, error reporting, and adding custom 
actions.  We also have a large file schema that we are using (because 
our MSI has over 40,000 files in it).

We are able to package the entire Enthought Python Distribution (which 
includes python 2.5.2 plus about 70 other packages as eggs) into an MSI 
using a modified version of Martin's tool.   If you want to check out 
the beta you can get the 174 MB download here (the official beta 
announcement will be tomorrow so you might want to wait a bit):

http://code.enthought.com/downloads/epd/enthought_python-2.5.2001.msi

My main purpose in posting to this thread is to offer kudos to Martin 
and advertise the utility of the msi-generation tools that he has 
provided.

Thanks for your very useful tool Martin,


-Travis Oliphant


From martin at v.loewis.de  Tue Feb 26 22:19:00 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 26 Feb 2008 22:19:00 +0100
Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0
In-Reply-To: <47C41EA2.4070508@cheimes.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<47C17CD6.1040707@cheimes.de>	<8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>	<47C3592E.2080704@cheimes.de>	<7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org>
	<47C41EA2.4070508@cheimes.de>
Message-ID: <47C48244.4060404@v.loewis.de>

> The Windows builds are hidden in the development section. It took me 10
> minutes to find them because I was searching in the download section and
> for nightly builds. The *daily* builds are available at
> http://www.python.org/dev/daily-msi/

The builds occur 11:00 UTC (2.5), 12:00 UTC (2.6) and 13:00 UTC (3.0).
What that's at day or at night depends on where you live :-)

Regards,
Martin

From christian at cheimes.de  Tue Feb 26 23:42:42 2008
From: christian at cheimes.de (Christian Heimes)
Date: Tue, 26 Feb 2008 23:42:42 +0100
Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0
In-Reply-To: <47C48244.4060404@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<47C17CD6.1040707@cheimes.de>	<8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org>	<47C3592E.2080704@cheimes.de>	<7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org>
	<47C41EA2.4070508@cheimes.de> <47C48244.4060404@v.loewis.de>
Message-ID: <47C495E2.2090705@cheimes.de>

Martin v. L?wis wrote:
>> The Windows builds are hidden in the development section. It took me 10
>> minutes to find them because I was searching in the download section and
>> for nightly builds. The *daily* builds are available at
>> http://www.python.org/dev/daily-msi/
>
> The builds occur 11:00 UTC (2.5), 12:00 UTC (2.6) and 13:00 UTC (3.0).
> What that's at day or at night depends on where you live :-)

Trust me, the day and night cycle of our sun can be unrelated to the day
and night cycle of my life. I define morning as the time span after my
first coffee. I sometimes work until dawn so technically speaking 11:00
UTC could fit my definition of nightly build. :)

Christian

From martin at v.loewis.de  Wed Feb 27 21:56:12 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 27 Feb 2008 21:56:12 +0100
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>	<01a101c87738$da592b90$8f0b82b0$@com.au>	<47C2D841.2030501@cheimes.de>	<47C33AEA.2010600@v.loewis.de>	<47C3543B.8060303@cheimes.de>,
	<47C475EC.1020004@v.loewis.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com>
Message-ID: <47C5CE6C.8020602@v.loewis.de>

>> (unless a complete working solution is presented in that other
>> technology, and as long as that other technology still creates MSI
>> files with free-as-in-beer tools).
> 
> Just out of interest, what's the reason for enforcing that the
> installer must be an MSI?  Or, rather, if I were to present an
> alternative .exe installer that ticks all of the above boxes, exceeds
> the capabilities of the current installer and above all is easier to
> extend and maintain -- would that be a non-starter because it's not
> an MSI?

Not necessarily - it is just very hard to provide the same features
as MSI, but with a different tool. It seems that most tools have given
up the battle against MSI, and now provide just another layer on top
of MSI (just as my msilib does, or WiX).

Regards,
Martin


From tnelson at onresolve.com  Wed Feb 27 21:26:19 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 27 Feb 2008 12:26:19 -0800
Subject: [Python-3000] [Python-Dev]  Python 2.6 and 3.0
In-Reply-To: <47C475EC.1020004@v.loewis.de>
References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org>
	<bbaeab100802221554t55527c27w98084751b233316e@mail.gmail.com>
	<6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org>
	<47C1BE25.9050300@v.loewis.de>	<20080224201613.GL66273@nexus.in-nomine.org>
	<01a101c87738$da592b90$8f0b82b0$@com.au>	<47C2D841.2030501@cheimes.de>
	<47C33AEA.2010600@v.loewis.de>
	<47C3543B.8060303@cheimes.de>,<47C475EC.1020004@v.loewis.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com>

> (unless a complete working solution is presented in that other technology,
> and as long as that other technology still creates MSI files with free-as-in-beer tools).

Just out of interest, what's the reason for enforcing that the installer must be an MSI?  Or, rather, if I were to present an alternative .exe installer that ticks all of the above boxes, exceeds the capabilities of the current installer and above all is easier to extend and maintain -- would that be a non-starter because it's not an MSI?

    Trent.

From lists at cheimes.de  Thu Feb 28 20:49:46 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 28 Feb 2008 20:49:46 +0100
Subject: [Python-3000] Code freeze?
Message-ID: <47C7105A.20908@cheimes.de>

Hey Barry!

When are you planing to freeze the code of the trunk and branches/py3k
for the upcoming alpha releases? I'll merge the last modifications from
2.6 to 3.0 in a couple of minutes. All tests on Linux are looking good,
except for the two profile tests on 3.0. I'm going to test Windows later.

Christian

From arkanes at gmail.com  Thu Feb 28 21:03:51 2008
From: arkanes at gmail.com (Chris Mellon)
Date: Thu, 28 Feb 2008 14:03:51 -0600
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C7105A.20908@cheimes.de>
References: <47C7105A.20908@cheimes.de>
Message-ID: <4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com>

On Thu, Feb 28, 2008 at 1:49 PM, Christian Heimes <lists at cheimes.de> wrote:
> Hey Barry!
>
>  When are you planing to freeze the code of the trunk and branches/py3k
>  for the upcoming alpha releases? I'll merge the last modifications from
>  2.6 to 3.0 in a couple of minutes. All tests on Linux are looking good,
>  except for the two profile tests on 3.0. I'm going to test Windows later.
>

Would a call for third-party module maintainer updates be justified?
pysqlite in particular has advanced a couple versions since 2.5, and
it'd be nice if the latest made it into 2.6.

From barry at python.org  Thu Feb 28 21:51:13 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Feb 2008 15:51:13 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C7105A.20908@cheimes.de>
References: <47C7105A.20908@cheimes.de>
Message-ID: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote:

> Hey Barry!

Hi Christian!

> When are you planing to freeze the code of the trunk and branches/py3k
> for the upcoming alpha releases? I'll merge the last modifications  
> from
> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking  
> good,
> except for the two profile tests on 3.0. I'm going to test Windows  
> later.

Okay, let's go ahead and make it official.

I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern  
(UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to  
that: 2200 UTC Friday 29-Feb-2008.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8cewnEjvBPtnXfVAQJSpQP/aWjpFbTAETdHGnp0IOEagaXNojwJEBBl
llFAE6FQI+WjPxNDAG6Y8T0y4kdiBVubMA7yfp+wXZdn+zpO/4D5OtBVeAoGVjLj
Tg1Ws1Y2uEf7Ah4lRqLya1tfgO+rnKJ38vsCit58XACYVGKWDpD0mVu+An7+6Jmj
rtlEjwGpvFQ=
=jIiD
-----END PGP SIGNATURE-----

From barry at python.org  Thu Feb 28 21:55:26 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Feb 2008 15:55:26 -0500
Subject: [Python-3000] [Python-Dev]  Code freeze?
In-Reply-To: <4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com>
References: <47C7105A.20908@cheimes.de>
	<4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com>
Message-ID: <8C97586F-C049-4574-817C-34EB4D4C0C7A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 28, 2008, at 3:03 PM, Chris Mellon wrote:

> On Thu, Feb 28, 2008 at 1:49 PM, Christian Heimes <lists at cheimes.de>  
> wrote:
>> Hey Barry!
>>
>> When are you planing to freeze the code of the trunk and branches/ 
>> py3k
>> for the upcoming alpha releases? I'll merge the last modifications  
>> from
>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking  
>> good,
>> except for the two profile tests on 3.0. I'm going to test Windows  
>> later.
>>
>
> Would a call for third-party module maintainer updates be justified?
> pysqlite in particular has advanced a couple versions since 2.5, and
> it'd be nice if the latest made it into 2.6.

There's still plenty of time for stuff to make it into 2.6 -- this is  
only an alpha release!  I encourage people to get their changes in now  
if they want them reflected in the next alphas, but I intend to cut  
the release from whatever's in svn at 2300 UTC tomorrow.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8cfvnEjvBPtnXfVAQJKSwP+M5RzAFsQ3fNSwwYzRa+j3EaocwLgWJ+i
QVLxALoH+ha6yWQDmdQJCvm4H+DPBSnz9o/ejuJ3Wuhf9TEIWBLjuwB1rM8qm7mB
blaaNVV9eOXfORjlQ3PvwGblfwmGMGFUKfTuta8BZqkxtEzNKL5tDvgVdx98rbTV
VEyfj7ap3BM=
=dsql
-----END PGP SIGNATURE-----

From lists at cheimes.de  Thu Feb 28 22:15:04 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 28 Feb 2008 22:15:04 +0100
Subject: [Python-3000] Code freeze?
In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
Message-ID: <47C72458.8090306@cheimes.de>

Barry Warsaw wrote:
> Okay, let's go ahead and make it official.
> 
> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern
> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to
> that: 2200 UTC Friday 29-Feb-2008.

Linux is looking good. I've fixed some minor Windows issue in the last
30 minutes. I found one strange behavior. Some tests were failing
because iter(fileobj) where fileobj is a tempfile._TemporaryFileWrapper
failed. Apparently iter() doesn't use __getattr__ to acquire the
__iter__ method. Is this behavior deliberately?

Christian


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-3000/attachments/20080228/948cd274/attachment.pgp 

From eric+python-dev at trueblade.com  Thu Feb 28 22:03:28 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 28 Feb 2008 16:03:28 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
Message-ID: <47C721A0.6060802@trueblade.com>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote:
> 
>> Hey Barry!
> 
> Hi Christian!
> 
>> When are you planing to freeze the code of the trunk and branches/py3k
>> for the upcoming alpha releases? I'll merge the last modifications  
>> from
>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking  
>> good,
>> except for the two profile tests on 3.0. I'm going to test Windows  
>> later.
> 
> Okay, let's go ahead and make it official.
> 
> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern  
> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to  
> that: 2200 UTC Friday 29-Feb-2008.

Argh!  I was going to check the last of the PEP 3127 changes in tonight, 
but I won't make it by 6 pm EST.  I have them finished, but no tests 
written, so I'm not comfortable checking them in yet.  I guess it's no 
big deal that they slip until the next alpha.

Eric.


From stephen at xemacs.org  Thu Feb 28 23:07:04 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 29 Feb 2008 07:07:04 +0900
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
Message-ID: <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern  
 > (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to  
 > that: 2200 UTC Friday 29-Feb-2008.

Is that enough time for the buildbots to do their thing and for you to
look at the page?

Alterntaively, I guess you could just suggest that people check the
buildbot page for their platforms before downloading ....




From barry at python.org  Thu Feb 28 23:07:03 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Feb 2008 17:07:03 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C721A0.6060802@trueblade.com>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<47C721A0.6060802@trueblade.com>
Message-ID: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 28, 2008, at 4:03 PM, Eric Smith wrote:

> Barry Warsaw wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote:
>>> Hey Barry!
>> Hi Christian!
>>> When are you planing to freeze the code of the trunk and branches/ 
>>> py3k
>>> for the upcoming alpha releases? I'll merge the last  
>>> modifications  from
>>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking   
>>> good,
>>> except for the two profile tests on 3.0. I'm going to test  
>>> Windows  later.
>> Okay, let's go ahead and make it official.
>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern   
>> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to   
>> that: 2200 UTC Friday 29-Feb-2008.
>
> Argh!  I was going to check the last of the PEP 3127 changes in  
> tonight, but I won't make it by 6 pm EST.  I have them finished, but  
> no tests written, so I'm not comfortable checking them in yet.  I  
> guess it's no big deal that they slip until the next alpha.

Sorry, I notice my message might not have been clear.  As of this  
writing, you have 23 hours and 54 minute before code freeze :).

Code freeze: 2200 UTC 29-Feb-2008
Alpha making: 2300 UTC 29-Feb-2008

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8cwh3EjvBPtnXfVAQLRSAP/XalWCb1ArFUxoa0OQlA0T7Rw7Si3AFYu
mXRTlU0kqptiYDiHYbPaQwzFwI3xgUBCG5qUnFFq3PCLrxmd6ilYcWAhErSGmxOS
PAWfmrxZElAxqXHviOPQw5dlnoD8JhDudArgn74zRHHluazZYR4r+fCvN8fV0gKc
zRDt8Owdwng=
=uCn6
-----END PGP SIGNATURE-----

From barry at python.org  Thu Feb 28 23:09:45 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Feb 2008 17:09:45 -0500
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 28, 2008, at 5:07 PM, Stephen J. Turnbull wrote:

> Barry Warsaw writes:
>
>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern
>> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to
>> that: 2200 UTC Friday 29-Feb-2008.
>
> Is that enough time for the buildbots to do their thing and for you to
> look at the page?

I don't know, but we'll find out!  The first time (in years) will be a  
bit of a trial-and-error for me.  I can always get dinner in the  
middle of it. :)

> Alterntaively, I guess you could just suggest that people check the
> buildbot page for their platforms before downloading ....

Yes, good idea.  I'm only going to cut source tarballs for the alphas.

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8cxKXEjvBPtnXfVAQLgqwQAiiQUL1lhQM6EmYr8bvZNW2h97ziKGtpf
mVTiNOCJZ1xZ6vKkcmR3Asd+Cwm7B2eT+sVLEpq+Z6tbki+7o0wXC4V6h/ar+zMz
dQcR0QdwM218hioOiIVhtZXbWUTLGndWVviKBcoOdw6qwTbWFspCMV0+FEzdiNUN
V0bDUjn4zkI=
=uPib
-----END PGP SIGNATURE-----

From eric+python-dev at trueblade.com  Fri Feb 29 00:31:45 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 28 Feb 2008 18:31:45 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<47C721A0.6060802@trueblade.com>
	<59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>
Message-ID: <47C74461.20103@trueblade.com>

Barry Warsaw wrote:
>>> Okay, let's go ahead and make it official.
>>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern  
>>> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to  
>>> that: 2200 UTC Friday 29-Feb-2008.
>>
>> Argh!  I was going to check the last of the PEP 3127 changes in 
>> tonight, but I won't make it by 6 pm EST.  I have them finished, but 
>> no tests written, so I'm not comfortable checking them in yet.  I 
>> guess it's no big deal that they slip until the next alpha.
> 
> Sorry, I notice my message might not have been clear.  As of this 
> writing, you have 23 hours and 54 minute before code freeze :).
> 
> Code freeze: 2200 UTC 29-Feb-2008
> Alpha making: 2300 UTC 29-Feb-2008

Your message was clear, it's my reading comprehension that is low.  Now 
you've removed my excuse for not getting this done.  To the keyboard!

Eric.


From ncoghlan at gmail.com  Fri Feb 29 12:39:22 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 29 Feb 2008 21:39:22 +1000
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C72458.8090306@cheimes.de>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<47C72458.8090306@cheimes.de>
Message-ID: <47C7EEEA.3080405@gmail.com>

Christian Heimes wrote:
> Barry Warsaw wrote:
>> Okay, let's go ahead and make it official.
>>
>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern
>> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to
>> that: 2200 UTC Friday 29-Feb-2008.
> 
> Linux is looking good. I've fixed some minor Windows issue in the last
> 30 minutes. I found one strange behavior. Some tests were failing
> because iter(fileobj) where fileobj is a tempfile._TemporaryFileWrapper
> failed. Apparently iter() doesn't use __getattr__ to acquire the
> __iter__ method. Is this behavior deliberately?

The interpreter is actually permitted to bypass the instance when 
looking up magic methods. Whether it does or not is implementation 
dependent - even CPython varies its behaviour depending on the specific 
circumstance (e.g. the translation of with statements to bytecode ends 
up using normal GET_ATTR opcodes, so those will see __enter__ and 
__exit__ methods on instances, but most of the magic method lookups from 
C code bypass the instance and go straight to the relevant tp_* slot).

I'd call this behaviour a bug in _TemporaryFileWrapper, since it's 
delegation trick in __getattr__ isn't guaranteed to work for the magic 
methods. I'm actually becoming less and less enamoured of that shortcut 
every day - given that we've already had to spell out the file method 
and property delegation for SpooledTemporaryFile in 2.6, I'm tempted to 
start spelling it out for _TemporaryFileWrapper as well.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From barry at python.org  Fri Feb 29 13:00:44 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 07:00:44 -0500
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <fq8mbo$cb2$1@ger.gmane.org>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>
	<47C7C90B.5040707@scottdial.com> <fq8mbo$cb2$1@ger.gmane.org>
Message-ID: <44FA87F7-324E-4D2A-9005-820195A4F423@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 5:21 AM, Georg Brandl wrote:

> Scott Dial schrieb:
>> Barry Warsaw wrote:
>>>> Alterntaively, I guess you could just suggest that people check the
>>>> buildbot page for their platforms before downloading ....
>>>
>>> Yes, good idea.  I'm only going to cut source tarballs for the  
>>> alphas.
>>>
>>
>> I apologize for having doubt in your plan, and I can certainly
>> appreciate the work you will be doing as release manager. But..
>>
>> I don't understand who these alpha releases are supposed to be for,  
>> and
>> who they will serve.
>
> For one thing, releases generate "news", meaning that people will be  
> made
> aware that things are moving, that Python is well underway to its next
> major versions, and maybe will be more inclined to look at what's new,
> or check out a release.

I completely agree.

There's another very important aspect of doing alpha releases, and  
that is to debug the /process/ of releasing.  I need to see how far  
we've come in the years since I RM'd last.  How smoothly we can  
coordinate all the various players?  What does it take to update the  
website?  Where are the glitches in the process that slows everything  
down or blocks certain steps?  What is the state of the tools, and is  
there anything else we can automate?

I also want to see if sticking to the monthly cycle can make us all  
work more productively as a community, so that we know when bugs need  
to be fixed in order to show up in the next release, and so on.

Think of it this way: the alphas are for /us/ as much as for our users.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8fz7XEjvBPtnXfVAQI0zgQAnp8nva5qxIoN4ocr7uNXeNxJ+BdP2lMA
8GN1kv3R/+9ny6h/iakfyE0lt1hvHP+uchchd4zo1CWQ390h1W8eR0SpzbubRYFP
Bvdnup7K1zSSGe0ILGoa6Zv8VCYI4vET0PsxM+bkhj8NblwQQu1xUcc6CnYBtapM
c9TwZum/ODg=
=A2cs
-----END PGP SIGNATURE-----

From ncoghlan at gmail.com  Fri Feb 29 13:39:42 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 29 Feb 2008 22:39:42 +1000
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <44FA87F7-324E-4D2A-9005-820195A4F423@python.org>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>	<47C7C90B.5040707@scottdial.com>
	<fq8mbo$cb2$1@ger.gmane.org>
	<44FA87F7-324E-4D2A-9005-820195A4F423@python.org>
Message-ID: <47C7FD0E.3090306@gmail.com>

Barry Warsaw wrote:
> Think of it this way: the alphas are for /us/ as much as for our users.

In that vein, I think the monthly alphas may also help as a means for 
setting personal deadlines for things we're working on. "Implement for 
2.6" is a bit of a wishy-washy deadline at some vague point in the 
future - it doesn't provide much impetus to sit down and spend an 
afternoon or evening getting it done. "Implement for next 2.6 alpha, 
which will be cut in 3 weeks, 2 days and 16 hours" is a real concrete 
target that can be aimed for (even if I'm the only person that knows I'm 
aiming for it for a given bug fix or feature).

It should also motivate at least a monthly review of the buildbot 
status, and is already motivating a push towards making the buildbots a 
more reliable indicator of the health of the source tree.

Sure, they may mostly be internal milestones (with third parties waiting 
for a feature-complete alpha or the first beta), but I think there is a 
decent chance the approach will turn out to be beneficial.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From barry at python.org  Fri Feb 29 13:57:13 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 07:57:13 -0500
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <47C7FD0E.3090306@gmail.com>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>	<47C7C90B.5040707@scottdial.com>
	<fq8mbo$cb2$1@ger.gmane.org>
	<44FA87F7-324E-4D2A-9005-820195A4F423@python.org>
	<47C7FD0E.3090306@gmail.com>
Message-ID: <7FF2D914-8120-487E-A962-D306CA53F888@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 7:39 AM, Nick Coghlan wrote:

> Barry Warsaw wrote:
>> Think of it this way: the alphas are for /us/ as much as for our  
>> users.
>
> In that vein, I think the monthly alphas may also help as a means  
> for setting personal deadlines for things we're working on.  
> "Implement for 2.6" is a bit of a wishy-washy deadline at some vague  
> point in the future - it doesn't provide much impetus to sit down  
> and spend an afternoon or evening getting it done. "Implement for  
> next 2.6 alpha, which will be cut in 3 weeks, 2 days and 16 hours"  
> is a real concrete target that can be aimed for (even if I'm the  
> only person that knows I'm aiming for it for a given bug fix or  
> feature).
>
> It should also motivate at least a monthly review of the buildbot  
> status, and is already motivating a push towards making the  
> buildbots a more reliable indicator of the health of the source tree.
>
> Sure, they may mostly be internal milestones (with third parties  
> waiting for a feature-complete alpha or the first beta), but I think  
> there is a decent chance the approach will turn out to be beneficial.

That's my hope too!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8gBKnEjvBPtnXfVAQK5egP+JgrZd68JtkyhLOGhcd0aBjhQEJM4e2FO
HcRION8EeI4jhg5KQWOt/KmP6CBLblGq50RFI0afsLaJVapk4U6MFTB0LQus3LgX
myd0zK5egu+sblFfKFfe5mMKn5T1Mx5HCLuOE4g3uwuFtdddsGMvYwCyqjm3smq1
T38bbujy234=
=xSv1
-----END PGP SIGNATURE-----

From lists at cheimes.de  Fri Feb 29 14:09:51 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 29 Feb 2008 14:09:51 +0100
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C7EEEA.3080405@gmail.com>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<47C72458.8090306@cheimes.de>
	<47C7EEEA.3080405@gmail.com>
Message-ID: <fq908c$gaq$1@ger.gmane.org>

Nick Coghlan wrote:
> The interpreter is actually permitted to bypass the instance when 
> looking up magic methods. Whether it does or not is implementation 
> dependent - even CPython varies its behaviour depending on the specific 
> circumstance (e.g. the translation of with statements to bytecode ends 
> up using normal GET_ATTR opcodes, so those will see __enter__ and 
> __exit__ methods on instances, but most of the magic method lookups from 
> C code bypass the instance and go straight to the relevant tp_* slot).

I suspected the different behavior is related to C type slots as well.
tp_iter is part of the PyTypeObject struct and the __enter__ and
__exit__ methods are ordinary methods. I just wasn't sure if the
behavior is correct. I've never encountered a problem with the dark
corners of the language so far. ;)

I did some further testing. iter() doesn't even go through
__getattribute__. It grabs the class and does a lookup on the class.

>>> class Example(object):
...     def __getattribute__(self, name):
...         print name
...         return object.__getattribute__(self, name)
>>> e = Example()
>>> iter(e)
__class__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Example' object is not iterable

That's either a bug or at least a big gotcha for developers of proxy
objects. At least I expected that __getattribute__ is called for *every*
attribute including magic methods.

Is this documented somewhere? The docs say "See the
:meth:`__getattribute__` method below for a way to actually get total
control over attribute access.".

Christian


From ncoghlan at gmail.com  Fri Feb 29 15:30:39 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 01 Mar 2008 00:30:39 +1000
Subject: [Python-3000] Code freeze?
In-Reply-To: <fq908c$gaq$1@ger.gmane.org>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<47C72458.8090306@cheimes.de>	<47C7EEEA.3080405@gmail.com>
	<fq908c$gaq$1@ger.gmane.org>
Message-ID: <47C8170F.2050003@gmail.com>

Christian Heimes wrote:
> Is this documented somewhere? The docs say "See the
> :meth:`__getattribute__` method below for a way to actually get total
> control over attribute access.".

I just checked, and the restriction is now documented in the development 
docs in the section on special methods at [1]. It was backported from 
the Py3k docs and isn't part of the docs for 2.5 or earlier versions.

The gist is that special methods *must* be defined directly on a 
new-style class in order for the interpreter to reliably access them, 
but defining them on a new-style instance *may* still affect the 
interpreter's behaviour in some cases (but won't necessarily do so).

In composing this response, I also realised why the new tempfile tests 
worked for me before I checked them in: I was testing it on the trunk, 
where _TemporaryFileWrapper is a classic class. Special method lookup on 
classic classes doesn't include the 'direct-to-type' optimisation, so 
those tests work with the tempfile module as written on 2.x.

Unfortunately, proxies like _TemporaryFileWrapper that rely on 
__getattr__ to provide special methods simply won't work properly when 
implemented as new-style classes - for that, they need to spell out the 
overridden special methods the way SpooledTemporaryFile does.

I think we may need a -3 warning that detects situations where "__*__" 
attributes are retrieved via a __getattr__ implementation on a classic 
class - classes being used that way have a very high chance of breaking 
when ported to 3.0 and I can't see any possible way for 2to3 to detect them.

Cheers,
Nick.

[1]
http://docs.python.org/dev/reference/datamodel.html#special-method-names

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From lists at cheimes.de  Fri Feb 29 15:54:39 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 29 Feb 2008 15:54:39 +0100
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C8170F.2050003@gmail.com>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<47C72458.8090306@cheimes.de>	<47C7EEEA.3080405@gmail.com>	<fq908c$gaq$1@ger.gmane.org>
	<47C8170F.2050003@gmail.com>
Message-ID: <47C81CAF.6070004@cheimes.de>

Nick Coghlan wrote:
> In composing this response, I also realised why the new tempfile tests 
> worked for me before I checked them in: I was testing it on the trunk, 
> where _TemporaryFileWrapper is a classic class. Special method lookup on 
> classic classes doesn't include the 'direct-to-type' optimisation, so 
> those tests work with the tempfile module as written on 2.x.

tempfile is also using different implementation for TemporaryFile on
Win32. On Windows it's an alias for NamedTemporaryFile while on Unix
it's using a file descriptor of an unlinked file.

Christian

From guido at python.org  Fri Feb 29 17:16:14 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Feb 2008 08:16:14 -0800
Subject: [Python-3000] test_profile failing
Message-ID: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>

Am I the only one for whom test_profile and test_cProfile are failing
with a current Py3k head checkout?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Feb 29 17:20:07 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 11:20:07 -0500
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
Message-ID: <EB8CDB10-1B0F-4185-8E73-3270C8EF3A3A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote:

> Am I the only one for whom test_profile and test_cProfile are failing
> with a current Py3k head checkout?

No, they're failing across the board afaict.  I won't have time to  
look at them before tonight, but it would be great if we can fix them  
before I cut the alphas.  I think they're the only failures in the 3.0  
test suite.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8gwuHEjvBPtnXfVAQJclQP/Sz90X6qj0BWxaadmzO4Kr+bqJL2yq4gK
vscxkuGxt6leh/ojMgn8esSoGGZsy0GhCQgfagP93txc7MI36ZiYheZoyRFUwUCG
3Y/W+0E0yArShXhgQ/V8yfIN1tTq7iQxzIiJvHrZ9g2xOMh+j7leSxRk8j+vW7TL
+ye8u8r7dvQ=
=C0ir
-----END PGP SIGNATURE-----

From facundobatista at gmail.com  Fri Feb 29 17:25:20 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 29 Feb 2008 14:25:20 -0200
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
Message-ID: <e04bdf310802290825n187da935qc2ab7b16e264c59d@mail.gmail.com>

2008/2/29, Guido van Rossum <guido at python.org>:

> Am I the only one for whom test_profile and test_cProfile are failing
>  with a current Py3k head checkout?

Both fail for me too.

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From fdrake at acm.org  Fri Feb 29 17:55:46 2008
From: fdrake at acm.org (Fred Drake)
Date: Fri, 29 Feb 2008 11:55:46 -0500
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
Message-ID: <CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>

On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote:
> Am I the only one for whom test_profile and test_cProfile are failing
> with a current Py3k head checkout?


They were failing for me a couple of days ago; checking now, they  
still are, and test_itertools has been added to the list of failing  
tests:

test test_itertools failed -- Traceback (most recent call last):
   File "/Users/fdrake/projects/python/py3k/Lib/test/ 
test_itertools.py", line 698, in test_chain
     self.assertRaises(TypeError, chain, N(s))
AssertionError: TypeError not raised by chain

This is on Mac OS X 10.5.2 with no local changes.  I did a "make  
clean; make; make test" and still get the error.  I've no time to look  
into it right now.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From guido at python.org  Fri Feb 29 18:41:13 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Feb 2008 09:41:13 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
Message-ID: <ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>

On Fri, Feb 29, 2008 at 8:55 AM, Fred Drake <fdrake at acm.org> wrote:
> On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote:
>
> > Am I the only one for whom test_profile and test_cProfile are failing
>  > with a current Py3k head checkout?
>
>
>  They were failing for me a couple of days ago; checking now, they
>  still are, and test_itertools has been added to the list of failing
>  tests:
>
>  test test_itertools failed -- Traceback (most recent call last):
>    File "/Users/fdrake/projects/python/py3k/Lib/test/
>  test_itertools.py", line 698, in test_chain
>      self.assertRaises(TypeError, chain, N(s))
>  AssertionError: TypeError not raised by chain
>
>  This is on Mac OS X 10.5.2 with no local changes.  I did a "make
>  clean; make; make test" and still get the error.  I've no time to look
>  into it right now.

We should not release with failing tests. We've got to keep *some*
standard of quality.

It looks like the failures are shallow -- there's a "golden" expected
output which includes some silliness like the exact number of CPU
seconds used.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Feb 29 18:49:03 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Feb 2008 09:49:03 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
Message-ID: <ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>

On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum <guido at python.org> wrote:
>  It looks like the failures are shallow -- there's a "golden" expected
>  output which includes some silliness like the exact number of CPU
>  seconds used.

Actually I take it back. The same approach works fine in the trunk, so
I suspect it's a merge problem. It may be quite subtle, alas. :-(

Still, we shouldn't release with this failing. It would reduce
confidence in the releases and generate tons of spurious customer
support calls ^W^W^W bug reports.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric+python-dev at trueblade.com  Fri Feb 29 19:08:24 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Fri, 29 Feb 2008 13:08:24 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<47C721A0.6060802@trueblade.com>
	<59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>
Message-ID: <47C84A18.5030602@trueblade.com>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Feb 28, 2008, at 4:03 PM, Eric Smith wrote:
> 
>> Barry Warsaw wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote:
>>>> Hey Barry!
>>> Hi Christian!
>>>> When are you planing to freeze the code of the trunk and branches/py3k
>>>> for the upcoming alpha releases? I'll merge the last modifications  
>>>> from
>>>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking  
>>>> good,
>>>> except for the two profile tests on 3.0. I'm going to test Windows  
>>>> later.
>>> Okay, let's go ahead and make it official.
>>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern  
>>> (UTC-5) time or 2300 UTC.  Let's freeze the tree one hour prior to  
>>> that: 2200 UTC Friday 29-Feb-2008.
>>
>> Argh!  I was going to check the last of the PEP 3127 changes in 
>> tonight, but I won't make it by 6 pm EST.  I have them finished, but 
>> no tests written, so I'm not comfortable checking them in yet.  I 
>> guess it's no big deal that they slip until the next alpha.
> 
> Sorry, I notice my message might not have been clear.  As of this 
> writing, you have 23 hours and 54 minute before code freeze :).
> 
> Code freeze: 2200 UTC 29-Feb-2008
> Alpha making: 2300 UTC 29-Feb-2008

It turns out I'm not going to make this first alpha with the rest of the 
PEP 3127 changes.  The code doesn't handle all combinations of (int, 
long) and (str, unicode).  I'll get it finished before the next alpha.

Eric.

From stephen at xemacs.org  Fri Feb 29 19:59:03 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 01 Mar 2008 03:59:03 +0900
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <47C7C90B.5040707@scottdial.com>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>
	<47C7C90B.5040707@scottdial.com>
Message-ID: <873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp>

Not speaking for Barry or anyone else but myself.  This is an
explanation of how I understand the process and why I welcome it.

Scott Dial writes:

 > I don't understand who these alpha releases are supposed to be for, and 
 > who they will serve.

An alpha test is internal.  It is comprised of those tests the
developers working on the product (and/or an affiliated QA department)
can imagine.

Similarly, an alpha release is of, by, and for the developers.

 > I'm not sure what developer outside of the core community would
 > want to work with something missing key features and released
 > fairly arbitrarily.

Developers outside of the core community are not targeted.  That said,
how do you know that key features are missing?  Part of the point of a
release is that it says "*this stuff* is working".  I want *my* key
feature to be part of "this stuff", not *your* key feature.  Given "my
feature," I download to give the team feedback as early as possible.
Lacking yours, you don't.  Next alpha, presumably our roles will be
reversed.

 > More to the point, I don't know why a developer wouldn't just checkout 
 > from SVN in any case. Certainly if they are going to help root out bugs, 
 > then we would like them to be using the trunk if possible. I fear that 
 > once an alpha is 2 weeks old, we will start saying "please check if its 
 > still a problem on the trunk."

Software development in practice is a matter of "take two steps back,
then three steps forward."  The point of an alpha release is to
checkpoint what is a regression, and what is a temporary "feature"
introduced by new development.  The latter is admissible on the trunk,
but the goal should be none from one alpha to the next.  In other
words, a bug introduced by new development should be fixed by the next
alpha.  If it can't be, the developer misjudged the timing of the
merge to mainline; it wasn't ready yet.

 > A mild concern is how this effects checkins with individuals either 
 > trying to meet a deadline or even wait until after an alpha release to 
 > checkin a large change. I'm not sure this will happen, but having 
 > releases scheduled without feature milestones attached certainly changes 
 > the environment for work to be done in.

Scheduling feature milestones assumes that people put priority on
those features at a given time.  Barry can't assume that yet.  Once
the rhythm is established, Barry can start asking for, and some people
will start volunteering, milestone commitments for given releases.

I think that it probably is desirable to to put that deadline pressure
on.  Individuals who rush to get their work in, and cause alpha-to-
alpha regressions, can be advised to wait in the future in similar
circumstances.  Once the rhythm is established, people can expect that
alphas will be consistently increasing in features and consistently
decreasing in defects.  If that's not true, something's wrong with the
process, and the team needs to step back and do something about it.

But in the nature of software development, *monotone improvement is
not something you want to, or even can, impose on the trunk at all
points in time*.

From barry at python.org  Fri Feb 29 20:22:46 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 14:22:46 -0500
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
Message-ID: <DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 12:49 PM, Guido van Rossum wrote:

> On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum <guido at python.org>  
> wrote:
>> It looks like the failures are shallow -- there's a "golden" expected
>> output which includes some silliness like the exact number of CPU
>> seconds used.
>
> Actually I take it back. The same approach works fine in the trunk, so
> I suspect it's a merge problem. It may be quite subtle, alas. :-(
>
> Still, we shouldn't release with this failing. It would reduce
> confidence in the releases and generate tons of spurious customer
> support calls ^W^W^W bug reports.

Of course, the /real/ bug is that a commit was allowed that broke the  
test suite.  This failure goes back to the first rev on the buildbot  
page 61103 so it'll take a little more investigation to see what broke  
it.

I'll see what can be done to fix this particular problem, and try to  
be more attentive to buildbot failure notices when they happen so we  
can nip it closer to the bud.  Ultimately, it would be nice if we had  
a process that prevented any breaking patch from landing, but we're  
not there yet.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8hbhnEjvBPtnXfVAQIUxQQAhPzrDAdBxbIRqAHSK2ipj3eD27Hvq6FR
vZFDwL7rV/tobvnacPAiZCoTXKg39PXoPZQHhPbheB7mxqMcHRLEb9Kf0GD0qhBL
GKu6p+wgJ7PUOciVlXdykE/E+3+rT0v0MXU3RcvZW7zVRidsLZmQP26OjWgFm0RW
S6VuFw7E2e0=
=kOv3
-----END PGP SIGNATURE-----

From barry at python.org  Fri Feb 29 20:23:26 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 14:23:26 -0500
Subject: [Python-3000] Code freeze?
In-Reply-To: <47C84A18.5030602@trueblade.com>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<47C721A0.6060802@trueblade.com>
	<59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org>
	<47C84A18.5030602@trueblade.com>
Message-ID: <5B7CA66C-E966-4308-9FD6-4AD7A47D0D19@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 1:08 PM, Eric Smith wrote:

> Barry Warsaw wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> On Feb 28, 2008, at 4:03 PM, Eric Smith wrote:
>>> Barry Warsaw wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote:
>>>>> Hey Barry!
>>>> Hi Christian!
>>>>> When are you planing to freeze the code of the trunk and  
>>>>> branches/py3k
>>>>> for the upcoming alpha releases? I'll merge the last  
>>>>> modifications  from
>>>>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are  
>>>>> looking  good,
>>>>> except for the two profile tests on 3.0. I'm going to test  
>>>>> Windows  later.
>>>> Okay, let's go ahead and make it official.
>>>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm  
>>>> Eastern  (UTC-5) time or 2300 UTC.  Let's freeze the tree one  
>>>> hour prior to  that: 2200 UTC Friday 29-Feb-2008.
>>>
>>> Argh!  I was going to check the last of the PEP 3127 changes in  
>>> tonight, but I won't make it by 6 pm EST.  I have them finished,  
>>> but no tests written, so I'm not comfortable checking them in  
>>> yet.  I guess it's no big deal that they slip until the next alpha.
>> Sorry, I notice my message might not have been clear.  As of this  
>> writing, you have 23 hours and 54 minute before code freeze :).
>> Code freeze: 2200 UTC 29-Feb-2008
>> Alpha making: 2300 UTC 29-Feb-2008
>
> It turns out I'm not going to make this first alpha with the rest of  
> the PEP 3127 changes.  The code doesn't handle all combinations of  
> (int, long) and (str, unicode).  I'll get it finished before the  
> next alpha.

No problem Eric.  You know when the next alpha is coming out anyway. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8hbrnEjvBPtnXfVAQKRvgP/Qe5/vCtHPX5LKklLTtcD4pnR/AtRs/V2
/VNeqkvXnr4Ooe8LwNKFMo4Yy0pdvjEQbLR/FGKojG2JIJ0kXyE2ObQXFsKCKAxV
TCr8ZPeh5wqGLYuJAXtTz/UJzf5CzkgbKY9p32u5yy5qXiP7rgGpi8JvSZJKN3cP
bApx/pGyNdU=
=GaSZ
-----END PGP SIGNATURE-----

From barry at python.org  Fri Feb 29 20:33:34 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 14:33:34 -0500
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <47C7105A.20908@cheimes.de>
	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>
	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>
	<47C7C90B.5040707@scottdial.com>
	<873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <079EC957-A3D3-48F9-8A77-A8BEB402E749@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 1:59 PM, Stephen J. Turnbull wrote:

> I think that it probably is desirable to to put that deadline pressure
> on.  Individuals who rush to get their work in, and cause alpha-to-
> alpha regressions, can be advised to wait in the future in similar
> circumstances.  Once the rhythm is established, people can expect that
> alphas will be consistently increasing in features and consistently
> decreasing in defects.  If that's not true, something's wrong with the
> process, and the team needs to step back and do something about it.

I agree, and what I already think is broken about our process is that  
changes can land that break the buildbots.  Ideally, this should never  
happen, but our build/test environment has two flaws.  First, it's  
retroactive not proactive.  Second, the tests themselves are not  
always stable.

Given the wide range of platforms and the volunteer nature of the  
buildbot farm, I don't think there's much right now that we can do  
about the second point.  We can do things to mitigate it though, such  
as take a majority-success approach, or give more weight to more  
stable platforms and buildbots.

The second one is tougher because more work on the process is  
necessary, and it would change our workflow for committing changes to  
the tree.  Even with its faults, I'm a big fan of PQM <https://launchpad.net/pqm 
 > though that's not something I think we're ready for.  The bigger  
question though is whether we as a development community would change  
the way we work so that nothing lands if it doesn't pass all the  
tests.  We'd be trading some inconvenience (and administrative  
headaches) for better overall quality and always-releasable guarantees.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8heD3EjvBPtnXfVAQIj/gP/dyf1oavy7y4gTrKHi+j/m+0y4DJstIDf
Fh2MhqExWtCX6V8M3mOn46uRwStwtz9+TUEznNEC8xsxq734GtVyi9Vw6cLpmZQ6
uQp+IBT+nkqNz3sDd8N/ewAGPBO5Ml2m+yn+rfi2XaT5Vfi5akSR/aDwJKGC71fL
8IaWHo5XKEM=
=JQMb
-----END PGP SIGNATURE-----

From brett at python.org  Fri Feb 29 20:44:05 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Feb 2008 11:44:05 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
Message-ID: <bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>

On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>
> On Feb 29, 2008, at 12:49 PM, Guido van Rossum wrote:
>
>  > On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum <guido at python.org>
>  > wrote:
>  >> It looks like the failures are shallow -- there's a "golden" expected
>  >> output which includes some silliness like the exact number of CPU
>  >> seconds used.
>  >
>  > Actually I take it back. The same approach works fine in the trunk, so
>  > I suspect it's a merge problem. It may be quite subtle, alas. :-(
>  >
>  > Still, we shouldn't release with this failing. It would reduce
>  > confidence in the releases and generate tons of spurious customer
>  > support calls ^W^W^W bug reports.
>
>  Of course, the /real/ bug is that a commit was allowed that broke the
>  test suite.  This failure goes back to the first rev on the buildbot
>  page 61103 so it'll take a little more investigation to see what broke
>  it.
>
>  I'll see what can be done to fix this particular problem, and try to
>  be more attentive to buildbot failure notices when they happen so we
>  can nip it closer to the bud.  Ultimately, it would be nice if we had
>  a process that prevented any breaking patch from landing, but we're
>  not there yet.

A quick tweak to test_profile to print out what it got for output
shows that some changes to io.py led to a different tracing.

-Brett

From barry at python.org  Fri Feb 29 20:57:50 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Feb 2008 14:57:50 -0500
Subject: [Python-3000] test_profile failing
In-Reply-To: <bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
	<bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
Message-ID: <EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote:

> On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org>  
> wrote:
>>
>> I'll see what can be done to fix this particular problem, and try to
>> be more attentive to buildbot failure notices when they happen so we
>> can nip it closer to the bud.  Ultimately, it would be nice if we had
>> a process that prevented any breaking patch from landing, but we're
>> not there yet.
>
> A quick tweak to test_profile to print out what it got for output
> shows that some changes to io.py led to a different tracing.

Thanks Brett.  Do we just need to regen the expected output then?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR8hjvnEjvBPtnXfVAQJNPgP/WSq1x89laWuptso3MNYdtmwnY7L9pZbN
0IoFlqSbPQ/iB42lbE6apAdBNwUjPBAX+IpwFMKEtbDEQzBK5GZ/smksOxyw152E
yMwJaLCnsAf/SS57cxxRv2s6bfB7XF/A1XeodtqlV3O+5seV/zHvX+5IycQ0HJ0R
Ap4uOwC8+II=
=XpFM
-----END PGP SIGNATURE-----

From brett at python.org  Fri Feb 29 21:01:26 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Feb 2008 12:01:26 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
	<bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
	<EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
Message-ID: <bbaeab100802291201u57d40bbat4621afde41e772af@mail.gmail.com>

On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>
> On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote:
>
>  > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org>
>  > wrote:
>  >>
>
> >> I'll see what can be done to fix this particular problem, and try to
>  >> be more attentive to buildbot failure notices when they happen so we
>  >> can nip it closer to the bud.  Ultimately, it would be nice if we had
>  >> a process that prevented any breaking patch from landing, but we're
>  >> not there yet.
>  >
>  > A quick tweak to test_profile to print out what it got for output
>  > shows that some changes to io.py led to a different tracing.
>
>  Thanks Brett.  Do we just need to regen the expected output then?

Nope, it's still failing. I think there might be a slight issue with
how it is generating. I am going to work on this for another half hour
before I need to take off for an appointment. I will send another
update before I take off.

-Brett

From brett at python.org  Fri Feb 29 21:39:38 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Feb 2008 12:39:38 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
	<bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
	<EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
Message-ID: <bbaeab100802291239v3642e465y3c3df42486ac890a@mail.gmail.com>

On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>
> On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote:
>
>  > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org>
>  > wrote:
>  >>
>
> >> I'll see what can be done to fix this particular problem, and try to
>  >> be more attentive to buildbot failure notices when they happen so we
>  >> can nip it closer to the bud.  Ultimately, it would be nice if we had
>  >> a process that prevented any breaking patch from landing, but we're
>  >> not there yet.
>  >
>  > A quick tweak to test_profile to print out what it got for output
>  > shows that some changes to io.py led to a different tracing.
>
>  Thanks Brett.  Do we just need to regen the expected output then?

Ah, the joys of expected output. =)

So here is the problem, and it is probably going to mean test_profile
needs to be rewritten...

When you run ``./python Lib/test/test_profile.py -r`` it runs the
profile and rewrites the file with the golden output. That's great and
all, but there was a slight difference with that golden output and
what was being generated by ``./python Lib/test/test_profile.py``. You
know what that difference was? An extra write call in io.py.

Turns out that when the golden output is generated that all streams
have their buffers empty. But when unittest prints its whole "__test__
.. ok" output it has stuff in a buffer since the "ok" has not been
written yet (in other words sys.stdout has not been fully flushed yet
while the test is running). If you change the print() call in
test_profile.regenerate_expected_output() to ``print('Regenerating %s
... ' % filename, end='')`` then sys.stdout gets a flush() call during
the profiling for the golden output and everyone is happy.

But guess what happens when you run ``./python Lib/test/regrtest.py
test_profile``? That's right, there is no extra flush call!
regrtest.py outputs a single line, runs the test, and then prints
another line when it knows the result of the test.. Actually, there is
one less flush call in the regrtest.py case than in the generated
stuff, so I suspect regrtest or someone is doing an explicit flush or
something.

In other words, having the golden output have any I/O in it is making
it impossible to test. What would probably be best is to just strip
out any and all lines that do not directly involve profilee.py and
only compare those lines. I have no clue if a simple 'contains' check
will suffice because I don't know if the stat numbers are consistent
enough.

Anyway, I am out of time. Hopefully someone can finish up this work
some time today.

-Brett

From guido at python.org  Fri Feb 29 22:15:32 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 29 Feb 2008 13:15:32 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <bbaeab100802291239v3642e465y3c3df42486ac890a@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
	<bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
	<EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
	<bbaeab100802291239v3642e465y3c3df42486ac890a@mail.gmail.com>
Message-ID: <ca471dc20802291315o9beed6j1c9575bc47da4e9b@mail.gmail.com>

On Fri, Feb 29, 2008 at 12:39 PM, Brett Cannon <brett at python.org> wrote:
> On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw <barry at python.org> wrote:
>  > -----BEGIN PGP SIGNED MESSAGE-----
>  >  Hash: SHA1
>  >
>  >
>  > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote:
>  >
>  >  > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org>
>  >  > wrote:
>  >  >>
>  >
>  > >> I'll see what can be done to fix this particular problem, and try to
>  >  >> be more attentive to buildbot failure notices when they happen so we
>  >  >> can nip it closer to the bud.  Ultimately, it would be nice if we had
>  >  >> a process that prevented any breaking patch from landing, but we're
>  >  >> not there yet.
>  >  >
>  >  > A quick tweak to test_profile to print out what it got for output
>  >  > shows that some changes to io.py led to a different tracing.
>  >
>  >  Thanks Brett.  Do we just need to regen the expected output then?
>
>  Ah, the joys of expected output. =)
>
>  So here is the problem, and it is probably going to mean test_profile
>  needs to be rewritten...
>
>  When you run ``./python Lib/test/test_profile.py -r`` it runs the
>  profile and rewrites the file with the golden output. That's great and
>  all, but there was a slight difference with that golden output and
>  what was being generated by ``./python Lib/test/test_profile.py``. You
>  know what that difference was? An extra write call in io.py.
>
>  Turns out that when the golden output is generated that all streams
>  have their buffers empty. But when unittest prints its whole "__test__
>  .. ok" output it has stuff in a buffer since the "ok" has not been
>  written yet (in other words sys.stdout has not been fully flushed yet
>  while the test is running). If you change the print() call in
>  test_profile.regenerate_expected_output() to ``print('Regenerating %s
>  ... ' % filename, end='')`` then sys.stdout gets a flush() call during
>  the profiling for the golden output and everyone is happy.
>
>  But guess what happens when you run ``./python Lib/test/regrtest.py
>  test_profile``? That's right, there is no extra flush call!
>  regrtest.py outputs a single line, runs the test, and then prints
>  another line when it knows the result of the test.. Actually, there is
>  one less flush call in the regrtest.py case than in the generated
>  stuff, so I suspect regrtest or someone is doing an explicit flush or
>  something.
>
>  In other words, having the golden output have any I/O in it is making
>  it impossible to test. What would probably be best is to just strip
>  out any and all lines that do not directly involve profilee.py and
>  only compare those lines. I have no clue if a simple 'contains' check
>  will suffice because I don't know if the stat numbers are consistent
>  enough.
>
>  Anyway, I am out of time. Hopefully someone can finish up this work
>  some time today.

Good analysis. Can't we insert a flush somewhere before the profiler
is started so that the starting point is always with empty output
buffers?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Fri Feb 29 23:38:38 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 29 Feb 2008 14:38:38 -0800
Subject: [Python-3000] test_profile failing
In-Reply-To: <ca471dc20802291315o9beed6j1c9575bc47da4e9b@mail.gmail.com>
References: <ca471dc20802290816l125bd069y17e300542c17c7e6@mail.gmail.com>
	<CFDE3ABF-4F02-441C-BC86-5CA39E095319@acm.org>
	<ca471dc20802290941s25f8081ewbd50f50dee93a1fa@mail.gmail.com>
	<ca471dc20802290949l1327c5e1r598fe95ddce15696@mail.gmail.com>
	<DD5AE829-48CB-4CB2-8950-CCC1C62CADAF@python.org>
	<bbaeab100802291144v1a9c74f1g1d5e502cd15b2721@mail.gmail.com>
	<EE9C9CE8-739B-465C-9EE1-FE5BC8901442@python.org>
	<bbaeab100802291239v3642e465y3c3df42486ac890a@mail.gmail.com>
	<ca471dc20802291315o9beed6j1c9575bc47da4e9b@mail.gmail.com>
Message-ID: <bbaeab100802291438l1e99d864ke1888d4f19504bb1@mail.gmail.com>

On Fri, Feb 29, 2008 at 1:15 PM, Guido van Rossum <guido at python.org> wrote:
>
> On Fri, Feb 29, 2008 at 12:39 PM, Brett Cannon <brett at python.org> wrote:
>  > On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw <barry at python.org> wrote:
>  >  > -----BEGIN PGP SIGNED MESSAGE-----
>  >  >  Hash: SHA1
>  >  >
>  >  >
>  >  > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote:
>  >  >
>  >  >  > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw <barry at python.org>
>  >  >  > wrote:
>  >  >  >>
>  >  >
>  >  > >> I'll see what can be done to fix this particular problem, and try to
>  >  >  >> be more attentive to buildbot failure notices when they happen so we
>  >  >  >> can nip it closer to the bud.  Ultimately, it would be nice if we had
>  >  >  >> a process that prevented any breaking patch from landing, but we're
>  >  >  >> not there yet.
>  >  >  >
>  >  >  > A quick tweak to test_profile to print out what it got for output
>  >  >  > shows that some changes to io.py led to a different tracing.
>  >  >
>  >  >  Thanks Brett.  Do we just need to regen the expected output then?
>  >
>  >  Ah, the joys of expected output. =)
>  >
>  >  So here is the problem, and it is probably going to mean test_profile
>  >  needs to be rewritten...
>  >
>  >  When you run ``./python Lib/test/test_profile.py -r`` it runs the
>  >  profile and rewrites the file with the golden output. That's great and
>  >  all, but there was a slight difference with that golden output and
>  >  what was being generated by ``./python Lib/test/test_profile.py``. You
>  >  know what that difference was? An extra write call in io.py.
>  >
>  >  Turns out that when the golden output is generated that all streams
>  >  have their buffers empty. But when unittest prints its whole "__test__
>  >  .. ok" output it has stuff in a buffer since the "ok" has not been
>  >  written yet (in other words sys.stdout has not been fully flushed yet
>  >  while the test is running). If you change the print() call in
>  >  test_profile.regenerate_expected_output() to ``print('Regenerating %s
>  >  ... ' % filename, end='')`` then sys.stdout gets a flush() call during
>  >  the profiling for the golden output and everyone is happy.
>  >
>  >  But guess what happens when you run ``./python Lib/test/regrtest.py
>  >  test_profile``? That's right, there is no extra flush call!
>  >  regrtest.py outputs a single line, runs the test, and then prints
>  >  another line when it knows the result of the test.. Actually, there is
>  >  one less flush call in the regrtest.py case than in the generated
>  >  stuff, so I suspect regrtest or someone is doing an explicit flush or
>  >  something.
>  >
>  >  In other words, having the golden output have any I/O in it is making
>  >  it impossible to test. What would probably be best is to just strip
>  >  out any and all lines that do not directly involve profilee.py and
>  >  only compare those lines. I have no clue if a simple 'contains' check
>  >  will suffice because I don't know if the stat numbers are consistent
>  >  enough.
>  >
>  >  Anyway, I am out of time. Hopefully someone can finish up this work
>  >  some time today.
>
>  Good analysis. Can't we insert a flush somewhere before the profiler
>  is started so that the starting point is always with empty output
>  buffers?

Flushing both stdout and stderr just before the logging call still
doesn't fix it for running the test through regrtest. There is an
extra call to 'exec' (not sure if it is just a method on some object
named that or the actual built-in) triggering a call to IOBase.flush()
that is only being triggered when the test is run under regrtest.
Looking in regrtest the only exec() call is when you hunt for
refleaks, not general testing so I am not sure where it is coming
from.

-Brett

From scott+python-dev at scottdial.com  Fri Feb 29 09:57:47 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 29 Feb 2008 03:57:47 -0500
Subject: [Python-3000] [Python-Dev] Code freeze?
In-Reply-To: <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>
References: <47C7105A.20908@cheimes.de>	<00E337B6-FFC8-43C8-BA9F-142725954320@python.org>	<87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org>
Message-ID: <47C7C90B.5040707@scottdial.com>

Barry Warsaw wrote:
>> Alterntaively, I guess you could just suggest that people check the
>> buildbot page for their platforms before downloading ....
> 
> Yes, good idea.  I'm only going to cut source tarballs for the alphas.
> 

I apologize for having doubt in your plan, and I can certainly 
appreciate the work you will be doing as release manager. But..

I don't understand who these alpha releases are supposed to be for, and 
who they will serve. When I first saw this plan, I thought "ok, you're 
the man.." But now I wonder even more if they are only going to be 
source tarballs. Who is the intended audience of these alphas? It seems 
like cutting only source tarballs is targeting developers, and if that 
is the case, I wonder if you have misplaced your motivations.

I'm not sure what developer outside of the core community would want to 
work with something missing key features and released fairly 
arbitrarily. Ok, I grant you that it's a monthly cycle, but there are no 
feature milestones involved, so it's fairly arbitrary from a utility 
standpoint; it's not clear to me that we are even worrying about all of 
the buildbots being green before releasing.

More to the point, I don't know why a developer wouldn't just checkout 
from SVN in any case. Certainly if they are going to help root out bugs, 
then we would like them to be using the trunk if possible. I fear that 
once an alpha is 2 weeks old, we will start saying "please check if its 
still a problem on the trunk."

A mild concern is how this effects checkins with individuals either 
trying to meet a deadline or even wait until after an alpha release to 
checkin a large change. I'm not sure this will happen, but having 
releases scheduled without feature milestones attached certainly changes 
the environment for work to be done in.

I guess I am hoping to understand better how these alphas serve us. 
Again, I apologize if that sounded judgmental, but I wanted to make sure 
that releasing alphas was the best plan. And again, I appreciate your 
enthusiasm and effort.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu